Using Ml Production

Using ML Production

Overview

This meta-skill routes you to the right production deployment skill based on your concern. Load this when you need to move ML models to production but aren't sure which specific aspect to address.

Core Principle: Production concerns fall into four categories. Identify the concern first, then route to the appropriate skill. Tools and infrastructure choices are implementation details, not routing criteria.

Capability tiers, not model IDs. Sheets describe model and hardware capabilities (e.g., "frontier reasoning model", "mid-tier serving GPU", "edge accelerator") rather than naming specific SKUs. Vendor model names and exact GPU SKUs change quarterly; capability tiers are stable.

Knowledge-cutoff acknowledgement. Tool inventory is calibrated to 2026-05. Fast-moving areas — LLM serving stacks (vLLM, SGLang, TensorRT-LLM, TGI, Triton), observability platforms (Phoenix, Langfuse, OTel GenAI), quantization toolchains (torch.ao.quantization, AWQ, GPTQ), and MLOps platforms — should be re-checked against vendor docs before architecting new systems. Treat sheet content as a structured starting point, not a substitute for current documentation.

When to Use

Load this skill when:

• Deploying ML models (classical or LLM) to production
• Optimizing model inference (speed, size, cost)
• Setting up MLOps workflows (tracking, automation, CI/CD)
• Monitoring or debugging production models, including LLM-specific signals (hallucination rate, tool-call success, prompt-injection attempts)
• Curating, documenting, releasing, or handing over a dataset — including label quality, deduplication, synthetic-data provenance, and eval-set construction
• User mentions: "production", "deploy", "serve model", "MLOps", "monitoring", "optimize inference", "vLLM", "SGLang", "TensorRT-LLM", "Phoenix", "Langfuse", "dataset", "data card", "Croissant", "labelling", "annotation", "synthetic data", "eval set"

Don't use for: Training optimization (use training-optimization), model architecture selection (use neural-architectures), PyTorch infrastructure (use pytorch-engineering), prompt/RAG/agent design quality (use llm-specialist).

How to Access Reference Sheets

IMPORTANT: All reference sheets are located in the SAME DIRECTORY as this SKILL.md file.

When this skill is loaded from: skills/using-ml-production/SKILL.md

Reference sheets like quantization-for-inference.md are at: skills/using-ml-production/quantization-for-inference.md

NOT at: skills/quantization-for-inference.md ← WRONG PATH

When you see a link like [quantization-for-inference.md](quantization-for-inference.md), read the file from the same directory as this SKILL.md.

Routing by Concern

Category 1: Model Optimization

Symptoms: "Model too slow", "inference latency high", "model too large", "need to optimize for edge", "reduce model size", "speed up inference", "quantize model", "INT8/INT4", "AWQ", "GPTQ"

When to route here:

• Model itself is the bottleneck (not infrastructure)
• Need to reduce model size or increase inference speed
• Deploying to resource-constrained hardware (edge, mobile)
• Cost optimization through model efficiency
• Quantization operations (the mechanics of converting weights, calibration, kernel fit)

Routes to:

• quantization-for-inference.md - torch.ao.quantization, AWQ, GPTQ, INT8/INT4, post-training quantization, QAT, calibration
• model-compression-techniques.md - Pruning, distillation, architecture optimization
• hardware-optimization-strategies.md - GPU/CPU/edge tuning, batch sizing, hardware/quant-format fit

Key question to ask: "Is the MODEL the bottleneck, or is it infrastructure/serving?"

LLM quantization split: Choosing which quantization format suits a given LLM task (quality vs throughput tradeoffs, instruction-following degradation) is llm-specialist territory. The operations — wiring up AWQ/GPTQ, running calibration, validating kernel support — live here.

Category 2: Serving Infrastructure

Symptoms: "How to serve model", "need API endpoint", "deploy to production", "containerize model", "scale serving", "load balancing", "traffic management", "vLLM", "SGLang", "TensorRT-LLM", "TGI", "Triton", "continuous batching", "PagedAttention", "speculative decoding", "KV-cache management"

When to route here:

• Need to expose model as API or service (classical model or LLM)
• Questions about serving patterns (REST, gRPC, batch, streaming)
• Deployment strategies (gradual rollout, A/B testing)
• Scaling concerns (traffic, replicas, autoscaling)
• LLM-serving stacks: vLLM, SGLang, TensorRT-LLM, TGI, Triton — covered in the LLM-serving Part of model-serving-patterns.md alongside continuous batching, PagedAttention, and speculative decoding ops

Routes to:

• model-serving-patterns.md - FastAPI, TorchServe (maintenance mode — see routing mistakes), gRPC, ONNX, batching, containerization, plus LLM-serving Part: vLLM / SGLang / TensorRT-LLM / TGI / Triton, continuous batching, PagedAttention, speculative decoding
• deployment-strategies.md - A/B testing, canary, shadow mode, rollback procedures
• scaling-and-load-balancing.md - Horizontal scaling, autoscaling, load balancing, cost optimization

Key distinction:

• Serving patterns = HOW to expose model (API, container, batching, LLM-serving stack)
• Deployment strategies = HOW to roll out safely (gradual, testing, rollback)
• Scaling = HOW to handle traffic (replicas, autoscaling, balancing)

Category 3: MLOps & Data

Symptoms: "Track experiments", "version models", "automate deployment", "reproducibility", "CI/CD for ML", "feature store", "model registry", "experiment management", "version prompts", "version RAG eval set", "dataset release", "data card", "datasheet", "Croissant", "label quality", "inter-annotator agreement", "annotation", "deduplicate training data", "synthetic data", "eval-set contamination", "curate dataset", "dataset handover"

When to route here:

• Need workflow/process improvements
• Want to track experiments, version models, version prompts, or version RAG eval sets
• Need to automate training-to-deployment pipeline
• Team collaboration and reproducibility concerns
• Need to release, document, hand over, or share a dataset as a production artifact
• Label quality, deduplication, synthetic-data provenance, or eval-set construction is the concern

Routes to:

• experiment-tracking-and-versioning.md - MLflow, Weights & Biases, Comet, Hugging Face Hub, model registries, prompt versioning, RAG/eval-set versioning, lineage
• mlops-pipeline-automation.md - CI/CD for ML, feature stores, data validation, automated retraining, orchestration
• dataset-curation-and-quality.md - Dataset release contract, Croissant / Datasheets / Data Cards, coverage and slices, dedup (MinHash/LSH, SemDeDup), label quality (Krippendorff's α, confident learning, annotation platforms), synthetic-data provenance and ratio discipline, eval-set construction and freeze/rotate policy

Key distinction:

• Experiment tracking = Research/development phase (track runs, version models, version prompts/eval sets)
• Pipeline automation = Production phase (automate workflows, CI/CD)
• Dataset curation = The data artifact itself (what's in it, is it right, can you defend and hand it over)

Data sheet split — three-way, and it matters:

• What to version and why it's a release → dataset-curation-and-quality.md
• How to version it (DVC, lakeFS, hashing, registries, lineage) → experiment-tracking-and-versioning.md
• How to enforce it in CI (Great Expectations, Pandera, eval gates) → mlops-pipeline-automation.md

Schema validation proves the data is well-formed; the curation sheet covers whether it is right — duplication, label error, missing slices, contamination. Route to both when the query is "our data quality is bad" without further detail.

Multi-concern: Queries like "track experiments AND automate deployment" → route to BOTH skills

Category 4: Observability

Symptoms: "Monitor production", "model degrading", "detect drift", "production debugging", "alert on failures", "model not working in prod", "performance issues in production", "LLM in production", "hallucination rate", "tool-call success", "cost-per-request spiking", "prompt-injection attempts", "drift in RAG retrieval quality"

When to route here:

• Model already deployed, need to monitor or debug
• Detecting production issues (drift, errors, degradation, accuracy regressions)
• Setting up alerts and dashboards
• Root cause analysis for production failures
• LLM-specific telemetry: trace generations, score outputs, track cost/token spend, capture tool-call traces, surface prompt-injection signals

Routes to:

• production-monitoring-and-alerting.md - Metrics, drift detection, dashboards, alerts, SLAs, LLM observability platforms: Phoenix, Langfuse, OpenTelemetry GenAI semantic conventions for trace/eval/cost monitoring
• production-debugging-techniques.md - Error analysis, profiling, rollback procedures, post-mortems, tool-call failure forensics

Key distinction:

• Monitoring = Proactive (set up metrics, alerts, detect issues early)
• Debugging = Reactive (diagnose and fix existing issues)

LLM observability split: The plumbing — Phoenix/Langfuse/OTel-GenAI deployment, retention, dashboards, alerts on cost/latency/error budgets — lives here. The eval methodology feeding those dashboards (which metrics to compute, how to score hallucinations, how to build judges) lives in llm-specialist (llm-evaluation-metrics.md). Wire them together: methodology defines the signal; this pack ships it.

"Performance" ambiguity:

• If "performance" = speed/latency → might be Category 1 (optimization) or Category 2 (serving/scaling)
• If "performance" = accuracy degradation → Category 4 (observability - drift detection)
• If "performance" = LLM output quality → Category 4 (observability — wire Phoenix/Langfuse) and llm-specialist (eval methodology)
• Ask clarifying question: "By performance, do you mean inference speed, model accuracy, or LLM output quality?"

Routing Decision Tree

User query → Identify primary concern

Is model THE problem (size/speed)?
  YES → Category 1: Model Optimization
  NO → Continue

Is it about HOW to expose/deploy model (incl. LLM-serving stack)?
  YES → Category 2: Serving Infrastructure
  NO → Continue

Is it about workflow/process/automation?
  YES → Category 3: MLOps & Data
  NO → Continue

Is it about the DATASET itself (what's in it, label quality, dedup, synthetic data,
eval-set construction, releasing or handing it over)?
  YES → Category 3: dataset-curation-and-quality.md
  NO → Continue

Is it about monitoring/debugging in production (incl. LLM telemetry)?
  YES → Category 4: Observability
  NO → Ask clarifying question

Is the question about LLM generation quality, prompt design, RAG retrieval design, or agent behavior design?
  YES → Hand off to llm-specialist (this pack does ops, not generation quality)

Ambiguous? → Ask ONE question to clarify concern category

Clarification Questions for Ambiguous Queries

Query: "My model is too slow"

Ask: "Is this inference latency (how fast predictions are), or training time?"

• Training → Route to training-optimization (wrong pack)
• Inference → Follow-up: "Have you profiled to find bottlenecks?"
  • Model is bottleneck → Category 1 (optimization)
  • Infrastructure/batching issue → Category 2 (serving)
  • LLM-specific (KV cache, batching strategy) → Category 2 LLM-serving Part + llm-specialist llm-inference-optimization.md

Query: "I need to deploy my model"

Ask: "What's your deployment target — cloud server, edge device, batch processing, or LLM endpoint?"

• Cloud/server → Category 2 (serving-patterns, then maybe deployment-strategies if gradual rollout needed)
• Edge/mobile → Category 1 (optimization first for size/speed) + Category 2 (serving)
• Batch → Category 2 (serving-patterns - batch processing)
• LLM endpoint → Category 2 LLM-serving Part of model-serving-patterns.md + llm-specialist (llm-inference-optimization.md for capability/strategy choice)

Query: "My model isn't performing well in production"

Ask: "By performance, do you mean inference speed, prediction accuracy, or LLM output quality (hallucination, tool-call failure)?"

• Speed → Category 1 (optimization) or Category 2 (serving/scaling)
• Accuracy → Category 4 (observability — drift detection, monitoring)
• LLM output quality → Category 4 (Phoenix/Langfuse plumbing) + llm-specialist (llm-evaluation-metrics.md for methodology, prompt-engineering-patterns.md / rag-architecture-patterns.md for fixes)

Query: "Set up MLOps for my team"

Ask: "What's the current pain point — experiment tracking, automated deployment, prompt/eval-set versioning, or some combination?"

• Tracking/versioning (incl. prompt/eval-set) → Category 3 (experiment-tracking-and-versioning)
• Automation/CI/CD → Category 3 (mlops-pipeline-automation)
• Dataset quality/curation/release → Category 3 (dataset-curation-and-quality)
• Multiple → Route to multiple skills

Multi-Concern Scenarios

Some queries span multiple categories. Route to ALL relevant skills in logical order:

Principle: Route in execution order (what needs to happen first).

Relationship with Other Packs

With llm-specialist

The two packs split along the ops vs. generation-quality boundary.

ml-production owns (this pack):

• General serving stacks: vLLM, SGLang, TensorRT-LLM, TGI, Triton — deployment, batching ops, KV-cache config, scaling
• Quantization for inference as ops: torch.ao.quantization, AWQ, GPTQ wiring, calibration, hardware fit
• MLOps: CI/CD, registries, feature stores, retraining
• Deployment patterns: containers, canary, A/B, rollback
• Monitoring & observability platforms: Phoenix, Langfuse, OpenTelemetry GenAI semantic conventions, dashboards, alerts, drift detection

llm-specialist owns (sister pack — 10 reference sheets):

• Prompt engineering (prompt-engineering-patterns.md)
• Reasoning-model use (reasoning-models.md)
• Agentic patterns + MCP (agentic-patterns-and-mcp.md)
• RAG architecture (rag-architecture-patterns.md)
• Fine-tuning strategy choice (llm-finetuning-strategies.md)
• Context engineering & prompt caching (context-engineering-and-prompt-caching.md, context-window-management.md)
• Evaluation methodology (llm-evaluation-metrics.md)
• Inference-strategy choice (llm-inference-optimization.md)
• Safety / alignment (llm-safety-alignment.md)

Concrete bidirectional triggers:

Rule of thumb: If the question is "what should the model/prompt/agent do and how should it be designed?" → llm-specialist. If the question is "how do we run, observe, and operate it at scale?" → ml-production. Most LLM production questions need both.

With training-optimization

Clear boundary:

• training-optimization = Training phase (convergence, hyperparameters, training speed)
• ml-production = Inference phase (deployment, serving, monitoring) plus the dataset as a production artifact

The dataset is not an exception to the inference-phase framing. dataset-curation-and-quality.md treats the dataset as a versioned production artifact with a release contract and quality gates — composition, provenance, documentation, handover, and the drift→re-collection loop. That is operational discipline, and it belongs here. What belongs to training-optimization is what you do with data during a training run (curriculum, sampling schedules, augmentation for convergence), not what the dataset is.

"Too slow" disambiguation:

• Training slow → training-optimization
• Inference slow → ml-production

Bidirectional: training-optimization should send users here when they're done training; ml-production should send users back when they need to retrain (drift response, dataset updates).

With data-engineering concerns

Pipeline and warehouse mechanics — batch/streaming pipelines, ELT/ETL, dbt, dimensional and lakehouse modelling, orchestration as a data-platform concern, CDC — are not in this pack. dataset-curation-and-quality.md is deliberately scoped to the ML training/eval corpus as a model-facing artifact: what is in it, whether it is right, and how it is released. If the question is "how do we build and model the warehouse this data comes from," that is a data-engineering question, not an ml-production one.

With pytorch-engineering

pytorch-engineering covers: Foundation (distributed training, profiling, memory management)

ml-production covers: Production-specific (serving APIs, deployment patterns, MLOps, production observability)

When to use both:

• "Profile production inference" → pytorch-engineering (profiling techniques) + ml-production (production context, observability wiring)
• "Optimize serving performance" → ml-production (serving patterns) + pytorch-engineering (low-level profiling, CUDA/memory)

Bidirectional: pytorch-engineering profiling skills point here for production-context observability; this pack's debugging sheet points back to pytorch-engineering for low-level CUDA/memory work.

With ordis-security-architect

For AI/LLM threat modeling — prompt injection, data exfiltration via tool-calls, supply-chain risk on model artifacts, PII leakage in logs, model-registry access control — route to ordis-security-architect (threat modeling, controls design, security review). This pack handles the operational signals (prompt-injection attempt rate as an observability metric) but the threat model and control design live in security-architect.

Common Routing Mistakes

Common Rationalizations (Don't Do These)

Red Flags Checklist

If you catch yourself thinking ANY of these, STOP and clarify:

• "I'll guess optimization vs serving" → ASK which is the bottleneck
• "Performance probably means speed" → ASK speed, accuracy, or LLM output quality
• "Deploy = serving-patterns only" → Consider deployment-strategies and monitoring too
• "They mentioned [tool], route based on tool" → Route by CONCERN, not tool
• "MLOps = one skill" → Could span experiment tracking, automation, dataset curation, and prompt/eval-set versioning
• "Skip question to save time" → Clarifying prevents wrong routing
• "LLM in production = llm-specialist alone" → Default to dual-routing
• "Data quality means schema validation" → ASK whether the concern is well-formedness (mlops) or correctness (dataset-curation)
• "The dataset is given" → It is an artifact with a release contract; route to dataset-curation-and-quality

When in doubt: Ask ONE clarifying question. 10 seconds of clarification prevents minutes of wrong-skill loading.

Routing Summary Table

Integration Examples

Example 1: Full Production Pipeline

Query: "I trained a model, now I need to put it in production"

Routing:

1. Ask: "What's your deployment target and are there performance concerns? Is it an LLM or a classical model?"
2. If "cloud deployment, classical model, fast enough":
   • model-serving-patterns.md (expose as API)
   • deployment-strategies.md (if gradual rollout needed)
   • production-monitoring-and-alerting.md (set up observability)
3. If "edge device, model too large":
   • quantization-for-inference.md (reduce size first)
   • model-serving-patterns.md (edge deployment pattern)
   • production-monitoring-and-alerting.md (if possible on edge)
4. If "LLM endpoint":
   • llm-specialist llm-inference-optimization.md (capability tier + strategy choice)
   • model-serving-patterns.md (LLM-serving Part: vLLM/SGLang/TensorRT-LLM/TGI/Triton)
   • production-monitoring-and-alerting.md (Phoenix/Langfuse + cost monitoring)
   • llm-specialist llm-evaluation-metrics.md (methodology to feed observability)

Example 2: Optimization Decision

Query: "My inference is slow"

Routing:

1. Ask: "Have you profiled to find the bottleneck — is it the model, serving infrastructure, or KV-cache/batching for an LLM?"
2. If "not profiled yet":
   • production-debugging-techniques.md (profile first to diagnose)
   • Then route based on findings
3. If "model is bottleneck":
   • hardware-optimization-strategies.md (check if hardware tuning helps)
   • If not enough → quantization-for-inference.md or model-compression-techniques.md
4. If "infrastructure/batching is bottleneck":
   • model-serving-patterns.md (batching strategies, LLM-serving Part for continuous batching/PagedAttention/speculative decoding)
   • scaling-and-load-balancing.md (if traffic-related)

Example 3: MLOps Maturity

Query: "We need better ML workflows"

Routing:

1. Ask: "What's the current pain point — can't reproduce experiments, manual deployment, prompt/eval-set drift, or some combination?"
2. If "can't reproduce, need to track experiments":
   • experiment-tracking-and-versioning.md
3. If "manual deployment is slow":
   • mlops-pipeline-automation.md
4. If "prompts and eval sets keep drifting":
   • experiment-tracking-and-versioning.md (prompt + eval-set versioning patterns)
   • llm-specialist llm-evaluation-metrics.md (what to put in those eval sets)
5. If multiple:
   • experiment-tracking-and-versioning.md (establish tracking first)
   • mlops-pipeline-automation.md (then automate workflow)

Example 4: LLM in Production with Quality Issues

Query: "Our LLM keeps hallucinating in production and costs are spiking"

Routing:

1. production-monitoring-and-alerting.md — wire Phoenix/Langfuse, capture traces, set cost-budget alerts, surface hallucination/eval-score drift
2. llm-specialist llm-evaluation-metrics.md — define hallucination scoring methodology to feed the observability stack
3. llm-specialist prompt-engineering-patterns.md and/or rag-architecture-patterns.md — remediate the underlying generation-quality issue
4. production-debugging-techniques.md — root-cause cost spikes (token bloat? retry storms? bad routing?)

When NOT to Use ml-production Skills

Skip ml-production when:

• Still designing/training model → Use neural-architectures, training-optimization
• PyTorch infrastructure issues → Use pytorch-engineering
• LLM generation quality only (prompts, RAG retrieval design, fine-tuning strategy choice, agent design, eval methodology, safety/alignment) → Use llm-specialist
• Classical ML deployment → ml-production still applies; consider gradient boosting / sklearn serving paths inside serving-patterns
• AI/LLM threat modeling → Use ordis-security-architect
• Data pipelines, warehouses, ELT/dbt, dimensional modelling → data-engineering territory, not this pack
• The statistics of splits, leakage, and paired comparison → Use yzmir-counterfactual-statistics

Red flag: If model isn't trained yet, probably don't need ml-production — except for the dataset. Dataset curation, labelling, release, and eval-set construction happen before training and belong here (dataset-curation-and-quality.md). "Finish training first" applies to serving, scaling, and observability, not to the data the training consumes.

Success Criteria

You've routed correctly when:

• Identified concern category (optimization, serving, MLOps, observability)
• Asked clarifying question for ambiguous queries
• Routed to appropriate skill(s) in logical order
• Didn't let tool choices (Kubernetes, TorchServe, vLLM) dictate routing
• Recognized multi-concern scenarios and routed to multiple skills
• For LLM questions, defaulted to dual-routing across ml-production and llm-specialist unless the question is purely ops or purely generation-quality
• Flagged maintenance-mode tools (e.g., TorchServe) instead of recommending them uncritically
• Pointed at ordis-security-architect when threat modeling, not just observability, was at issue

ML Production Specialist Skills Catalog

After routing, load the appropriate specialist skill for detailed guidance. This pack contains exactly 11 reference sheets:

1. quantization-for-inference.md - torch.ao.quantization, AWQ, GPTQ, INT8/INT4 ops, post-training quantization, QAT, calibration, kernel/hardware fit (LLM format choice lives in llm-specialist)
2. model-compression-techniques.md - Pruning (structured/unstructured), knowledge distillation, architecture optimization, model size reduction
3. hardware-optimization-strategies.md - GPU/CPU/edge tuning, batch sizing, memory optimization, hardware-specific acceleration (TensorRT, ONNX Runtime), capability-tier hardware selection
4. model-serving-patterns.md - FastAPI, gRPC, ONNX, batching, containerization (Docker), REST/gRPC APIs; LLM-serving Part: vLLM, SGLang, TensorRT-LLM, TGI, Triton, continuous batching, PagedAttention, speculative decoding, KV-cache management; TorchServe covered as legacy/maintenance-mode
5. deployment-strategies.md - A/B testing, canary deployment, shadow mode, gradual rollout, rollback procedures, blue-green deployment
6. scaling-and-load-balancing.md - Horizontal scaling, autoscaling, load balancing, traffic management, cost optimization, replica management
7. experiment-tracking-and-versioning.md - MLflow, Weights & Biases, Comet, Hugging Face Hub, model registries, prompt versioning, RAG / eval-set versioning, lineage, reproducibility
8. mlops-pipeline-automation.md - CI/CD for ML, feature stores, data validation, automated retraining, orchestration (Airflow, Kubeflow, Prefect, Dagster)
9. production-monitoring-and-alerting.md - Metrics tracking, drift detection, dashboards, alerting, SLAs; LLM observability: Phoenix, Langfuse, OpenTelemetry GenAI semantic conventions; tool-call telemetry, prompt-injection signal, RAG-retrieval-quality drift, cost/token monitoring
10. production-debugging-techniques.md - Error analysis, production profiling, rollback procedures, post-mortems, root cause analysis, tool-call failure forensics
11. dataset-curation-and-quality.md - The dataset as a versioned production artifact: release contract and composition manifest, documentation standards (Croissant 1.1, Datasheets for Datasets, Data Cards, HF dataset cards), coverage and slice design, deduplication (MinHash/LSH, SemDeDup, datatrove), label quality (Cohen's κ / Fleiss' κ / Krippendorff's α selection, confident learning via cleanlab, annotation platforms — Argilla, Label Studio, doccano, Prodigy; Snorkel OSS flagged low-maintenance), synthetic-data provenance tagging and ratio discipline (model collapse, ablation ladders), eval-set construction and freeze/rotate policy, drift→targeted re-collection

References

• Primary router: plugins/yzmir-ai-engineering-expert/skills/using-ai-engineering/SKILL.md
• Sister packs (bidirectional):
  • yzmir-llm-specialist/using-llm-specialist — generation quality, prompt/RAG/agent/eval/safety design (10 sheets including reasoning-models.md, agentic-patterns-and-mcp.md, context-engineering-and-prompt-caching.md)
  • yzmir-training-optimization/using-training-optimization — training-phase convergence and speed
  • yzmir-pytorch-engineering/using-pytorch-engineering — distributed training, low-level profiling, CUDA/memory
  • ordis-security-architect/using-security-architect — AI/LLM threat modeling, controls, security review, PII handling in shared datasets
  • yzmir-counterfactual-statistics/using-counterfactual-statistics — the statistics beneath dataset work: independent units, grouped splits and the leakage taxonomy, paired comparison, winner's curse