# Using Ml Production

> Router skill directing to deployment, optimization, MLOps, and monitoring guides.

- Skill: `tachyon-beep/using-ml-production` (Agent Skill, multi-file: 12 files)
- Install (CLI): `npx skillmds@latest add tachyon-beep/using-ml-production`
- Raw SKILL.md: https://api.skillmd.com/api/skills/tachyon-beep/using-ml-production/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: tachyon-beep (https://skillmd.com/u/tachyon-beep)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/tachyon-beep/using-ml-production

---


# Using ML Production

## Overview

This meta-skill routes you to the right production deployment skill based on your concern. Load this when you need to move ML models to production but aren't sure which specific aspect to address.

**Core Principle**: Production concerns fall into four categories. Identify the concern first, then route to the appropriate skill. Tools and infrastructure choices are implementation details, not routing criteria.

**Capability tiers, not model IDs.** Sheets describe model and hardware capabilities (e.g., "frontier reasoning model", "mid-tier serving GPU", "edge accelerator") rather than naming specific SKUs. Vendor model names and exact GPU SKUs change quarterly; capability tiers are stable.

**Knowledge-cutoff acknowledgement.** Tool inventory is calibrated to 2026-05. Fast-moving areas — LLM serving stacks (vLLM, SGLang, TensorRT-LLM, TGI, Triton), observability platforms (Phoenix, Langfuse, OTel GenAI), quantization toolchains (`torch.ao.quantization`, AWQ, GPTQ), and MLOps platforms — should be re-checked against vendor docs before architecting new systems. Treat sheet content as a structured starting point, not a substitute for current documentation.

## When to Use

Load this skill when:
- Deploying ML models (classical or LLM) to production
- Optimizing model inference (speed, size, cost)
- Setting up MLOps workflows (tracking, automation, CI/CD)
- Monitoring or debugging production models, including LLM-specific signals (hallucination rate, tool-call success, prompt-injection attempts)
- Curating, documenting, releasing, or handing over a **dataset** — including label quality, deduplication, synthetic-data provenance, and eval-set construction
- User mentions: "production", "deploy", "serve model", "MLOps", "monitoring", "optimize inference", "vLLM", "SGLang", "TensorRT-LLM", "Phoenix", "Langfuse", "dataset", "data card", "Croissant", "labelling", "annotation", "synthetic data", "eval set"

**Don't use for**: Training optimization (use `training-optimization`), model architecture selection (use `neural-architectures`), PyTorch infrastructure (use `pytorch-engineering`), prompt/RAG/agent design quality (use `llm-specialist`).

---

## How to Access Reference Sheets

**IMPORTANT**: All reference sheets are located in the SAME DIRECTORY as this SKILL.md file.

When this skill is loaded from:
  `skills/using-ml-production/SKILL.md`

Reference sheets like `quantization-for-inference.md` are at:
  `skills/using-ml-production/quantization-for-inference.md`

NOT at:
  `skills/quantization-for-inference.md` ← WRONG PATH

When you see a link like `[quantization-for-inference.md](quantization-for-inference.md)`, read the file from the same directory as this SKILL.md.

---

## Routing by Concern

### Category 1: Model Optimization

**Symptoms**: "Model too slow", "inference latency high", "model too large", "need to optimize for edge", "reduce model size", "speed up inference", "quantize model", "INT8/INT4", "AWQ", "GPTQ"

**When to route here**:
- Model itself is the bottleneck (not infrastructure)
- Need to reduce model size or increase inference speed
- Deploying to resource-constrained hardware (edge, mobile)
- Cost optimization through model efficiency
- Quantization *operations* (the mechanics of converting weights, calibration, kernel fit)

**Routes to**:
- [quantization-for-inference.md](quantization-for-inference.md) - `torch.ao.quantization`, AWQ, GPTQ, INT8/INT4, post-training quantization, QAT, calibration
- [model-compression-techniques.md](model-compression-techniques.md) - Pruning, distillation, architecture optimization
- [hardware-optimization-strategies.md](hardware-optimization-strategies.md) - GPU/CPU/edge tuning, batch sizing, hardware/quant-format fit

**Key question to ask**: "Is the MODEL the bottleneck, or is it infrastructure/serving?"

**LLM quantization split**: Choosing *which* quantization format suits a given LLM task (quality vs throughput tradeoffs, instruction-following degradation) is `llm-specialist` territory. The *operations* — wiring up AWQ/GPTQ, running calibration, validating kernel support — live here.

---

### Category 2: Serving Infrastructure

**Symptoms**: "How to serve model", "need API endpoint", "deploy to production", "containerize model", "scale serving", "load balancing", "traffic management", "vLLM", "SGLang", "TensorRT-LLM", "TGI", "Triton", "continuous batching", "PagedAttention", "speculative decoding", "KV-cache management"

**When to route here**:
- Need to expose model as API or service (classical model or LLM)
- Questions about serving patterns (REST, gRPC, batch, streaming)
- Deployment strategies (gradual rollout, A/B testing)
- Scaling concerns (traffic, replicas, autoscaling)
- LLM-serving stacks: vLLM, SGLang, TensorRT-LLM, TGI, Triton — **covered in the LLM-serving Part of [model-serving-patterns.md](model-serving-patterns.md)** alongside continuous batching, PagedAttention, and speculative decoding ops

**Routes to**:
- [model-serving-patterns.md](model-serving-patterns.md) - FastAPI, TorchServe (maintenance mode — see routing mistakes), gRPC, ONNX, batching, containerization, **plus LLM-serving Part: vLLM / SGLang / TensorRT-LLM / TGI / Triton, continuous batching, PagedAttention, speculative decoding**
- [deployment-strategies.md](deployment-strategies.md) - A/B testing, canary, shadow mode, rollback procedures
- [scaling-and-load-balancing.md](scaling-and-load-balancing.md) - Horizontal scaling, autoscaling, load balancing, cost optimization

**Key distinction**:
- Serving patterns = HOW to expose model (API, container, batching, LLM-serving stack)
- Deployment strategies = HOW to roll out safely (gradual, testing, rollback)
- Scaling = HOW to handle traffic (replicas, autoscaling, balancing)

---

### Category 3: MLOps & Data

**Symptoms**: "Track experiments", "version models", "automate deployment", "reproducibility", "CI/CD for ML", "feature store", "model registry", "experiment management", "version prompts", "version RAG eval set", "dataset release", "data card", "datasheet", "Croissant", "label quality", "inter-annotator agreement", "annotation", "deduplicate training data", "synthetic data", "eval-set contamination", "curate dataset", "dataset handover"

**When to route here**:
- Need workflow/process improvements
- Want to track experiments, version models, version prompts, or version RAG eval sets
- Need to automate training-to-deployment pipeline
- Team collaboration and reproducibility concerns
- Need to release, document, hand over, or share a **dataset** as a production artifact
- Label quality, deduplication, synthetic-data provenance, or eval-set construction is the concern

**Routes to**:
- [experiment-tracking-and-versioning.md](experiment-tracking-and-versioning.md) - MLflow, Weights & Biases, Comet, Hugging Face Hub, model registries, prompt versioning, RAG/eval-set versioning, lineage
- [mlops-pipeline-automation.md](mlops-pipeline-automation.md) - CI/CD for ML, feature stores, data validation, automated retraining, orchestration
- [dataset-curation-and-quality.md](dataset-curation-and-quality.md) - Dataset release contract, Croissant / Datasheets / Data Cards, coverage and slices, dedup (MinHash/LSH, SemDeDup), label quality (Krippendorff's α, confident learning, annotation platforms), synthetic-data provenance and ratio discipline, eval-set construction and freeze/rotate policy

**Key distinction**:
- Experiment tracking = Research/development phase (track runs, version models, version prompts/eval sets)
- Pipeline automation = Production phase (automate workflows, CI/CD)
- Dataset curation = The **data artifact itself** (what's in it, is it right, can you defend and hand it over)

**Data sheet split — three-way, and it matters**:
- **What to version and why it's a release** → `dataset-curation-and-quality.md`
- **How to version it** (DVC, lakeFS, hashing, registries, lineage) → `experiment-tracking-and-versioning.md`
- **How to enforce it in CI** (Great Expectations, Pandera, eval gates) → `mlops-pipeline-automation.md`

Schema validation proves the data is *well-formed*; the curation sheet covers whether it is *right* — duplication, label error, missing slices, contamination. Route to both when the query is "our data quality is bad" without further detail.

**Multi-concern**: Queries like "track experiments AND automate deployment" → route to BOTH skills

---

### Category 4: Observability

**Symptoms**: "Monitor production", "model degrading", "detect drift", "production debugging", "alert on failures", "model not working in prod", "performance issues in production", "LLM in production", "hallucination rate", "tool-call success", "cost-per-request spiking", "prompt-injection attempts", "drift in RAG retrieval quality"

**When to route here**:
- Model already deployed, need to monitor or debug
- Detecting production issues (drift, errors, degradation, accuracy regressions)
- Setting up alerts and dashboards
- Root cause analysis for production failures
- LLM-specific telemetry: trace generations, score outputs, track cost/token spend, capture tool-call traces, surface prompt-injection signals

**Routes to**:
- [production-monitoring-and-alerting.md](production-monitoring-and-alerting.md) - Metrics, drift detection, dashboards, alerts, SLAs, **LLM observability platforms: Phoenix, Langfuse, OpenTelemetry GenAI semantic conventions** for trace/eval/cost monitoring
- [production-debugging-techniques.md](production-debugging-techniques.md) - Error analysis, profiling, rollback procedures, post-mortems, tool-call failure forensics

**Key distinction**:
- Monitoring = Proactive (set up metrics, alerts, detect issues early)
- Debugging = Reactive (diagnose and fix existing issues)

**LLM observability split**: The *plumbing* — Phoenix/Langfuse/OTel-GenAI deployment, retention, dashboards, alerts on cost/latency/error budgets — lives here. The *eval methodology* feeding those dashboards (which metrics to compute, how to score hallucinations, how to build judges) lives in `llm-specialist` (`llm-evaluation-metrics.md`). Wire them together: methodology defines the signal; this pack ships it.

**"Performance" ambiguity**:
- If "performance" = speed/latency → might be Category 1 (optimization) or Category 2 (serving/scaling)
- If "performance" = accuracy degradation → Category 4 (observability - drift detection)
- If "performance" = LLM output quality → Category 4 (observability — wire Phoenix/Langfuse) **and** llm-specialist (eval methodology)
- **Ask clarifying question**: "By performance, do you mean inference speed, model accuracy, or LLM output quality?"

---

## Routing Decision Tree

```
User query → Identify primary concern

Is model THE problem (size/speed)?
  YES → Category 1: Model Optimization
  NO → Continue

Is it about HOW to expose/deploy model (incl. LLM-serving stack)?
  YES → Category 2: Serving Infrastructure
  NO → Continue

Is it about workflow/process/automation?
  YES → Category 3: MLOps & Data
  NO → Continue

Is it about the DATASET itself (what's in it, label quality, dedup, synthetic data,
eval-set construction, releasing or handing it over)?
  YES → Category 3: dataset-curation-and-quality.md
  NO → Continue

Is it about monitoring/debugging in production (incl. LLM telemetry)?
  YES → Category 4: Observability
  NO → Ask clarifying question

Is the question about LLM generation quality, prompt design, RAG retrieval design, or agent behavior design?
  YES → Hand off to llm-specialist (this pack does ops, not generation quality)

Ambiguous? → Ask ONE question to clarify concern category
```

---

## Clarification Questions for Ambiguous Queries

### Query: "My model is too slow"

**Ask**: "Is this inference latency (how fast predictions are), or training time?"
- Training → Route to `training-optimization` (wrong pack)
- Inference → Follow-up: "Have you profiled to find bottlenecks?"
  - Model is bottleneck → Category 1 (optimization)
  - Infrastructure/batching issue → Category 2 (serving)
  - LLM-specific (KV cache, batching strategy) → Category 2 LLM-serving Part **+** llm-specialist `llm-inference-optimization.md`

### Query: "I need to deploy my model"

**Ask**: "What's your deployment target — cloud server, edge device, batch processing, or LLM endpoint?"
- Cloud/server → Category 2 (serving-patterns, then maybe deployment-strategies if gradual rollout needed)
- Edge/mobile → Category 1 (optimization first for size/speed) + Category 2 (serving)
- Batch → Category 2 (serving-patterns - batch processing)
- LLM endpoint → Category 2 LLM-serving Part of `model-serving-patterns.md` **+** llm-specialist (`llm-inference-optimization.md` for capability/strategy choice)

### Query: "My model isn't performing well in production"

**Ask**: "By performance, do you mean inference speed, prediction accuracy, or LLM output quality (hallucination, tool-call failure)?"
- Speed → Category 1 (optimization) or Category 2 (serving/scaling)
- Accuracy → Category 4 (observability — drift detection, monitoring)
- LLM output quality → Category 4 (Phoenix/Langfuse plumbing) + llm-specialist (`llm-evaluation-metrics.md` for methodology, `prompt-engineering-patterns.md` / `rag-architecture-patterns.md` for fixes)

### Query: "Set up MLOps for my team"

**Ask**: "What's the current pain point — experiment tracking, automated deployment, prompt/eval-set versioning, or some combination?"
- Tracking/versioning (incl. prompt/eval-set) → Category 3 (experiment-tracking-and-versioning)
- Automation/CI/CD → Category 3 (mlops-pipeline-automation)
- Dataset quality/curation/release → Category 3 (dataset-curation-and-quality)
- Multiple → Route to multiple skills

---

## Multi-Concern Scenarios

Some queries span multiple categories. Route to ALL relevant skills in logical order:

| Scenario | Route Order | Why |
|----------|-------------|-----|
| "Optimize and deploy model" | 1. Optimization → 2. Serving | Optimize BEFORE deploying |
| "Deploy and monitor model" | 1. Serving → 2. Observability | Deploy BEFORE monitoring |
| "Track experiments and automate deployment" | 1. Experiment tracking → 2. Pipeline automation | Track BEFORE automating |
| "Quantize model and serve with vLLM" | 1. Quantization (ops) → 2. LLM-serving Part of serving-patterns | Optimize BEFORE serving |
| "Deploy with A/B testing and monitor" | 1. Deployment strategies → 2. Monitoring | Deploy strategy BEFORE monitoring |
| "Deploy LLM to production" | 1. llm-specialist (model/strategy choice) → 2. ml-production serving + 3. ml-production observability | Generation-quality choice frames the ops |
| "RAG production system" | 1. llm-specialist (`rag-architecture-patterns.md`) → 2. ml-production (vector-store deployment, retrieval-quality observability, latency monitoring) | Architecture decisions frame the ops |
| "Agent in production" | 1. llm-specialist (`agentic-patterns-and-mcp.md`) → 2. ml-production (tool-call observability, error rates, cost monitoring) → 3. axiom-engineering-foundations (sandboxing, system design) | Agent design frames ops + safety |

**Principle**: Route in execution order (what needs to happen first).

---

## Relationship with Other Packs

### With llm-specialist

The two packs split along the **ops vs. generation-quality boundary**.

**ml-production owns** (this pack):
- General serving stacks: vLLM, SGLang, TensorRT-LLM, TGI, Triton — deployment, batching ops, KV-cache config, scaling
- Quantization for inference *as ops*: `torch.ao.quantization`, AWQ, GPTQ wiring, calibration, hardware fit
- MLOps: CI/CD, registries, feature stores, retraining
- Deployment patterns: containers, canary, A/B, rollback
- Monitoring & observability platforms: Phoenix, Langfuse, OpenTelemetry GenAI semantic conventions, dashboards, alerts, drift detection

**llm-specialist owns** (sister pack — 10 reference sheets):
- Prompt engineering (`prompt-engineering-patterns.md`)
- Reasoning-model use (`reasoning-models.md`)
- Agentic patterns + MCP (`agentic-patterns-and-mcp.md`)
- RAG architecture (`rag-architecture-patterns.md`)
- Fine-tuning strategy choice (`llm-finetuning-strategies.md`)
- Context engineering & prompt caching (`context-engineering-and-prompt-caching.md`, `context-window-management.md`)
- Evaluation methodology (`llm-evaluation-metrics.md`)
- Inference-strategy choice (`llm-inference-optimization.md`)
- Safety / alignment (`llm-safety-alignment.md`)

**Concrete bidirectional triggers**:

| Query | llm-specialist contributes | ml-production contributes |
|-------|---------------------------|---------------------------|
| "Deploy LLM to production" | Inference-strategy + capability-tier choice | Serving stack, monitoring, scaling |
| "Quantize LLM" | Which format suits the task (quality tradeoffs) | `torch.ao.quantization`, AWQ/GPTQ ops, hardware fit |
| "LLM observability" | Eval methodology to feed into dashboards | Phoenix / Langfuse / OTel-GenAI plumbing |
| "RAG production system" | RAG architecture + retrieval design | Vector-store deployment, retrieval-quality observability, latency monitoring |
| "Agent in production" | Agentic patterns, MCP, anti-patterns | Tool-call observability, error rates, cost monitoring (also: axiom-engineering-foundations for sandboxing/system design) |

**Rule of thumb**: If the question is "what should the model/prompt/agent do and how should it be designed?" → llm-specialist. If the question is "how do we run, observe, and operate it at scale?" → ml-production. Most LLM production questions need **both**.

### With training-optimization

**Clear boundary**:
- training-optimization = Training phase (convergence, hyperparameters, training speed)
- ml-production = Inference phase (deployment, serving, monitoring) **plus the dataset as a production artifact**

**The dataset is not an exception to the inference-phase framing.** `dataset-curation-and-quality.md` treats the dataset as a *versioned production artifact with a release contract and quality gates* — composition, provenance, documentation, handover, and the drift→re-collection loop. That is operational discipline, and it belongs here. What belongs to training-optimization is what you do with data *during* a training run (curriculum, sampling schedules, augmentation for convergence), not what the dataset **is**.

**"Too slow" disambiguation**:
- Training slow → training-optimization
- Inference slow → ml-production

**Bidirectional**: training-optimization should send users here when they're done training; ml-production should send users back when they need to retrain (drift response, dataset updates).

### With data-engineering concerns

Pipeline and warehouse mechanics — batch/streaming pipelines, ELT/ETL, dbt, dimensional and lakehouse modelling, orchestration as a data-platform concern, CDC — are **not** in this pack. `dataset-curation-and-quality.md` is deliberately scoped to the ML training/eval corpus as a *model-facing* artifact: what is in it, whether it is right, and how it is released. If the question is "how do we build and model the warehouse this data comes from," that is a data-engineering question, not an ml-production one.

### With pytorch-engineering

**pytorch-engineering covers**: Foundation (distributed training, profiling, memory management)

**ml-production covers**: Production-specific (serving APIs, deployment patterns, MLOps, production observability)

**When to use both**:
- "Profile production inference" → pytorch-engineering (profiling techniques) + ml-production (production context, observability wiring)
- "Optimize serving performance" → ml-production (serving patterns) + pytorch-engineering (low-level profiling, CUDA/memory)

**Bidirectional**: pytorch-engineering profiling skills point here for production-context observability; this pack's debugging sheet points back to pytorch-engineering for low-level CUDA/memory work.

### With ordis-security-architect

For AI/LLM threat modeling — prompt injection, data exfiltration via tool-calls, supply-chain risk on model artifacts, PII leakage in logs, model-registry access control — route to `ordis-security-architect` (threat modeling, controls design, security review). This pack handles the *operational signals* (prompt-injection attempt rate as an observability metric) but the *threat model and control design* live in security-architect.

---

## Common Routing Mistakes

| Query | Wrong Route | Correct Route | Why |
|-------|-------------|---------------|-----|
| "Model too slow in production" | Immediately to quantization | Ask: inference or training? Then model vs infrastructure? | Could be serving/batching issue, not model |
| "Deploy with Kubernetes" | Defer to Kubernetes docs | Category 2: serving-patterns or deployment-strategies | Kubernetes is tool choice, not routing concern |
| "Set up MLOps" | Route to one skill | Ask about specific pain point — could be tracking AND automation AND prompt/eval-set versioning | MLOps spans multiple skills |
| "Performance issues" | Assume accuracy | Ask: speed, accuracy, or LLM output quality? | Performance is ambiguous |
| "Deploy LLM" | Skip llm-specialist, route only here | Route both: llm-specialist (strategy/capability choice), ml-production (serving stack, observability, scaling) | Generation-quality choices frame the ops |
| "Just use TorchServe" | Route uncritically to TorchServe content | Flag: TorchServe is in maintenance mode (PyTorch deprecated active development). Route to LLM-serving Part of `model-serving-patterns.md` (vLLM/SGLang/TensorRT-LLM/TGI/Triton) for LLMs; for classical models route to serving-patterns but recommend evaluating Triton or framework-native serving | Don't recommend a maintenance-mode tool for new systems |
| "We use TorchServe" (existing system) | Skip routing | Route to serving-patterns, note migration path | Existing systems still need ops guidance, but flag the deprecation |
| "Hallucinations in production" | Route only to llm-specialist | Route both: ml-production (Phoenix/Langfuse to detect+alert) + llm-specialist (eval methodology + fix via prompt/RAG/fine-tune) | Detection is ops; remediation is generation quality |
| "Quantize an LLM" | Route only to quantization-for-inference | Route both: llm-specialist (which format suits the task) + ml-production (AWQ/GPTQ ops, hardware fit) | Format choice and ops are different concerns |
| "Our data quality is bad" | Route to mlops-pipeline-automation (data validation) | Route both: `dataset-curation-and-quality.md` (is the data *right* — dedup, label error, coverage, contamination) + mlops-pipeline-automation (how to *enforce* it in CI) | Schema validation proves well-formedness, not correctness |
| "Version our dataset" | Route only to experiment-tracking-and-versioning | Ask: mechanics or release contract? DVC/hashing → experiment-tracking; what a release *contains* and how it's documented/handed over → dataset-curation-and-quality | Versioning tooling ≠ knowing what to version |
| "Our eval set is contaminated" | Route to dataset-curation only | Route by model type: general construction/hygiene → dataset-curation-and-quality; LLM n-gram/pretraining contamination → llm-specialist `llm-evaluation-metrics.md` Part 10; the *statistics* of leakage → `yzmir-counterfactual-statistics` | Three packs own three different layers |
| "Model isn't trained yet, so skip this pack" | Defer entirely to training packs | Dataset curation, labelling, and eval-set construction happen pre-training and live here | The "finish training first" red flag applies to serving, not data |

---

## Common Rationalizations (Don't Do These)

| Excuse | Reality |
|--------|---------|
| "User mentioned Kubernetes, route to deployment" | Tools are implementation details. Route by concern first. |
| "Slow = optimization, route to quantization" | Slow could be infrastructure. Clarify model vs serving bottleneck. |
| "They said deploy, must be serving-patterns" | Could need serving + deployment-strategies + monitoring. Don't assume single concern. |
| "MLOps = experiment tracking" | MLOps spans tracking AND automation AND prompt/eval-set versioning. Ask which pain point. |
| "Performance obviously means speed" | Could mean accuracy or LLM output quality. Clarify. |
| "They're technical, skip clarification" | Technical users still benefit from clarifying questions. |
| "LLM question, route only to llm-specialist" | Most LLM production questions need both packs. Default to dual-route. |
| "TorchServe still works, just route there" | TorchServe is in maintenance mode. Flag and offer alternatives. |
| "Data quality = schema validation, route to mlops" | Schema checks can't see duplication, label error, missing slices, or contamination. Route to dataset-curation too. |
| "The dataset is an input, not our concern" | The dataset is a versioned production artifact with a release contract. It is squarely this pack's concern. |
| "They're pre-training, wrong pack" | Dataset curation and eval-set construction precede training and belong here. Only serving/scaling/observability require a trained model. |

---

## Red Flags Checklist

If you catch yourself thinking ANY of these, STOP and clarify:

- "I'll guess optimization vs serving" → ASK which is the bottleneck
- "Performance probably means speed" → ASK speed, accuracy, or LLM output quality
- "Deploy = serving-patterns only" → Consider deployment-strategies and monitoring too
- "They mentioned [tool], route based on tool" → Route by CONCERN, not tool
- "MLOps = one skill" → Could span experiment tracking, automation, dataset curation, and prompt/eval-set versioning
- "Skip question to save time" → Clarifying prevents wrong routing
- "LLM in production = llm-specialist alone" → Default to dual-routing
- "Data quality means schema validation" → ASK whether the concern is well-formedness (mlops) or correctness (dataset-curation)
- "The dataset is given" → It is an artifact with a release contract; route to dataset-curation-and-quality

**When in doubt**: Ask ONE clarifying question. 10 seconds of clarification prevents minutes of wrong-skill loading.

---

## Routing Summary Table

| User Concern | Ask Clarifying | Route To | Also Consider |
|--------------|----------------|----------|---------------|
| Model slow/large | Inference or training? | Optimization skills | If inference, check serving too |
| Deploy classical model | Target (cloud/edge/batch)? | Serving patterns | Deployment strategies for gradual rollout |
| Deploy LLM | Capability tier and target? | LLM-serving Part of serving-patterns + llm-specialist | Observability + cost monitoring |
| Production monitoring | Proactive or reactive? | Monitoring OR debugging | Both if setting up + fixing issues |
| LLM observability | Plumbing or methodology? | Monitoring (plumbing) + llm-specialist (methodology) | Cost + drift alerts |
| MLOps setup | Tracking, automation, dataset curation, or prompt/eval versioning? | Experiment tracking AND/OR automation AND/OR dataset-curation | Often multiple needed |
| Dataset quality/curation | Well-formed (schema) or right (content)? | Dataset-curation-and-quality | mlops-pipeline-automation to enforce in CI |
| Dataset release/handover | Internal handover or external share? | Dataset-curation-and-quality (release contract, Croissant) | security-architect for PII/sharing threat model |
| Label quality / annotation | Measured or assumed? | Dataset-curation-and-quality (κ/α, confident learning) | Taxonomy problems are not model problems |
| Synthetic data expansion | Contamination checked? | Dataset-curation-and-quality (provenance, ablation ladder) | counterfactual-statistics for the comparison stats |
| Performance issues | Speed, accuracy, or LLM quality? | Optimization OR observability OR llm-specialist | Depends on clarification |
| Scale serving | Traffic pattern? | Scaling-and-load-balancing | Serving patterns if not set up yet |

---

## Integration Examples

### Example 1: Full Production Pipeline

**Query**: "I trained a model, now I need to put it in production"

**Routing**:
1. Ask: "What's your deployment target and are there performance concerns? Is it an LLM or a classical model?"
2. If "cloud deployment, classical model, fast enough":
   - [model-serving-patterns.md](model-serving-patterns.md) (expose as API)
   - [deployment-strategies.md](deployment-strategies.md) (if gradual rollout needed)
   - [production-monitoring-and-alerting.md](production-monitoring-and-alerting.md) (set up observability)
3. If "edge device, model too large":
   - [quantization-for-inference.md](quantization-for-inference.md) (reduce size first)
   - [model-serving-patterns.md](model-serving-patterns.md) (edge deployment pattern)
   - [production-monitoring-and-alerting.md](production-monitoring-and-alerting.md) (if possible on edge)
4. If "LLM endpoint":
   - llm-specialist `llm-inference-optimization.md` (capability tier + strategy choice)
   - [model-serving-patterns.md](model-serving-patterns.md) (LLM-serving Part: vLLM/SGLang/TensorRT-LLM/TGI/Triton)
   - [production-monitoring-and-alerting.md](production-monitoring-and-alerting.md) (Phoenix/Langfuse + cost monitoring)
   - llm-specialist `llm-evaluation-metrics.md` (methodology to feed observability)

### Example 2: Optimization Decision

**Query**: "My inference is slow"

**Routing**:
1. Ask: "Have you profiled to find the bottleneck — is it the model, serving infrastructure, or KV-cache/batching for an LLM?"
2. If "not profiled yet":
   - [production-debugging-techniques.md](production-debugging-techniques.md) (profile first to diagnose)
   - Then route based on findings
3. If "model is bottleneck":
   - [hardware-optimization-strategies.md](hardware-optimization-strategies.md) (check if hardware tuning helps)
   - If not enough → [quantization-for-inference.md](quantization-for-inference.md) or [model-compression-techniques.md](model-compression-techniques.md)
4. If "infrastructure/batching is bottleneck":
   - [model-serving-patterns.md](model-serving-patterns.md) (batching strategies, LLM-serving Part for continuous batching/PagedAttention/speculative decoding)
   - [scaling-and-load-balancing.md](scaling-and-load-balancing.md) (if traffic-related)

### Example 3: MLOps Maturity

**Query**: "We need better ML workflows"

**Routing**:
1. Ask: "What's the current pain point — can't reproduce experiments, manual deployment, prompt/eval-set drift, or some combination?"
2. If "can't reproduce, need to track experiments":
   - [experiment-tracking-and-versioning.md](experiment-tracking-and-versioning.md)
3. If "manual deployment is slow":
   - [mlops-pipeline-automation.md](mlops-pipeline-automation.md)
4. If "prompts and eval sets keep drifting":
   - [experiment-tracking-and-versioning.md](experiment-tracking-and-versioning.md) (prompt + eval-set versioning patterns)
   - llm-specialist `llm-evaluation-metrics.md` (what to put in those eval sets)
5. If multiple:
   - [experiment-tracking-and-versioning.md](experiment-tracking-and-versioning.md) (establish tracking first)
   - [mlops-pipeline-automation.md](mlops-pipeline-automation.md) (then automate workflow)

### Example 4: LLM in Production with Quality Issues

**Query**: "Our LLM keeps hallucinating in production and costs are spiking"

**Routing**:
1. [production-monitoring-and-alerting.md](production-monitoring-and-alerting.md) — wire Phoenix/Langfuse, capture traces, set cost-budget alerts, surface hallucination/eval-score drift
2. llm-specialist `llm-evaluation-metrics.md` — define hallucination scoring methodology to feed the observability stack
3. llm-specialist `prompt-engineering-patterns.md` and/or `rag-architecture-patterns.md` — remediate the underlying generation-quality issue
4. [production-debugging-techniques.md](production-debugging-techniques.md) — root-cause cost spikes (token bloat? retry storms? bad routing?)

---

## When NOT to Use ml-production Skills

**Skip ml-production when:**
- Still designing/training model → Use neural-architectures, training-optimization
- PyTorch infrastructure issues → Use pytorch-engineering
- LLM generation quality only (prompts, RAG retrieval design, fine-tuning strategy choice, agent design, eval methodology, safety/alignment) → Use llm-specialist
- Classical ML deployment → ml-production still applies; consider gradient boosting / sklearn serving paths inside serving-patterns
- AI/LLM threat modeling → Use ordis-security-architect
- Data pipelines, warehouses, ELT/dbt, dimensional modelling → data-engineering territory, not this pack
- The *statistics* of splits, leakage, and paired comparison → Use `yzmir-counterfactual-statistics`

**Red flag**: If model isn't trained yet, probably don't need ml-production — **except for the dataset**. Dataset curation, labelling, release, and eval-set construction happen *before* training and belong here ([dataset-curation-and-quality.md](dataset-curation-and-quality.md)). "Finish training first" applies to serving, scaling, and observability, not to the data the training consumes.

---

## Success Criteria

You've routed correctly when:
- Identified concern category (optimization, serving, MLOps, observability)
- Asked clarifying question for ambiguous queries
- Routed to appropriate skill(s) in logical order
- Didn't let tool choices (Kubernetes, TorchServe, vLLM) dictate routing
- Recognized multi-concern scenarios and routed to multiple skills
- For LLM questions, defaulted to dual-routing across ml-production and llm-specialist unless the question is purely ops or purely generation-quality
- Flagged maintenance-mode tools (e.g., TorchServe) instead of recommending them uncritically
- Pointed at ordis-security-architect when threat modeling, not just observability, was at issue

---

## ML Production Specialist Skills Catalog

After routing, load the appropriate specialist skill for detailed guidance. **This pack contains exactly 11 reference sheets**:

1. [quantization-for-inference.md](quantization-for-inference.md) - `torch.ao.quantization`, AWQ, GPTQ, INT8/INT4 ops, post-training quantization, QAT, calibration, kernel/hardware fit (LLM format *choice* lives in llm-specialist)
2. [model-compression-techniques.md](model-compression-techniques.md) - Pruning (structured/unstructured), knowledge distillation, architecture optimization, model size reduction
3. [hardware-optimization-strategies.md](hardware-optimization-strategies.md) - GPU/CPU/edge tuning, batch sizing, memory optimization, hardware-specific acceleration (TensorRT, ONNX Runtime), capability-tier hardware selection
4. [model-serving-patterns.md](model-serving-patterns.md) - FastAPI, gRPC, ONNX, batching, containerization (Docker), REST/gRPC APIs; **LLM-serving Part: vLLM, SGLang, TensorRT-LLM, TGI, Triton, continuous batching, PagedAttention, speculative decoding, KV-cache management**; TorchServe covered as legacy/maintenance-mode
5. [deployment-strategies.md](deployment-strategies.md) - A/B testing, canary deployment, shadow mode, gradual rollout, rollback procedures, blue-green deployment
6. [scaling-and-load-balancing.md](scaling-and-load-balancing.md) - Horizontal scaling, autoscaling, load balancing, traffic management, cost optimization, replica management
7. [experiment-tracking-and-versioning.md](experiment-tracking-and-versioning.md) - MLflow, Weights & Biases, Comet, Hugging Face Hub, model registries, prompt versioning, RAG / eval-set versioning, lineage, reproducibility
8. [mlops-pipeline-automation.md](mlops-pipeline-automation.md) - CI/CD for ML, feature stores, data validation, automated retraining, orchestration (Airflow, Kubeflow, Prefect, Dagster)
9. [production-monitoring-and-alerting.md](production-monitoring-and-alerting.md) - Metrics tracking, drift detection, dashboards, alerting, SLAs; **LLM observability: Phoenix, Langfuse, OpenTelemetry GenAI semantic conventions**; tool-call telemetry, prompt-injection signal, RAG-retrieval-quality drift, cost/token monitoring
10. [production-debugging-techniques.md](production-debugging-techniques.md) - Error analysis, production profiling, rollback procedures, post-mortems, root cause analysis, tool-call failure forensics
11. [dataset-curation-and-quality.md](dataset-curation-and-quality.md) - The dataset as a versioned production artifact: release contract and composition manifest, documentation standards (**Croissant 1.1**, Datasheets for Datasets, Data Cards, HF dataset cards), coverage and slice design, deduplication (MinHash/LSH, SemDeDup, `datatrove`), label quality (Cohen's κ / Fleiss' κ / Krippendorff's α selection, confident learning via `cleanlab`, annotation platforms — Argilla, Label Studio, doccano, Prodigy; Snorkel OSS flagged low-maintenance), synthetic-data provenance tagging and ratio discipline (model collapse, ablation ladders), eval-set construction and freeze/rotate policy, drift→targeted re-collection

---

## References

- Primary router: `plugins/yzmir-ai-engineering-expert/skills/using-ai-engineering/SKILL.md`
- Sister packs (bidirectional):
  - `yzmir-llm-specialist/using-llm-specialist` — generation quality, prompt/RAG/agent/eval/safety design (10 sheets including `reasoning-models.md`, `agentic-patterns-and-mcp.md`, `context-engineering-and-prompt-caching.md`)
  - `yzmir-training-optimization/using-training-optimization` — training-phase convergence and speed
  - `yzmir-pytorch-engineering/using-pytorch-engineering` — distributed training, low-level profiling, CUDA/memory
  - `ordis-security-architect/using-security-architect` — AI/LLM threat modeling, controls, security review, PII handling in shared datasets
  - `yzmir-counterfactual-statistics/using-counterfactual-statistics` — the statistics beneath dataset work: independent units, grouped splits and the leakage taxonomy, paired comparison, winner's curse