# Modelopt Quant Recipe Search

> Use when the user asks to find, search for, or optimize the best quantization recipe for a model, including direct requests like "find the best quantization recipe and generate a PTQ checkpoint." Guides the multi-candidate loop: choose compute-vs-memory success metrics, select ModelOpt recipe baselines, design AutoQuant/manual recipe deltas, interpret sensitivity, and decide next candidates. Do NOT use for a single known PTQ recipe run (use modelopt-ptq), serving (use modelopt-deployment), creating/running evals (use modelopt-evaluation or modelopt-launching-evals), monitoring jobs (use modelopt-monitor), MLflow browsing (use modelopt-accessing-mlflow), or comparing completed baseline-vs-candidate scores only (use modelopt-compare-results).

- Skill: `nvidia/modelopt-quant-recipe-search` (Agent Skill, multi-file: 3 files)
- Install (CLI): `npx get-skill add nvidia/modelopt-quant-recipe-search`
- Raw SKILL.md: https://api.skillmd.com/api/skills/nvidia/modelopt-quant-recipe-search/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: NVIDIA (https://skillmd.com/u/nvidia)
- Updated: 2026-11-16
- Page: https://skillmd.com/skills/nvidia/modelopt-quant-recipe-search

---


# Quant Recipe Search

Use this skill when quantization is an iterative recipe search, not a one-off
PTQ run. The skill owns strategy: define success, choose the search space,
sequence candidates, and decide the next iteration. It delegates checkpoint
generation, serving, evaluation, monitoring, and metric comparison to the
existing execution skills.

Treat a direct request such as "find the best quantization recipe and generate a
PTQ checkpoint for this model" as enough to start. Recover local state first,
then ask only for missing decisions that change the search.

## Skill Boundaries

- Use `modelopt-ptq` to produce and validate checkpoints.
- Use `modelopt-deployment` to serve checkpoints and debug serving-specific flags.
- Use `modelopt-evaluation` to create NEL configs and submit evals.
- Use `modelopt-launching-evals` to run, resume, debug, and analyze NEL runs.
- Use `modelopt-monitor` for active job tracking.
- Use `modelopt-accessing-mlflow` for MLflow artifact lookup.
- Use `modelopt-compare-results` for validated baseline-vs-candidate deltas and score-field comparability.

Do not duplicate those workflows here. This skill should leave the user with a clear recipe portfolio, success metric, experiment sequence, and next decision.

## Problem

The task is to find the best recipe for a user-defined target, not merely to
produce a quantized checkpoint. A generated PTQ checkpoint is only a candidate.
It becomes a recommended recipe only after evaluation and comparison against the
matching baseline.

Required inputs before planning candidates:

- **Optimization goal:** compute/throughput, memory/latency, or a custom metric.
- **Primary quantization family:** for example NVFP4, W4A16 NVFP4, FP8/W8A8,
  INT4/AWQ, or a custom mixed set.
- **Benchmark set or baseline results:** the user-defined acceptance surface.

If any of these are missing, ask for them. Do not silently default to FP8/W8A8
or call a checkpoint "best" before evaluation.

Default success rule: maximize the chosen performance objective while keeping
each benchmark within 1 percentage point of the matching BF16/FP16 baseline.
Near-threshold or noisy regressions require reruns before making a decision.

## Search Space

Keep the search space explicit. A candidate recipe is a tuple across these axes:

- **Numeric format:** FP8/W8A8, NVFP4/W4A4, W4A16 NVFP4, INT4/AWQ, or mixed
  formats such as NVFP4+FP8.
- **Calibration/search algorithm:** max calibration, MSE calibration, GPTQ,
  AWQ, AutoQuant scoring, and calibration dataset or sample-count variants.
- **Selection method:** manual/heuristic rules, sensitivity-guided manual
  recipes, AutoQuant selection, or a hybrid of AutoQuant plus manual overrides.
- **Module family:** attention, MLP, MoE experts, routers/gates, embeddings,
  `lm_head`, adapters, vision encoders, and model-specific modules.
- **Layer position:** first/last transformer-layer counts or explicit ordinal
  ranges to keep in BF16. First 3-4 and last 1-2 layers are common starting
  candidate ranges, not defaults.
- **Runtime fusion constraints:** modules fused by the inference library must
  use compatible quantization. Examples: vLLM Qwen `linear_attn.in_proj_qkvz`
  and fused MoE expert projections such as gate/up (`w1`/`w3`).
- **Calibration budget:** dataset mix, sample count, sequence length, and batch
  settings.

Do not collapse the search to one dimension such as numeric format only. Read
`references/recipe_iteration.md` when choosing concrete axes or candidates.

## Design Workflow

1. **Recover state**
   - Read result tables, recipe logs, AutoQuant states, sensitivity reports, and
     experiment notes before proposing new work.
   - Ask `modelopt-monitor`, `modelopt-launching-evals`, or `modelopt-compare-results` to recover active
     job state and completed metrics when needed.

2. **Define the target**
   - Confirm the optimization goal, primary quantization family, benchmark set,
     accuracy-loss threshold, calibration budget, and cost metric.
   - Include quantization metadata such as scale storage in active-cost or size
     estimates.

3. **Pick baselines and first candidates**
   - Always include BF16/FP16 and a near-lossless FP8/W8A8 baseline unless FP8
     itself is the target.
   - For ModelOpt work, start from `modelopt_recipes`: model-specific recipes
     first, then general PTQ presets or recipe fragments.
   - Add an AutoQuant candidate in the requested primary family when AutoQuant
     is available. Expect AutoQuant to find a better trade-off than a first
     manual recipe, but validate that assumption with the same evals.
   - Add at least one manual or sensitivity-guided candidate so AutoQuant can be
     compared against controlled ablations and there is a fallback if AutoQuant
     misses the best frontier or hits runtime constraints.
   - When sensitivity or model behavior implicates boundary layers, add a
     controlled first-layer, last-layer, or combined BF16 exclusion candidate.
     Do not preserve boundary layers without testing the trade-off.

4. **Generate candidates**
   - Delegate checkpoint generation and PTQ validation to `modelopt-ptq`.
   - Change one major axis at a time: format, calibration algorithm, module
     family, layer position, granularity, or calibration data.
   - Use AutoQuant for broad candidate generation and sensitivity reports; use
     manual recipes for controlled module-family ablations and overrides.
   - Resolve positional ordinals against the model's transformer block sequence.
     For manual recipes, add those blocks to the recipe exclusions. For
     AutoQuant or hybrid candidates, pass positional exclusions into the
     selected implementation when supported; otherwise apply a manual override
     to its result and record the limitation.

5. **Gate before scaling**
   - Validate checkpoint coverage and metadata.
   - Reject or rewrite recipes that mix quantization algorithms inside a fused
     runtime group.
   - Ensure positional exclusions preserve complete fused runtime groups, then
     evaluate the candidate against the same BF16 baseline and acceptance
     criteria as every other recipe.
   - If the checkpoint is valid but serving fails due to runtime support, do not
     reject the recipe immediately. Delegate to `modelopt-deployment` / `modelopt-debug` for small
     patches or flags, then rerun a pipe-clean check.

## Iteration Loop

1. Run cheap screen evals for every candidate that passes the gates.
2. Compare accuracy, verbosity/token usage, and active cost against baselines.
3. Rerun noisy or near-threshold results before labeling a regression.
4. Decide the next candidate:
   - Accuracy drop: protect or ablate sensitive module families, try MSE/GPTQ,
     use AutoQuant sensitivity to choose overrides, or test first/last-layer
     BF16 exclusions when evidence points to boundary sensitivity.
   - Poor performance/cost: quantize the next high-cost active family, adjust
     active-cost objective, or try a more aggressive format.
   - AutoQuant underperforms manual recipes: inspect sensitivity reports,
     achieved bits, excluded modules, and runtime-fusion constraints; keep the
     manual recipe in the portfolio instead of forcing the AutoQuant result.
   - Runtime incompatibility: rewrite around fused groups or isolate deployment
     support from checkpoint quality.
   - Repeated AutoQuant recipes: inspect achieved bits and recipe hashes, then
     adjust constraints before launching a larger sweep.
5. Promote only when `modelopt-compare-results` shows no failed external sanity check,
   the candidate is comparable to the validated measured baseline, and the
   user-defined goal is met. An externally unverified baseline is non-blocking.

When the user has authorized QAD, a candidate is a recipe plus the QAD settings
from the `modelopt-qad` skill's recipe loop, which ends each recipe with a pass or a
clear miss. A PTQ recipe that already meets the threshold counts as a pass.
Step the layout from that outcome:

- After a pass, add one module group to the passing recipe, in order of
  savings on the success metric; the MTP head is its own group. A group can be
  a slice: its middle layers with the first and last few kept, or some of its
  projections (such as GDN `in_proj` only, or attention `linear_qkv` only).
  Find and evaluate the new layout's best PTQ recipe, starting from the
  passing recipe's calibration, before its QAD. Step one layout at a time;
  the `modelopt-qad` recipe loop runs its knob variants in parallel.
- After a clear miss, shrink the added group to a slice; if that misses too,
  drop the group and try the next one on the last passing recipe. If no recipe
  has passed yet, report no QAD pass and return to the accuracy-drop steps
  above.
- Stop when every group has been added or dropped, or the QAD budget is spent.
  Return the passing recipe that best meets the success metric, with its QAD
  checkpoint.

Maintain a recipe portfolio table with recipe name, objective, active-cost
estimate, calibration notes, checkpoint path, eval/log references, accuracy,
verbosity, positional exclusions, and decision.

## References

- For recipe design, search-space details, sensitivity, and active-cost
  accounting, read `references/recipe_iteration.md`.
- For a concrete prior case study, read `references/qwen36_case_study.md` only
  when Qwen3.5/Qwen3.6 details are relevant.

