# Add Recipe > Use when the user asks to add, update, validate, or publish a vLLM model recipe in this repo. Covers canonical Hugging Face identity, source/image requirements, checkpoint staging, hardware evidence, YAML-to-UI relationships, command validation, JSON generation, and staging the reviewed files. - Skill: `vllm-project/add-recipe` (Agent Skill) - Install (CLI): `npx skillmds@latest add vllm-project/add-recipe` - Raw SKILL.md: https://api.skillmd.com/api/skills/vllm-project/add-recipe/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - Author: vllm-project (https://skillmd.com/u/vllm-project) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/vllm-project/add-recipe --- # Author and validate a vLLM recipe Recipes are YAML files at `models//.yaml`. The path mirrors HuggingFace (`huggingface.co//`), and the site/API are generated at build time from these files + `taxonomy.yaml` + `strategies/*.yaml`. ## Component contract: evidence → YAML → rendered surface Do not author the YAML as isolated documentation. Each input controls a different consumer, and a plausible guide does not prove that the command builder shows the same thing. ```text HF repo + config.json ───────────────► model identity, architecture, context, variants runtime source ref + distribution ──► Install block and reproducible prerequisites hardware profile + strategy YAML ───► selectable topology and generated parallel flags real serving evidence ──────────────► meta.hardware verified + tested guide section recipe YAML ─► loader ─► command-synthesis ─► CommandBuilder / Verify / Bench └────────────────────────► static JSON API guide markdown ──────────────────────► Guide accordion and explicit reference links ``` | Component | Put it in | What consumes or displays it | Evidence required | |---|---|---|---| | Canonical recipe identity | file path + `model.model_id` | route, page title, header HF link, JSON path | exact HF repo; preserve org/repo case | | Quantized or alternate checkpoint | `variants..model_id`, including `variants.default.model_id` when the preferred default differs from the family route | Variant pill, generated Serve/Verify/Bench model id, variant redirect | exact accessible checkpoint and its config | | Alternate-checkpoint reference | `guide` Overview, serving section, and References | clickable Guide links | link the exact tested repo; the header HF link still points to the canonical recipe path | | Runtime distribution | `model.install` and `model.docker_image` | Install tabs and Docker-wrapped command | only show an image/wheel that actually exists and contains support | | Extra prerequisite | `dependencies[]` | Extra install block after Install and before Serve | additive only; it does not replace the Install tabs | | Base runtime behavior | `model.base_args/base_env` | every generated command | required for every variant/hardware/strategy | | Variant behavior | `variants.*.extra_args/extra_env` | selected variant command | required only for that checkpoint/precision | | Hardware status | `meta.hardware` | recipe-wide verified caption or disabled hardware pill | `verified` only after an end-to-end run representative of the recipe; `unsupported` only after a confirmed current blocker; absent means no badge | | Hardware-specific behavior | `hardware_overrides` | command/env after selecting that GPU generation | tested or documented runtime requirement; status and override are independent | | Strategy availability | `compatible_strategies` | Strategy pills | architecture/topology support, not model popularity | | Strategy customization | `strategy_overrides` | TP sizing or role-specific generated command | explicit topology requirement | | Feature | `features` + `opt_in_features` | Feature pills and final command args | do not imply a feature was tested merely because base chat passed | | Capacity hint | `variants.*.vram_minimum_gb` + taxonomy | VRAM warning and auto-fit TP hint | total-parameter sizing formula; not a hardware support gate | | Narrative and claims | `guide` | Guide accordion only | separate tested facts from advertised/model-card capabilities | | Agent/API output | generated `public//.json` | static consumers | generated by validation; never stage `public/` | ### Rendering consequences that are easy to miss - The header's **View on HuggingFace** link is derived from the recipe file path, not the selected variant. If FP8/NVFP4/etc. is a different repo, link it explicitly in the Overview, its serving section, and References. - `meta.hardware` is not variant-specific. A family recipe may keep the base HF path while `variants.default.model_id` selects a separately published preferred checkpoint; identify the tested variant explicitly in the guide so the recipe-wide badge is not mistaken for evidence about untested variants. - `variants..model_id` changes the generated serve command and the Verify/Bench model id. A guide-only FP8 command does not update the interactive builder. - `dependencies[]` is additive. For unreleased source support, place the complete build command in `model.install.pip.command`; if no image exists, set `model.install.docker: false`. Do not leave a generic Docker or nightly-wheel tab visible when it cannot run the model. - `nightly_required: true` changes the default pip text and shows a nightly badge. It does not prove the nightly contains the needed model registry entry. - `meta.hardware.: verified` is a claim about the selected recipe path, variant, and working serving flow. Record the tested variant/context in the guide. - The initial builder state should reproduce the successful baseline: `default` variant, default-on features, `strategy_overrides.single_node_tp.tp`, and `base_args` all affect it. Move untested features to `opt_in_features` and encode the tested TP/context rather than relying on auto-sizing or model-card limits. - `guide` commands are not synthesized or checked against the builder. Preview both and reconcile model id, TP, flags, environment, and install method. ## End-to-end steps 1. **Resolve current sources before writing claims.** Fetch the runtime repo and record the exact commit that contains model registration and required kernel/quantization fixes. Branch names move and similarly named branches may not contain the same code. Recheck the remote ref immediately before the final build. If support is unreleased, build from source rather than selecting an image that has not been published. 2. **Confirm every HF id.** Preserve the exact `/` string. Resolve the canonical recipe repo and every alternate checkpoint independently. Verify access using the configured credential through the normal HF client; do not substitute a similarly named cached model or invent an alias. 3. **Fetch model metadata.** Run `bash scripts/hf-info.sh /` or inspect the staged checkpoint's `config.json` / `params.json`. Extract: - `architecture`: `moe` if `num_experts`, `num_local_experts`, `moe.num_experts`, or a `*MoE*` architecture name is present. Otherwise `dense`. - the exact `architectures[]` / `model_type`; confirm the runtime resolves that architecture without a custom alias. - `parameter_count`: total params (e.g. `"671B"`, `"70B"`). Use HF model card or the sum of shard sizes. - `active_parameters`: for MoE, the activated-per-token count (e.g. `"37B"` on DeepSeek-V3.2). For dense, equal to `parameter_count`. - `context_length`: `max_position_embeddings` from `config.json` (for VL models, from `text_config.max_position_embeddings`). - `min_vllm_version`: earliest stable support if known. For a source-only preview, also record the tested commit/version; a future target version alone is not enough. 4. **Read the README — don't skip this.** Run `curl -sL "https://huggingface.co///resolve/main/README.md"` and scan the install / serve / usage sections in full. Configs are not enough; model authors put load-bearing requirements in prose. Mine the README for: - **`min_vllm_version` / `nightly_required`** — phrases like "install vllm nightly", "requires nightly wheels", or an install snippet using `--extra-index-url https://wheels.vllm.ai/nightly` mean `min_vllm_version: "nightly"` + `nightly_required: true`. A specific tag like "vLLM >= 0.12.0" sets that version. Don't default to `0.11.0` when the README says otherwise. - **`dependencies:`** — any pip line beyond `vllm` itself: version pins (`mistral_common >= 1.11.1`, `transformers >= 5.4.0`), extras (`vllm[audio]`), source installs (`pip install git+...`), DeepGEMM pins, etc. Pin them even when the README says "auto-installed" — users on stale wheel caches need an explicit upgrade path. Each entry needs a one-line `note` saying *why*. - **Parser flags for `features:`** — `--tool-call-parser `, `--reasoning-parser `, `--enable-auto-tool-choice`. Use the exact parser name the README specifies. - **Companion / draft repos** — EAGLE / MTP / Eagle3 heads, NVFP4 quants, instruct vs base. Wire as `spec_decoding` feature (draft pointer in `--speculative-config`) or a sibling variant with `model_id:` override. Copy the recommended `--speculative-config` JSON verbatim from the README. - **Recommended serve flags** — `--tensor-parallel-size`, `--gpu-memory-utilization`, `--max-num-batched-tokens`, `--max-num-seqs` go into the guide's launch command and into variant `extra_args` when they're variant-specific. - **Hardware guidance / sampling defaults** — "recommended on 8xH200" lines inform variant `description` + `vram_minimum_gb`; recommended `temperature` / `top_p` / `reasoning_effort` go in the guide's Client Usage block. 5. **Cross-check upstream vLLM support.** The README is a snapshot — if it was written at a moment when only nightly worked, that claim rots once stable ships. **Never copy the README's "vLLM nightly" claim verbatim without checking.** Run these in parallel: - `gh search issues --repo vllm-project/vllm "" --state all --limit 20` — bug reports tell you which versions users are actually running on (e.g. an issue body saying "vLLM 0.18.0 + this model crashes" is positive proof the model loads on 0.18.0). - `gh search prs --repo vllm-project/vllm "" --merged --limit 10` — locate the support PR; `gh pr view --json mergedAt` gives the date, cross-reference against `gh release list --repo vllm-project/vllm` to find the minimum release. - **`curl` the registry and supported-models docs at the candidate tag** — this is the most authoritative check: ```bash curl -sL "https://raw.githubusercontent.com/vllm-project/vllm//vllm/model_executor/models/registry.py" | grep -i "" curl -sL "https://raw.githubusercontent.com/vllm-project/vllm//docs/models/supported_models.md" | grep -B2 -A4 "" ``` `supported_models.md` often documents **required flags that the model card omits** — e.g. Voxtral Realtime needs `--tokenizer-mode mistral` per vLLM docs, but the HF README doesn't mention it. Always read this file for the recipe's target tag. - `gh release view --repo vllm-project/vllm --json body` + grep for the model name — release-note mentions confirm support officially landed. - For newer architectures, also search the model author's repo (e.g. `PaddlePaddle/PaddleOCR`, `deepseek-ai/DeepSeek-VL2`) for "vllm" discussions — authors often post the canonical launch command and known issues there. What to extract: - **`min_vllm_version`** — set to the **lowest stable tag where the model actually works**, not what the README claims. Walk forward from the support-PR's release tag, but bump up if there are known parser/tokenizer/quant bugs fixed in a later release (the v0.20.0-style "Mistral Grammar factory" / "tool parser HF-tokenizer fix" entries are signals to bump). Only use `min_vllm_version: "nightly"` + `nightly_required: true` when the registry at the latest stable tag genuinely lacks the architecture — and double-check by curling `registry.py` at that tag. If support is still an open issue (no PR merged), flag this to the user before authoring. For derivative releases (e.g. PaddleOCR-VL-1.5 vs 1.0) with identical `architectures` / `model_type` / `auto_map`, the existing handler usually loads them via `--trust-remote-code` even before a dedicated PR — note this assumption in your reply. - **Required serve flags hidden in upstream docs** — copy any `must be served with ` lines from `supported_models.md` straight into `model.base_args` (and call them out in the guide's launch command). These are not optional and the README often doesn't mention them. - **Troubleshooting** — recurring errors and fixes from issue comments (e.g. "needs `--enforce-eager` on 0.11.x", "transformers>=5 required", "`--mm-processor-cache-gb 0` to avoid OOM"). Surface these in the guide's Troubleshooting section, or as inline tips next to the launch command if they're load-bearing. - **Links to put in `guide`'s References** — the model card, vLLM support PR (not the recipe-request issue — see below), and any author-side deployment doc. These give users a path forward if their setup breaks. **What NOT to put in References**: the recipe-request issue in `vllm-project/recipes` (e.g. `#459`) is a tracking ticket, not a user-facing reference. It belongs in the PR description body (`Closes #459`), never in the YAML's `## References` section. 6. **Stage and verify the exact checkpoint.** Pin the HF revision, download the exact repo, and verify all shards/bytes. Record checkpoint id, revision, shard count, and total size when useful. A completed download is not proof of integrity. 7. **Run on the target hardware.** Capture device name, GPU count/topology, source fingerprint, Transformers dependency, full serve arguments, resolved architecture, weight-load completion, and meaningful warnings. Keep advertised maximum context separate from the smaller context used for a smoke test. 8. **Prove inference, not merely startup.** Require all of: - `/health` succeeds; - `/v1/models` returns the canonical selected model id; - `/v1/chat/completions` produces content and `finish_reason: stop`; - the response fingerprint matches the intended source build. Test multimodal, tool calling, reasoning parsing, speculative decoding, offload, long context, and other optional paths separately before calling those paths verified. 9. **Translate evidence into YAML.** Write `models//.yaml` following the component contract and schema below. The tested command belongs in the builder fields; the guide records exact evidence, limitations, and explicit checkpoint links. 10. **Register the provider (if new).** If `` isn't already in `src/lib/providers.js`, add `display_name` and `/providers/.png` (or `.jpeg`). 11. **Validate data and rendering.** Run the JSON generator, inspect the generated model JSON, then preview the page when dependencies are available. Exercise every relevant Hardware / Variant / Strategy / Feature selection and compare the generated command to the tested command. The Guide alone is not a builder preview. 12. **Stage only intended source files.** Stage the recipe and provider entry if needed; never stage generated `public/`, `node_modules/`, or unrelated operational artifacts. Commit only when the user asked for a commit, using the repository's DCO convention. ## YAML schema (top-level fields, in order) ```yaml meta: title: "..." # display name (e.g. "DeepSeek-V3.2") slug: "..." # lowercase-kebab (legacy, keep consistent with title) provider: "..." # human-readable org label (e.g. "DeepSeek") description: "..." # one-sentence summary date_added: YYYY-MM-DD # initial catalog addition; never change date_updated: YYYY-MM-DD # latest material recipe change difficulty: beginner|intermediate|advanced tasks: # one or more of: text, multimodal, omni, embedding - text performance_headline: "..." # optional pithy line for cards related_recipes: [] # optional list of "/" ids # Optional. Tri-state: # `verified` — you've run this recipe on this GPU end-to-end (green ✓). # `unsupported` — not yet runnable here today (compat gap, missing kernel, # upstream blocker). Pill disabled in UI with "Not yet # supported" tooltip. May flip later — revisit on updates. # absent — silent default, assumed to work. Don't mark "untested". hardware: h200: verified mi355x: verified # mi300x: unsupported # e.g. when a required kernel/feature is missing model: model_id: "/" # MUST match the filename path min_vllm_version: "0.11.0" # string, e.g. "0.12.0" # Optional — pin the Docker image shown in Install → Docker. Two forms: # docker_image: "vllm/vllm-openai:glm51" # string pins NVIDIA only; # # AMD/TPU still use brand defaults # docker_image: # object pins per-brand # nvidia: "vllm/vllm-openai:gemma4" # amd: "vllm/vllm-openai-rocm:gemma4" # tpu: "vllm/vllm-tpu:gemma4" # Missing keys fall back to `:latest` for that brand (vllm/vllm-openai, # vllm/vllm-openai-rocm, vllm/vllm-tpu). Use the object form when CUDA / # ROCm / TPU ship different pinned tags for the same recipe. docker_image: "" # Optional — set true when `min_vllm_version` hasn't shipped as a stable # release yet. Swaps the default pip command to nightly wheels # (https://wheels.vllm.ai/nightly/cu130) and adds a yellow "nightly" pill # to the Install header. Manual install.pip overrides still win. nightly_required: false # Optional — control the Install block's pip/Docker tabs. Each key accepts # `false` (hide the tab entirely) OR an object `{ command?, note? }` to # override the generated one-liner and/or show a note above it. # install.pip: false → no wheel available, Docker only # install.docker: false → no published image, pip only # install.pip.command: "..." → replace the pip command # install.pip.note: "..." → one-liner above the code block # For unreleased source-only support, put the complete clone/checkout/build # sequence in install.pip.command and set install.docker: false. # Tab ORDER follows the YAML key order — put `docker` first to make it the # default tab when Docker is the recommended install path. install: pip: command: "" note: "" docker: note: "" architecture: dense|moe parameter_count: "30B" # string with suffix (B or T) active_parameters: "30B" # same as parameter_count for dense models context_length: 131072 # integer (tokens) base_args: [] # flags always needed (trust-remote-code, etc.) base_env: {} # env vars always needed # Optional — only if the recipe needs extra pip installs beyond `uv pip install -U vllm`. # Rendered as an "extra install" block above the vllm serve command. dependencies: - note: "Why you need it (one line)" command: 'uv pip install -U "vllm[audio]"' optional: false # omit or false for required; true = dimmed in UI + excluded from "Copy all" brand: NVIDIA # optional: NVIDIA | AMD | Intel (or array). Omit for platform-agnostic deps. # Use this for CUDA-only kernels (xformers/DeepGEMM) or ROCm-only wheels — # the dep is hidden when the user picks a hardware pill of a different brand. features: tool_calling: # flip any of these pills; recipe chooses naming description: "..." args: ["--enable-auto-tool-choice", "--tool-call-parser", ""] reasoning: description: "..." args: ["--reasoning-parser", ""] spec_decoding: # USE spec_decoding, NOT mtp — unified key for description: "..." # MTP / Eagle3 / ERNIE-MTP / etc. args: ["--speculative-config", '{"method":"mtp","num_speculative_tokens":1}'] opt_in_features: # features that default OFF (users tick them on) - spec_decoding # spec decoding is opt-in unless the model docs insist variants: default: # ALWAYS include a `default` variant precision: bf16|fp8|nvfp4|fp4|int4|int8|awq|gptq|mxfp4 vram_minimum_gb: # params × bytes × 1.2 (see formula below) description: "..." fp8: # optional extra variants model_id: "" # only if the quantized variant is a different HF repo precision: fp8 vram_minimum_gb: description: "..." supported_hardware: [mi355x] # optional exact hardware-profile allowlist extra_args: [] extra_env: {} compatible_strategies: # subset of the SERVING strategies in strategies/*.yaml - single_node_tp # always include this as a baseline - single_node_tep # for MoE - single_node_dep # for MoE - multi_node_tp - multi_node_tp_pp # TP within node + PP across nodes - multi_node_dep # for MoE - multi_node_tep # for MoE - pd_cluster # only if the recipe documents PD # Do NOT list kv_store_* ids — the KV Offload options (Simple + both # Mooncake modes) are implicit on every non-omni recipe. # Optional opt-OUT for the Mooncake pills on the command builder's # "KV Offload" row. Fail-open like meta.hardware: absent = assumed to work # (pill enabled on any scalable GPU); `unsupported` disables the pill and # makes the JSON API skip that strategy on that hardware. Off and Simple # need no gating (Simple is taxonomy-driven — `taxonomy.yaml → # kv_offload.simple` — and needs nothing per-recipe). kv_cache_strategy_hardware: kv_store_distributed_mooncake: gb200: unsupported hardware_overrides: # optional per-generation flags hopper: { extra_args: [], extra_env: {} } blackwell: { extra_args: [], extra_env: {} } amd: { extra_args: [], extra_env: {} } strategy_overrides: # optional per-strategy tweaks single_node_tp: tp: 1 # optional — default TP size for this strategy. # Lets a small model run below full-node TP # (e.g. Gemma 4 fits on 1 GPU → tp: 1). Omit # to default to the node's gpu_count. Clamped # to [1, gpu_count]. When effective TP < # gpu_count the UI shows a "using N of M GPUs" # hint under the Hardware pill. TEP/DEP and # multi-node ignore `tp:` (topology requires # full pool). extra_args: [] extra_env: {} guide: | # markdown, rendered as the Guide accordion ## Overview ... ## Prerequisites ... ## Launch command ... ## Benchmarking ... ## References - [Model card](https://huggingface.co//) ``` ## VRAM formula `vram_minimum_gb = ceil(params × bytes_per_param × 1.2)` where params is the **total** parameter count (MoE includes inactive experts — they still live in VRAM). | Precision | Bytes/param | |-----------|-------------| | bf16, fp16 | 2 | | fp8, int8, awq, gptq (8-bit) | 1 | | int4, nvfp4, fp4, mxfp4 (4-bit) | 0.5 | Example: a 70B BF16 model → `70 × 2 × 1.2 = 168 GB`. Round up. If the variant is `model_id`-overridden and the override is a different base model with its own param count (e.g. a distilled FP4 checkpoint), use the override's parameter count — verify it via HF. **Mixed-precision quants (NVFP4 / ModelOpt) — don't trust the bytes-per-param table.** NVIDIA ModelOpt NVFP4 checkpoints are *not* uniformly 4-bit: only the MLP linears drop to NVFP4 (W4A16), while attention linears + KV cache stay FP8 and embeddings/norms stay higher precision. `hf_quant_config.json` shows `quant_algo: MIXED_PRECISION` in this case. The pure `params × 0.5 × 1.2` formula then **underestimates** — e.g. `nvidia/Qwen3.6-27B-NVFP4` is ~21.9 GB on disk, not the 13.5 GB the table implies, so `27B × 0.5 × 1.2 = 17` is wrong (the weights alone exceed it). For any mixed-precision checkpoint, size from the **real weight footprint** instead: ```bash # total_size is in bytes → GB; then × 1.2 for KV/activation overhead curl -sL "https://huggingface.co///resolve/main/model.safetensors.index.json" \ | python3 -c "import json,sys; print(round(json.load(sys.stdin)['metadata']['total_size']/1e9*1.2))" ``` So `vram_minimum_gb = ceil(real_checkpoint_GB × 1.2)` (Qwen3.6-27B-NVFP4 → `ceil(21.9 × 1.2) = 27`). The bytes-per-param table stays correct for *uniform* quants (plain int4/awq/gptq/fp8, and full-model FP4). ## Naming and conventions - **Feature keys**: prefer `tool_calling`, `reasoning`, `spec_decoding`. Don't use `mtp` — it's been renamed across the repo. - **Strategy list**: MoE recipes usually support every serving strategy; dense recipes are limited to `single_node_tp` and `multi_node_tp` (TEP/DEP require MoE). KV offload is a separate axis and is NOT listed here — Off / Simple / both Mooncake modes are implicit on every non-omni recipe and COMPOSE with whatever serving strategy is selected (each Mooncake instance runs the strategy's exact command; parallelism never comes from the KV layer). - **KV Offload gating**: fail-open. Only add `kv_cache_strategy_hardware` when a Mooncake mode is known NOT to work on a specific GPU — mark that strategy × GPU pair `unsupported`. Absence = assumed to work, same convention as `meta.hardware`. - **Variants**: quantized variants reuse the base name (`fp8`, `nvfp4`, `int4`). If the quantized checkpoint is authored by someone else (e.g. `nvidia/*-NVFP4`), set `model_id:` inside the variant. - **Tasks**: `omni` means served via vLLM-Omni (`vllm serve --omni`). Add a top-level `omni:` block listing the task ids the recipe supports — bare strings for catalog defaults (`tasks: [t2i]`) or `{ id, model_id?, vram_minimum_gb?, description?, extra_args?, curl?, benchmark? }` overrides when a task swaps the checkpoint (Wan2.2), needs per-task flags, or targets a non-LLM benchmark endpoint. `benchmark` is a static command override when the default `vllm bench serve` command is not applicable. Leave `omni.serve_binary` unset — `vllm serve --omni` is correct for every recipe on vLLM 0.20.0+, where the `vllm` console-script delegates to vllm-omni on the `--omni` flag. No recipe currently overrides it; only a recipe pinned below 0.20.0 would need to. The catalog is `src/lib/omni-tasks.js`; do not add `--omni` to `model.base_args` (auto-injected). ## Validation checklist Before committing: 1. `node scripts/build-recipes-api.mjs` succeeds and the new recipe appears in the line count. 2. `node -e "const d = require('./public//.json'); console.log(d.model.parameter_count, d.variants.default.vram_minimum_gb)"` prints sensible values. 3. Inspect `git diff --check` and confirm the YAML top-level key order matches the schema. 4. If the recipe marks any KV-store mode `unsupported` under `kv_cache_strategy_hardware`, spot-check `public///hw//strategies/`: no `kv_store_*` file may exist for an opted-out GPU (they are emitted for all other scalable hardware by default). 5. Preview the recipe route and select each authored hardware, variant, strategy, and feature. Verify the Install tabs, serve command, Verify model id, Bench model id, verified badge, and Guide links. 6. Compare the displayed command with the exact successful runtime command. Clearly label any advertised-but-untested context length or optional feature. ## Commit Stage **only** the new recipe (and providers.js if edited): ```bash git add models//.yaml src/lib/providers.js git commit -s -m "Add / recipe" ``` Do not stage `public/` (it's generated) or the design docs.