Add Recipe

Author and validate a vLLM recipe

Recipes are YAML files at models/<hf_org>/<hf_repo>.yaml. The path mirrors HuggingFace (huggingface.co/<hf_org>/<hf_repo>), and the site/API are generated at build time from these files + taxonomy.yaml + strategies/*.yaml.

Component contract: evidence → YAML → rendered surface

Do not author the YAML as isolated documentation. Each input controls a different consumer, and a plausible guide does not prove that the command builder shows the same thing.

HF repo + config.json ───────────────► model identity, architecture, context, variants
runtime source ref + distribution ──► Install block and reproducible prerequisites
hardware profile + strategy YAML ───► selectable topology and generated parallel flags
real serving evidence ──────────────► meta.hardware verified + tested guide section
recipe YAML ─► loader ─► command-synthesis ─► CommandBuilder / Verify / Bench
            └────────────────────────► static JSON API
guide markdown ──────────────────────► Guide accordion and explicit reference links

Rendering consequences that are easy to miss

• The header's View on HuggingFace link is derived from the recipe file path, not the selected variant. If FP8/NVFP4/etc. is a different repo, link it explicitly in the Overview, its serving section, and References.
• meta.hardware is not variant-specific. A family recipe may keep the base HF path while variants.default.model_id selects a separately published preferred checkpoint; identify the tested variant explicitly in the guide so the recipe-wide badge is not mistaken for evidence about untested variants.
• variants.<key>.model_id changes the generated serve command and the Verify/Bench model id. A guide-only FP8 command does not update the interactive builder.
• dependencies[] is additive. For unreleased source support, place the complete build command in model.install.pip.command; if no image exists, set model.install.docker: false. Do not leave a generic Docker or nightly-wheel tab visible when it cannot run the model.
• nightly_required: true changes the default pip text and shows a nightly badge. It does not prove the nightly contains the needed model registry entry.
• meta.hardware.<id>: verified is a claim about the selected recipe path, variant, and working serving flow. Record the tested variant/context in the guide.
• The initial builder state should reproduce the successful baseline: default variant, default-on features, strategy_overrides.single_node_tp.tp, and base_args all affect it. Move untested features to opt_in_features and encode the tested TP/context rather than relying on auto-sizing or model-card limits.
• guide commands are not synthesized or checked against the builder. Preview both and reconcile model id, TP, flags, environment, and install method.

End-to-end steps

1. Resolve current sources before writing claims. Fetch the runtime repo and record the exact commit that contains model registration and required kernel/quantization fixes. Branch names move and similarly named branches may not contain the same code. Recheck the remote ref immediately before the final build. If support is unreleased, build from source rather than selecting an image that has not been published.

2. Confirm every HF id. Preserve the exact <org>/<repo> string. Resolve the canonical recipe repo and every alternate checkpoint independently. Verify access using the configured credential through the normal HF client; do not substitute a similarly named cached model or invent an alias.

3. Fetch model metadata. Run bash scripts/hf-info.sh <org>/<repo> or inspect the staged checkpoint's config.json / params.json. Extract:

   • architecture: moe if num_experts, num_local_experts, moe.num_experts, or a *MoE* architecture name is present. Otherwise dense.
   • the exact architectures[] / model_type; confirm the runtime resolves that architecture without a custom alias.
   • parameter_count: total params (e.g. "671B", "70B"). Use HF model card or the sum of shard sizes.
   • active_parameters: for MoE, the activated-per-token count (e.g. "37B" on DeepSeek-V3.2). For dense, equal to parameter_count.
   • context_length: max_position_embeddings from config.json (for VL models, from text_config.max_position_embeddings).
   • min_vllm_version: earliest stable support if known. For a source-only preview, also record the tested commit/version; a future target version alone is not enough.

4. Read the README — don't skip this. Run curl -sL "https://huggingface.co/<org>/<repo>/resolve/main/README.md" and scan the install / serve / usage sections in full. Configs are not enough; model authors put load-bearing requirements in prose. Mine the README for:

   • min_vllm_version / nightly_required — phrases like "install vllm nightly", "requires nightly wheels", or an install snippet using --extra-index-url https://wheels.vllm.ai/nightly mean min_vllm_version: "nightly" + nightly_required: true. A specific tag like "vLLM >= 0.12.0" sets that version. Don't default to 0.11.0 when the README says otherwise.
   • dependencies: — any pip line beyond vllm itself: version pins (mistral_common >= 1.11.1, transformers >= 5.4.0), extras (vllm[audio]), source installs (pip install git+...), DeepGEMM pins, etc. Pin them even when the README says "auto-installed" — users on stale wheel caches need an explicit upgrade path. Each entry needs a one-line note saying why.
   • Parser flags for features: — --tool-call-parser <name>, --reasoning-parser <name>, --enable-auto-tool-choice. Use the exact parser name the README specifies.
   • Companion / draft repos — EAGLE / MTP / Eagle3 heads, NVFP4 quants, instruct vs base. Wire as spec_decoding feature (draft pointer in --speculative-config) or a sibling variant with model_id: override. Copy the recommended --speculative-config JSON verbatim from the README.
   • Recommended serve flags — --tensor-parallel-size, --gpu-memory-utilization, --max-num-batched-tokens, --max-num-seqs go into the guide's launch command and into variant extra_args when they're variant-specific.
   • Hardware guidance / sampling defaults — "recommended on 8xH200" lines inform variant description + vram_minimum_gb; recommended temperature / top_p / reasoning_effort go in the guide's Client Usage block.

5. Cross-check upstream vLLM support. The README is a snapshot — if it was written at a moment when only nightly worked, that claim rots once stable ships. Never copy the README's "vLLM nightly" claim verbatim without checking. Run these in parallel:

   • gh search issues --repo vllm-project/vllm "<model-name>" --state all --limit 20 — bug reports tell you which versions users are actually running on (e.g. an issue body saying "vLLM 0.18.0 + this model crashes" is positive proof the model loads on 0.18.0).
   • gh search prs --repo vllm-project/vllm "<model-name>" --merged --limit 10 — locate the support PR; gh pr view <num> --json mergedAt gives the date, cross-reference against gh release list --repo vllm-project/vllm to find the minimum release.
   • curl the registry and supported-models docs at the candidate tag — this is the most authoritative check:

     curl -sL "https://raw.githubusercontent.com/vllm-project/vllm/<tag>/vllm/model_executor/models/registry.py" | grep -i "<arch>"
     curl -sL "https://raw.githubusercontent.com/vllm-project/vllm/<tag>/docs/models/supported_models.md" | grep -B2 -A4 "<arch>"
     

     supported_models.md often documents required flags that the model card omits — e.g. Voxtral Realtime needs --tokenizer-mode mistral per vLLM docs, but the HF README doesn't mention it. Always read this file for the recipe's target tag.
   • gh release view <tag> --repo vllm-project/vllm --json body + grep for the model name — release-note mentions confirm support officially landed.
   • For newer architectures, also search the model author's repo (e.g. PaddlePaddle/PaddleOCR, deepseek-ai/DeepSeek-VL2) for "vllm" discussions — authors often post the canonical launch command and known issues there.

   What to extract:

   • min_vllm_version — set to the lowest stable tag where the model actually works, not what the README claims. Walk forward from the support-PR's release tag, but bump up if there are known parser/tokenizer/quant bugs fixed in a later release (the v0.20.0-style "Mistral Grammar factory" / "tool parser HF-tokenizer fix" entries are signals to bump). Only use min_vllm_version: "nightly" + nightly_required: true when the registry at the latest stable tag genuinely lacks the architecture — and double-check by curling registry.py at that tag. If support is still an open issue (no PR merged), flag this to the user before authoring. For derivative releases (e.g. PaddleOCR-VL-1.5 vs 1.0) with identical architectures / model_type / auto_map, the existing handler usually loads them via --trust-remote-code even before a dedicated PR — note this assumption in your reply.
   • Required serve flags hidden in upstream docs — copy any must be served with <flag> lines from supported_models.md straight into model.base_args (and call them out in the guide's launch command). These are not optional and the README often doesn't mention them.
   • Troubleshooting — recurring errors and fixes from issue comments (e.g. "needs --enforce-eager on 0.11.x", "transformers>=5 required", "--mm-processor-cache-gb 0 to avoid OOM"). Surface these in the guide's Troubleshooting section, or as inline tips next to the launch command if they're load-bearing.
   • Links to put in guide's References — the model card, vLLM support PR (not the recipe-request issue — see below), and any author-side deployment doc. These give users a path forward if their setup breaks.

   What NOT to put in References: the recipe-request issue in vllm-project/recipes (e.g. #459) is a tracking ticket, not a user-facing reference. It belongs in the PR description body (Closes #459), never in the YAML's ## References section.

6. Stage and verify the exact checkpoint. Pin the HF revision, download the exact repo, and verify all shards/bytes. Record checkpoint id, revision, shard count, and total size when useful. A completed download is not proof of integrity.

7. Run on the target hardware. Capture device name, GPU count/topology, source fingerprint, Transformers dependency, full serve arguments, resolved architecture, weight-load completion, and meaningful warnings. Keep advertised maximum context separate from the smaller context used for a smoke test.

8. Prove inference, not merely startup. Require all of:

   • /health succeeds;
   • /v1/models returns the canonical selected model id;
   • /v1/chat/completions produces content and finish_reason: stop;
   • the response fingerprint matches the intended source build. Test multimodal, tool calling, reasoning parsing, speculative decoding, offload, long context, and other optional paths separately before calling those paths verified.

9. Translate evidence into YAML. Write models/<hf_org>/<hf_repo>.yaml following the component contract and schema below. The tested command belongs in the builder fields; the guide records exact evidence, limitations, and explicit checkpoint links.

10. Register the provider (if new). If <hf_org> isn't already in src/lib/providers.js, add display_name and /providers/<hf_org>.png (or .jpeg).

11. Validate data and rendering. Run the JSON generator, inspect the generated model JSON, then preview the page when dependencies are available. Exercise every relevant Hardware / Variant / Strategy / Feature selection and compare the generated command to the tested command. The Guide alone is not a builder preview.

12. Stage only intended source files. Stage the recipe and provider entry if needed; never stage generated public/, node_modules/, or unrelated operational artifacts. Commit only when the user asked for a commit, using the repository's DCO convention.

YAML schema (top-level fields, in order)

meta:
  title: "..."                    # display name (e.g. "DeepSeek-V3.2")
  slug: "..."                     # lowercase-kebab (legacy, keep consistent with title)
  provider: "..."                 # human-readable org label (e.g. "DeepSeek")
  description: "..."              # one-sentence summary
  date_added: YYYY-MM-DD          # initial catalog addition; never change
  date_updated: YYYY-MM-DD        # latest material recipe change
  difficulty: beginner|intermediate|advanced
  tasks:                          # one or more of: text, multimodal, omni, embedding
    - text
  performance_headline: "..."     # optional pithy line for cards
  related_recipes: []             # optional list of "<org>/<repo>" ids
  # Optional. Tri-state:
  #   `verified`    — you've run this recipe on this GPU end-to-end (green ✓).
  #   `unsupported` — not yet runnable here today (compat gap, missing kernel,
  #                   upstream blocker). Pill disabled in UI with "Not yet
  #                   supported" tooltip. May flip later — revisit on updates.
  #   absent        — silent default, assumed to work. Don't mark "untested".
  hardware:
    h200: verified
    mi355x: verified
    # mi300x: unsupported    # e.g. when a required kernel/feature is missing

model:
  model_id: "<hf_org>/<hf_repo>"  # MUST match the filename path
  min_vllm_version: "0.11.0"      # string, e.g. "0.12.0"
  # Optional — pin the Docker image shown in Install → Docker. Two forms:
  #   docker_image: "vllm/vllm-openai:glm51"      # string pins NVIDIA only;
  #                                               # AMD/TPU still use brand defaults
  #   docker_image:                               # object pins per-brand
  #     nvidia: "vllm/vllm-openai:gemma4"
  #     amd:    "vllm/vllm-openai-rocm:gemma4"
  #     tpu:    "vllm/vllm-tpu:gemma4"
  # Missing keys fall back to `:latest` for that brand (vllm/vllm-openai,
  # vllm/vllm-openai-rocm, vllm/vllm-tpu). Use the object form when CUDA /
  # ROCm / TPU ship different pinned tags for the same recipe.
  docker_image: ""
  # Optional — set true when `min_vllm_version` hasn't shipped as a stable
  # release yet. Swaps the default pip command to nightly wheels
  # (https://wheels.vllm.ai/nightly/cu130) and adds a yellow "nightly" pill
  # to the Install header. Manual install.pip overrides still win.
  nightly_required: false
  # Optional — control the Install block's pip/Docker tabs. Each key accepts
  # `false` (hide the tab entirely) OR an object `{ command?, note? }` to
  # override the generated one-liner and/or show a note above it.
  #   install.pip: false                 → no wheel available, Docker only
  #   install.docker: false              → no published image, pip only
  #   install.pip.command: "..."         → replace the pip command
  #   install.pip.note: "..."            → one-liner above the code block
  # For unreleased source-only support, put the complete clone/checkout/build
  # sequence in install.pip.command and set install.docker: false.
  # Tab ORDER follows the YAML key order — put `docker` first to make it the
  # default tab when Docker is the recommended install path.
  install:
    pip:
      command: ""
      note: ""
    docker:
      note: ""
  architecture: dense|moe
  parameter_count: "30B"          # string with suffix (B or T)
  active_parameters: "30B"        # same as parameter_count for dense models
  context_length: 131072          # integer (tokens)
  base_args: []                   # flags always needed (trust-remote-code, etc.)
  base_env: {}                    # env vars always needed

# Optional — only if the recipe needs extra pip installs beyond `uv pip install -U vllm`.
# Rendered as an "extra install" block above the vllm serve command.
dependencies:
  - note: "Why you need it (one line)"
    command: 'uv pip install -U "vllm[audio]"'
    optional: false               # omit or false for required; true = dimmed in UI + excluded from "Copy all"
    brand: NVIDIA                 # optional: NVIDIA | AMD | Intel (or array). Omit for platform-agnostic deps.
                                  # Use this for CUDA-only kernels (xformers/DeepGEMM) or ROCm-only wheels —
                                  # the dep is hidden when the user picks a hardware pill of a different brand.

features:
  tool_calling:                   # flip any of these pills; recipe chooses naming
    description: "..."
    args: ["--enable-auto-tool-choice", "--tool-call-parser", "<name>"]
  reasoning:
    description: "..."
    args: ["--reasoning-parser", "<name>"]
  spec_decoding:                  # USE spec_decoding, NOT mtp — unified key for
    description: "..."            # MTP / Eagle3 / ERNIE-MTP / etc.
    args: ["--speculative-config", '{"method":"mtp","num_speculative_tokens":1}']

opt_in_features:                  # features that default OFF (users tick them on)
  - spec_decoding                 # spec decoding is opt-in unless the model docs insist

variants:
  default:                        # ALWAYS include a `default` variant
    precision: bf16|fp8|nvfp4|fp4|int4|int8|awq|gptq|mxfp4
    vram_minimum_gb: <integer>    # params × bytes × 1.2 (see formula below)
    description: "..."
  fp8:                            # optional extra variants
    model_id: "<optional override>"   # only if the quantized variant is a different HF repo
    precision: fp8
    vram_minimum_gb: <integer>
    description: "..."
    supported_hardware: [mi355x]  # optional exact hardware-profile allowlist
    extra_args: []
    extra_env: {}

compatible_strategies:            # subset of the SERVING strategies in strategies/*.yaml
  - single_node_tp                # always include this as a baseline
  - single_node_tep               # for MoE
  - single_node_dep               # for MoE
  - multi_node_tp
  - multi_node_tp_pp              # TP within node + PP across nodes
  - multi_node_dep                # for MoE
  - multi_node_tep                # for MoE
  - pd_cluster                    # only if the recipe documents PD
  # Do NOT list kv_store_* ids — the KV Offload options (Simple + both
  # Mooncake modes) are implicit on every non-omni recipe.

# Optional opt-OUT for the Mooncake pills on the command builder's
# "KV Offload" row. Fail-open like meta.hardware: absent = assumed to work
# (pill enabled on any scalable GPU); `unsupported` disables the pill and
# makes the JSON API skip that strategy on that hardware. Off and Simple
# need no gating (Simple is taxonomy-driven — `taxonomy.yaml →
# kv_offload.simple` — and needs nothing per-recipe).
kv_cache_strategy_hardware:
  kv_store_distributed_mooncake:
    gb200: unsupported

hardware_overrides:               # optional per-generation flags
  hopper:    { extra_args: [], extra_env: {} }
  blackwell: { extra_args: [], extra_env: {} }
  amd:       { extra_args: [], extra_env: {} }

strategy_overrides:               # optional per-strategy tweaks
  single_node_tp:
    tp: 1                         # optional — default TP size for this strategy.
                                  # Lets a small model run below full-node TP
                                  # (e.g. Gemma 4 fits on 1 GPU → tp: 1). Omit
                                  # to default to the node's gpu_count. Clamped
                                  # to [1, gpu_count]. When effective TP <
                                  # gpu_count the UI shows a "using N of M GPUs"
                                  # hint under the Hardware pill. TEP/DEP and
                                  # multi-node ignore `tp:` (topology requires
                                  # full pool).
    extra_args: []
    extra_env: {}

guide: |                          # markdown, rendered as the Guide accordion
  ## Overview
  ...
  ## Prerequisites
  ...
  ## Launch command
  ...
  ## Benchmarking
  ...
  ## References
  - [Model card](https://huggingface.co/<hf_org>/<hf_repo>)

VRAM formula

vram_minimum_gb = ceil(params × bytes_per_param × 1.2) where params is the total parameter count (MoE includes inactive experts — they still live in VRAM).

Example: a 70B BF16 model → 70 × 2 × 1.2 = 168 GB. Round up.

If the variant is model_id-overridden and the override is a different base model with its own param count (e.g. a distilled FP4 checkpoint), use the override's parameter count — verify it via HF.

Mixed-precision quants (NVFP4 / ModelOpt) — don't trust the bytes-per-param table. NVIDIA ModelOpt NVFP4 checkpoints are not uniformly 4-bit: only the MLP linears drop to NVFP4 (W4A16), while attention linears + KV cache stay FP8 and embeddings/norms stay higher precision. hf_quant_config.json shows quant_algo: MIXED_PRECISION in this case. The pure params × 0.5 × 1.2 formula then underestimates — e.g. nvidia/Qwen3.6-27B-NVFP4 is ~21.9 GB on disk, not the 13.5 GB the table implies, so 27B × 0.5 × 1.2 = 17 is wrong (the weights alone exceed it). For any mixed-precision checkpoint, size from the real weight footprint instead:

# total_size is in bytes → GB; then × 1.2 for KV/activation overhead
curl -sL "https://huggingface.co/<org>/<repo>/resolve/main/model.safetensors.index.json" \
  | python3 -c "import json,sys; print(round(json.load(sys.stdin)['metadata']['total_size']/1e9*1.2))"

So vram_minimum_gb = ceil(real_checkpoint_GB × 1.2) (Qwen3.6-27B-NVFP4 → ceil(21.9 × 1.2) = 27). The bytes-per-param table stays correct for uniform quants (plain int4/awq/gptq/fp8, and full-model FP4).

Naming and conventions

• Feature keys: prefer tool_calling, reasoning, spec_decoding. Don't use mtp — it's been renamed across the repo.
• Strategy list: MoE recipes usually support every serving strategy; dense recipes are limited to single_node_tp and multi_node_tp (TEP/DEP require MoE). KV offload is a separate axis and is NOT listed here — Off / Simple / both Mooncake modes are implicit on every non-omni recipe and COMPOSE with whatever serving strategy is selected (each Mooncake instance runs the strategy's exact command; parallelism never comes from the KV layer).
• KV Offload gating: fail-open. Only add kv_cache_strategy_hardware when a Mooncake mode is known NOT to work on a specific GPU — mark that strategy × GPU pair unsupported. Absence = assumed to work, same convention as meta.hardware.
• Variants: quantized variants reuse the base name (fp8, nvfp4, int4). If the quantized checkpoint is authored by someone else (e.g. nvidia/*-NVFP4), set model_id: inside the variant.
• Tasks: omni means served via vLLM-Omni (vllm serve <model> --omni). Add a top-level omni: block listing the task ids the recipe supports — bare strings for catalog defaults (tasks: [t2i]) or { id, model_id?, vram_minimum_gb?, description?, extra_args?, curl?, benchmark? } overrides when a task swaps the checkpoint (Wan2.2), needs per-task flags, or targets a non-LLM benchmark endpoint. benchmark is a static command override when the default vllm bench serve command is not applicable. Leave omni.serve_binary unset — vllm serve <model> --omni is correct for every recipe on vLLM 0.20.0+, where the vllm console-script delegates to vllm-omni on the --omni flag. No recipe currently overrides it; only a recipe pinned below 0.20.0 would need to. The catalog is src/lib/omni-tasks.js; do not add --omni to model.base_args (auto-injected).

Validation checklist

Before committing:

1. node scripts/build-recipes-api.mjs succeeds and the new recipe appears in the line count.
2. node -e "const d = require('./public/<hf_org>/<hf_repo>.json'); console.log(d.model.parameter_count, d.variants.default.vram_minimum_gb)" prints sensible values.
3. Inspect git diff --check and confirm the YAML top-level key order matches the schema.
4. If the recipe marks any KV-store mode unsupported under kv_cache_strategy_hardware, spot-check public/<hf_org>/<hf_repo>/hw/<gpu>/strategies/: no kv_store_* file may exist for an opted-out GPU (they are emitted for all other scalable hardware by default).
5. Preview the recipe route and select each authored hardware, variant, strategy, and feature. Verify the Install tabs, serve command, Verify model id, Bench model id, verified badge, and Guide links.
6. Compare the displayed command with the exact successful runtime command. Clearly label any advertised-but-untested context length or optional feature.

Commit

Stage only the new recipe (and providers.js if edited):

git add models/<hf_org>/<hf_repo>.yaml src/lib/providers.js
git commit -s -m "Add <hf_org>/<hf_repo> recipe"

Do not stage public/ (it's generated) or the design docs.