NeMo Helix retrieval recipes
Conduct the Nemotron embed/rerank pipeline on NeMo Helix plugin CLIs.
Do not use nemo data-designer create.
Plugin skills (nemo-data-designer-plugin, nemo-customizer, nemo-evals-plugin)
are stage manuals. This skill owns family routing, artifact IDs, frozen eval, and
the 15% nDCG@10 / Recall@10 README bar.
Resolve CLI
Use nemo, or uv run nemo from the nemo-helix root. If neither works, route
to setup. The CLI defaults to http://localhost:8080; NHX_BASE_URL overrides
it for a remote platform. Confirm /health/ready on the resolved base URL before
submitting jobs, and recheck it before rereading a command that returned empty or
non-JSON output — that is an unreachable platform, not a missing entity.
Choose a family
Read only the matching recipe after this routing step:
- Low Recall@k, missing relevant documents, first-stage retrieval →
references/embed.md - Recall@100 OK but nDCG@10 poor, relevant docs buried in top-k →
references/rerank.md - User asks both or a two-stage stack → embed first (candidate coverage), then rerank.
Use one workspace. Stage 1 publishes an artifacts result inside its job fileset.
Use the result's full path fragment
(workspace/job-fileset#results/<attempt>/artifacts/) as Automodel
dataset.training; keep the trailing slash so the result contents are staged at
the training dataset root. Use that same result's eval_beir subtree for
retrieve-eval.
Keep query: / passage: prefixes for embed and
question:{query} \n \n passage:{passage} for rerank.
Safe workflow
- Confirm corpus governance: Stage 0 sends chunk text to Inference Gateway chat
models. Do not ask users to paste API keys. Use
provider/chat_provider/embed_provider, notNVIDIA_API_KEYon the job. - Dry-run schemas first:
nemo data-designer retrieval-generate --help,nemo customization automodel explain,nemo evals retrieve-eval explain. - Live SDG is the default for a user corpus; drive it through Stage 0 generation
knobs (see
references/sdg.md). It needs 50+ documents, since one file can dump every query into the test split and leave train empty (mining then crashes). Reach for a published Stage 0 dump only when the user has no corpus or wants a fast first eval. - Run Stage 0+1 once (
retrieval-run, or generate then prepare). Freeze the resultingeval_beir; never regenerate it for base vs tuned comparisons. - Confirm the training model entity has a non-null
fileset. Auto-discovered Inference Gateway model entities are endpoints, not trainable checkpoints. - Before Automodel, confirm
training.jsonlhas non-emptyneg_doclists (references/sdg.md). Convert-only Stage 1 leavesneg_doc: []; the collator then crashes samplingtrain_n_passages - 1negatives (default 4). Mine first (enable_mining: true). - Fine-tune with explicit
training.recipe: bi_encoderorcross_encoderandfinetuning_typeall_weightsorlora_mergedonly. retrieve-evalneeds IGW providers on every named model (k: [1,5,10,100],baseline+ tuned). Fileset-backed Automodel entities have emptymodel_providersuntil Deployment Manager serves them — followreferences/deploy.mdbefore submit.- Serving after eval is the same deploy contract. One GPU per NIM; Helm
imagePullSecretsmust include the NGC dockerconfig or the pod cannot pullnvcr.io. - Poll jobs at 60–300s. Parse JSON from stdout only (never
2>&1intojson.load).
Report absolute and relative nDCG@10, Recall@10, and Recall@100 on the same frozen
eval set. Prefer at least 100 queries; warn below 50. Treat 15% relative nDCG@10
and Recall@10 uplift as an indicative embed target, not a hard small-corpus gate.
The validated NVDocs bi-encoder recipe in references/embed.md scores nDCG@10
0.6318 and Recall@10 0.7124 on the 20,909-query split (+11.4% and +11.7% versus
the untuned base).
References: references/sdg.md (Stage 0+1 corpus and generation control),
references/embed.md, references/rerank.md, references/deploy.md
(Retriever / Ranking NIM via Deployment Manager).
For stage-specific failures, hand off to nemo-data-designer-plugin,
nemo-customizer, or nemo-evals-plugin. NIM serving for this recipe
stays in references/deploy.md, not the inference skill.