Inference

End-to-end reference for inference on the NeMo Helix — registering LLM backends as ModelProviders, wiring them to VirtualModels with Switchyard middleware (random routing, stage routing, capability classification) and `nemo-guardrails` content-safety rails, and hitting them via the nemo CLI. Use when the task involves registering inference providers, discovering served models, creating VirtualModels, configuring switchyard middleware, layering guardrails alongside routing, making OpenAI Chat Completions calls through IGW, or debugging routing failures locally. For platform startup, Switchyard install, and DB-reset prerequisites, see the setup playbook (`SETUP.md` at the repo root).

NVIDIA NeMo Updated 2.3k repo stars

File contents

nvidia-nemo/nemo-platform commit 764e689198

Frequently asked questions

npx get-skill add nvidia-nemo/inference-5