# Cosmos3 Inference

> Guide users through running Cosmos3 inference — offline batch generation, online serving with Ray and Gradio, parallelism options, input formats, sampling parameters, and prompt upsampling. Use when the user asks "how do I run inference", "how do I generate a video", "how do I serve the model", "what parameters should I use", or any question about running the model to produce outputs.

- Skill: `nvidia/cosmos3-inference` (Agent Skill)
- Install (CLI): `npx skillmds@latest add nvidia/cosmos3-inference`
- Raw SKILL.md: https://api.skillmd.com/api/skills/nvidia/cosmos3-inference/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: NVIDIA (https://skillmd.com/u/nvidia)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/nvidia/cosmos3-inference

---


# Cosmos3 Inference

## When to use this skill

- Use when a user wants to generate images or videos with Cosmos3
- Use when a user asks about inference parameters, input formats, or parallelism
- Use when a user wants to set up online serving (Ray Serve, Gradio)
- Use when a user asks about prompt engineering or upsampling
- For environment or import errors, hand off to **cosmos3-env-troubleshoot**

## Path convention

All paths below are relative to the cosmos3 package root (`../../../` from this skill file). All `uv run` / `python` commands should also be run from there.

## Where to find answers

| User question                                                           | Go to                                                                                   |
| ----------------------------------------------------------------------- | --------------------------------------------------------------------------------------- |
| How do I run inference? (single-GPU, multi-GPU)                         | `README.md` § Inference                                                                 |
| Which model should I use? (Nano vs Super, memory, shift)                | `README.md` § Models                                                                    |
| Which modality? (t2i, t2v, i2v, examples)                               | `README.md` § Modalities                                                                |
| What parallelism preset? (latency vs throughput)                        | `README.md` § Inference                                                                 |
| What input fields are available? (prompt, vision_path, num_frames, ...) | `docs/inference.md` § Sample Arguments                                                  |
| What are the default parameter values?                                  | `cosmos_framework/inference/defaults/<model_mode>/sample_args.json` (per-modality JSON) |
| How do I use custom defaults?                                           | `docs/inference.md` § Custom Defaults                                                   |
| How do I override a parameter? (precedence)                             | `docs/faq.md` § How do I override a default parameter?                                  |
| What is the shift parameter?                                            | `docs/faq.md` § What is the `shift` parameter?                                          |
| How many frames can I generate? (resolution caps)                       | `docs/faq.md` § How many frames can I generate?                                         |
| How do I start Ray Serve / Gradio / submit requests?                    | `docs/faq.md` § How do I run online inference with Ray?                                 |
| How do I upsample short prompts?                                        | `docs/faq.md` § Prompt upsampling                                                       |
| How do I use the low-level API? (examples/)                             | `examples/inference.py` (model API) / `examples/inference_pipeline.py` (pipeline API)   |
| All CLI flags                                                           | `uv run --all-extras --group=cu130 python -m cosmos_framework.scripts.inference --help` |

## Things not obvious from the docs

- **Path resolution**: relative paths in input JSON files are resolved relative to the **JSON file's directory**, not the working directory.
- **Seed**: always pass `--seed` for reproducible results. Without it, a random seed is used each time.
- **Resume**: interrupted runs can be resumed by re-running the same command — existing outputs are skipped automatically.
- **`--keep-going`**: continues processing remaining samples after a per-sample failure (e.g. guardrail rejection). Used in online serving by default.
- **Unique names**: every sample in a run must have a unique `name` field, or the script will error.

## Related skills

| Skill                                  | When to use                                    |
| -------------------------------------- | ---------------------------------------------- |
| `../cosmos3-setup/SKILL.md`            | Installation and environment setup             |
| `../cosmos3-codebase-nav/SKILL.md`     | Finding files, parameters, and configs in code |
| `../cosmos3-env-troubleshoot/SKILL.md` | Debugging environment and runtime errors       |