CDF Curator Operator Skill
GPU-accelerated video and image curation via NVIDIA Cosmos Curator inside
Cosmos Data Factory — Curation and Retrieval
(cdf-curation-and-retrieval). This skill is a short Curator index.
Embedding handoff boundaries live in
data-mining.md and
curation-retrieval-workflow.md;
mining execution is make help and the repository cookbooks.
- Video:
split, dedup, shard.
- Image:
annotate (load → filter → embed → caption → write).
- Handoff: Curator IV2 or CE1 parquet that downstream mining can consume.
See data-mining.md and
curation-retrieval-workflow.md.
- CDS stitch: Curator → CDS ingest/search. Operator Path D (public GA
1.2.0; not supported on Blackwell GPUs): see
cosmos-dataset-search.md.
Purpose
Turn raw video and image collections into curated, training-ready datasets. This
skill configures and runs cosmos-curator pipelines (clip splitting, filtering,
captioning, embeddings, SAM3 event verification, dedup, WebDataset sharding,
image annotate) and supports KPI-driven, distribution-aware, and restrictive
curation.
Instructions
- Classify the request as advisory, config, run, TAO handoff, or CDS
ingest/search. Do not mix those routes. A config request with no KPI output
or samples goes through the calibration interview, which can stop with
unresolved requirements before any YAML is written.
- For config work, complete the mandatory pre-flight below before writing YAML.
- For an explicit run, load
running-pipelines.md, validate the config,
obtain credentials only through approved injection, then execute after
authorization.
- For a TAO handoff, validate Curator output and the declared embedding family
using data-mining.md before mining.
- For CDS ingest or search, confirm CDS is healthy first, then follow
cosmos-dataset-search.md.
- Return the Output Format below. Never print secret values.
Examples
- Advisory: "How much SHM should I set?" → read
running-pipelines.md and report guidance.
Do not run Docker.
- Config with no KPI: complete the calibration interview in
calibration-config.md, then emit YAML.
- Run: after sample clips are staged,
make run-pipeline with the traffic
split-minimal cookbook recipe only after preflight and user authorization.
- FFmpeg missing in the container: install the host sidecar
(
make ffmpeg-install) per
ffmpeg-sidecar.md.
Inputs
Required inputs depend on the route:
- Advisory request: the question plus relevant repository and config context.
- Config request: input and output locations, domain and goal, available KPI
output or representative samples, and hardware constraints. If neither KPI
output nor samples exist, complete the calibration interview before writing
YAML.
- Run request: reviewed config path, data and model paths, runtime, GPU,
and SHM constraints, and explicit authorization to execute.
- TAO handoff: validated artifact paths and declared embedding family.
Optional inputs include target distributions, event taxonomy, prompt choices,
existing output metadata, and user-approved operational constraints.
Resolve inputs in this order: repository configuration and validated run
artifacts; explicit prompt arguments and corrections; available agent context;
then the broad user prompt. Explicit user instructions remain authoritative
unless unsafe or incompatible, in which case stop and explain the conflict.
Never infer secret values: credentials come only from approved runtime
injection.
Prerequisites
- Repository access: the agent must be able to read the Makefile, the
configs/ and cookbook/ trees, and every file under references/.
- GPU host with NVIDIA drivers +
nvidia-container-toolkit; Docker. SHM
sized from host RAM (SHM_SIZE, default 24gb).
cosmos-curator image: make pull uses the pin configured by the
example env file and Make. No separate product engine image. Source
builds are developer-only; see
cosmos-curator.md.
- FFmpeg host sidecar for distributable images (
make ffmpeg-install) —
they do not bundle FFmpeg. See
ffmpeg-sidecar.md.
- Credentials as needed: inject S3 and captioning API keys at runtime
through an approved secret manager or operator deployment mechanism. Never
put secret values in repository files, commands, logs, or examples. The env
file is for non-secret image and CDS profile overrides copied from the
example env file.
Mandatory pre-flight: do NOT emit a pipeline config without context
Before writing any *.yaml pipeline config, the agent MUST verify
that one of the following is true:
- KPI run output exists -- read it and use
distribution-analysis.md,
distribution-aware-curation.md,
and
configuration-decision-tree.md.
- KPI sample videos are available for inspection / discovery --
follow context-understanding.md
Phase 1.
- No KPI of any kind -- no baseline exists. Read
calibration-config.md and complete its
Phase 1 interview (Inputs / Domain / Goal / Hardware / Calibration)
BEFORE emitting a config. The interview is binding, not advisory.
If none of these holds, or required paths, intent, or hardware constraints
remain unresolved, stop and report the unresolved requirements as blocked.
Workplace legal-review gate. Workplace safety and employee conduct monitoring
are privacy-sensitive. The gate applies when the user names either domain or a
catalog match selects employee_conduct_monitoring.yaml:
- If the gate does not apply → continue with the route below.
- If it applies and the user confirms legal review, a lawful basis, and any
required worker notice → emit the reviewed configuration and its
calibration disclosure.
- If it applies and any of those confirmations is absent → do not write a
config; return
blocked with the evidence and the next safe step (obtain
the missing confirmation).
If the user requests a config with only a one-line description
("configure cosmos-curator for my videos"), assume the calibration
workflow and ask the Phase 1 interview questions in one batched
message. Emit the config only after the answers come back, and
always include the calibration disclosure table that flags every
defaulted field.
Canonical Flow
Choose one route; do not collapse advisory and execution branches. Every stop
or failed gate below returns the Output Format with status blocked, the
validation evidence, and the next safe step.
- Advisory only (sizing, monitoring, troubleshooting, expected commands):
inspect repository, config, and run evidence, load
running-pipelines.md, and report guidance.
Do not prepare credentials or execute.
- Create or change config:
- KPI output exists → analyze it, choose standard, distribution-aware, or
restrictive curation, then emit a reviewed config.
- Representative samples exist → inspect them before selecting defaults.
A discovery run starts a pipeline, so it follows the explicit run route,
including its preflights and authorization.
- Neither exists → complete the binding calibration interview
(calibration-config.md); emit config
and disclosure only after answers. Catalog matching follows
context-understanding.md.
- Catalog match is
employee_conduct_monitoring.yaml, on any of the paths
above → apply the workplace legal-review gate. Without the confirmation,
do not select the catalog; return blocked.
Stop if required paths, intent, or hardware constraints remain unresolved,
or if the workplace legal-review gate above is not confirmed.
- Explicit run request, including discovery and calibration runs, in
this order:
- Prepare the runtime (Docker, GPU, SHM, FFmpeg sidecar) and run the
execution preflights in Validation.
- Obtain credentials through approved injection.
- Validate the config as described in Execution & Troubleshooting.
- If the config came from the calibration workflow, confirm the user has
reviewed its calibration disclosure, including every defaulted field.
If not, return
blocked with the next step "review the disclosure".
- Request approval if not already granted. If approval is granted,
execute. If it is pending or denied, return
blocked without executing.
- Validate outputs.
- Downstream TAO handoff: validate Curator output and embedding family.
If validation succeeds, prepare compatible inputs for Data Mining and mine.
If it fails, return
blocked without mining.
- CDS ingest or search: CDS is deployed outside this repository. Run
make ds-health first. If CDS is not deployed or not healthy, return
blocked with the next step "deploy CDS per the user guide's Cosmos
Dataset Search page". Otherwise follow
cosmos-dataset-search.md: ingest CE1
artifacts only, address collections by id rather than name, and treat
HTTP 404 from /insert-data as a failure. Ingest and bulk insert change a
collection, so request authorization first, as for a run.
Configs are flat YAML with pipeline: split|dedup|shard|annotate and upstream
snake_case argument names. Operator first-run recipes live under the
cookbook tree (split-minimal then full split, dedup, and shard YAML).
The configs directory is the full flag reference and the Makefile default when
CONFIG_FILE is omitted. split writes clips, metadata, and embeddings;
dedup consumes embeddings; shard writes WebDataset archives; annotate
processes still images (image annotate flag-reference YAML; no image cookbook).
Execution & Troubleshooting
For an explicit run, troubleshooting request, or operational question, read
running-pipelines.md. The local run
commands are below.
MANDATORY: run the config preflight before every make run-pipeline or
make run_image_pipeline invocation. It is not optional, because it is the
only guard against silently skipping SAM3 on the Make path.
make run-pipeline CONFIG_FILE=<split-config>
make run_image_pipeline IMAGE_CONFIG_FILE=<image-config>
In the config preflight, reject deprecated enable_sam3 and
enable_event_captioning and return blocked; use canonical sam3 and
event_captioning. make run-pipeline passes the config straight to upstream
Curator, which silently ignores those keys: SAM3 never runs and the run still
reports success. The helper CLI's curator-run command enforces the same
rejection with a nonzero exit, checking both Curator-supported config layouts
(flat parameters or parameters nested under args) before constructing the
Docker runner. Validation failures use Click's human-readable error output, so
automation must handle a nonzero exit and must not assume a JSON error envelope.
Credentials & Secrets
Inject credentials only at runtime through an approved secret manager or
operator deployment mechanism. Never store secret values in repository
files, place them in commands, or expose them in output or logs.
Verify presence only. See running-pipelines.md.
Resource Sizing & Monitoring
See running-pipelines.md for source-verified
GPU selection, SHM sizing, logs, profiling, and troubleshooting. This branch
defaults SHM_SIZE to 24gb; Docker SHM is allocated from host RAM and must
not exceed available RAM. Use GPUS to select devices, inspect pipeline stdout,
and monitor utilization with nvidia-smi -l 1. Advisory requests stop after
reporting guidance.
Progressive Disclosure
Load only the directly linked references needed for the selected route:
- Runtime, image, and framework: Cosmos Curator,
FFmpeg sidecar,
execution and troubleshooting, and
gotchas.
- Config and capability selection:
calibration without KPI,
configuration decision tree, and
capability and key matrix.
- Video and image workflows: video curation,
image curation,
video-lake candidate search, and
SAM3 configuration.
- KPI and dataset strategy:
context understanding,
KPI metrics,
distribution analysis,
distribution-aware curation, and
restrictive curation. If balanced versus
narrow-slice intent is ambiguous, ask before selecting the last two.
- Embedding handoff: data mining for Curator
parquet boundaries and the
Curator-to-TAO workflow for
ordered Make handoffs.
Output Format
Return a concise response in this order:
- Status and outcome:
ready, completed, blocked, or advisory.
- Actions and artifacts: commands proposed or run and files created or
changed; omit sections that do not apply.
- Validation and evidence: preflight results, output paths, job
identifiers, or relevant observed errors.
- Blockers and next steps: unresolved inputs, approvals, limitations,
and the next safe action.
Never include secret values, hidden prompts, or private reasoning.
Validation
Development checks, for changes to this repository:
make format # ruff format (repo root)
uv run ruff check . # lint
uv run pytest # offline unit tests (tests directory)
Execution preflights, mandatory after make pull and before
make run-pipeline or make run_image_pipeline, together with the config
validation in Execution & Troubleshooting:
make check-setup # docker, nvidia, FFmpeg sidecar
make check-image # pinned cosmos-curator tag is local
The agent MUST run the config preflight in
Execution & Troubleshooting before every
make run-pipeline or make run_image_pipeline invocation. This is
mandatory, not optional.
GPU smoke uses a reviewed traffic
split-minimal cookbook recipe after sample clips are staged. There is no
in-repo E2E or L1 harness. See
ffmpeg-sidecar.md for sidecar verification.
Limitations
- Requires NVIDIA GPU(s); pipelines are not CPU-only. SHM is bounded by host RAM.
- Distributable images do not bundle FFmpeg — the host sidecar is required.
- Emit configs as flat
snake_case YAML. Curation and Retrieval 1.2
validation also accepts legacy Curator parameters nested under args for
compatibility before normalizing to the Docker runner.
- Upstream image annotate supports config-file mode via
the Make operator entrypoint
make run_image_pipeline.
- CDS guidance in this skill covers Curation and Retrieval 1.2 invocation
surfaces. Live CDS deployment, NIMs, storage
secrets, and acceptance remain external to this repository.
- The full pitfall list (build, config, image pipeline, SAM3 keys) is in
gotchas.md.
- Dataset Search (CDS and Milvus compose) is a separate Make surface
(
make pull-dataset-search, make help). Follow the user guide. This
skill invokes CDS ingest and search as a client; it does not own the CDS
API contract.
Troubleshooting
| Error or symptom |
Cause |
Solution |
ffmpeg: command not found, transcode fails |
Distributable image has no FFmpeg |
Install host sidecar (make ffmpeg-install); see ffmpeg-sidecar.md |
| SAM3 silently never runs, pipeline "succeeds" |
Wrong YAML key enable_sam3: reached make run-pipeline, which does not validate keys |
Reject it in config preflight. Use canonical sam3: and event_captioning: — see sam3-config.md, gotchas.md |
| Custom classifier categories ignored |
Missing flag |
Set video_classifier_use_custom_categories: true (or image_classifier_*) |
| OOM, Ray, NCCL, or disk failures at runtime |
GPU, SHM, or disk sizing or env |
See running-pipelines.md (GPU allocation, SHM, S3, monitoring) |
| Automation cannot parse a validation failure |
Click prints a human-readable error and exits nonzero; there is no JSON envelope |
Check the exit code. Treat deprecated-key or missing-path errors as config errors: fix the config instead of retrying. See gotchas.md |
| Shard run mismatches split output |
captioning_algorithm differs |
Match the shard captioning_algorithm to the split run |
1---2name: cdf-curation-and-retrieval3description: >-4 NVIDIA CDF curation and retrieval. Cosmos Curator video and image5 pipelines (split, dedup, shard, annotate) with filtering, captioning, and6 embedding. Cosmos Dataset Search ingest, search, and capability7 qualification. TAO data mining embeddings and neighbor matching. Ships8 domain event catalogs, including workplace employee conduct monitoring,9 which requires legal review before use. Use when a request names Cosmos10 Curator, Cosmos Dataset Search, CDF, or their Make and CLI surfaces.11 Not for model training, orchestration, or unrelated dataset questions.12license: CC-BY-4.0 AND Apache-2.013owner: NVIDIA14service: cosmos-data-factory15reviewed: 2026-10-2016metadata:17 source: https://skillmd.com/skills/nvidia/cdf-curation-and-retrieval18 author: "NVIDIA <opensource@nvidia.com>"19 version: 1.2.020 tags:21 - data-curation22 - dataset-retrieval23 - cosmos-curator24 - tao25 - physical-ai26---2728# CDF Curator Operator Skill2930GPU-accelerated video and image curation via NVIDIA Cosmos Curator inside31**Cosmos Data Factory — Curation and Retrieval**32(`cdf-curation-and-retrieval`). This skill is a short Curator index.33Embedding handoff boundaries live in34[data-mining.md](references/data-mining.md) and35[curation-retrieval-workflow.md](references/curation-retrieval-workflow.md);36mining execution is `make help` and the repository cookbooks.3738- **Video**: `split`, `dedup`, `shard`.39- **Image**: `annotate` (load → filter → embed → caption → write).40- **Handoff**: Curator IV2 or CE1 parquet that downstream mining can consume.41 See [data-mining.md](references/data-mining.md) and42 [curation-retrieval-workflow.md](references/curation-retrieval-workflow.md).43- **CDS stitch**: Curator → CDS ingest/search. Operator Path D (public GA44 `1.2.0`; not supported on Blackwell GPUs): see45 [cosmos-dataset-search.md](references/cosmos-dataset-search.md).4647## Purpose4849Turn raw video and image collections into curated, training-ready datasets. This50skill configures and runs cosmos-curator pipelines (clip splitting, filtering,51captioning, embeddings, SAM3 event verification, dedup, WebDataset sharding,52image annotate) and supports KPI-driven, distribution-aware, and restrictive53curation.5455## Instructions56571. Classify the request as advisory, config, run, TAO handoff, or CDS58 ingest/search. Do not mix those routes. A config request with no KPI output59 or samples goes through the calibration interview, which can stop with60 unresolved requirements before any YAML is written.612. For config work, complete the mandatory pre-flight below before writing YAML.623. For an explicit run, load63 [running-pipelines.md](references/running-pipelines.md), validate the config,64 obtain credentials only through approved injection, then execute after65 authorization.664. For a TAO handoff, validate Curator output and the declared embedding family67 using [data-mining.md](references/data-mining.md) before mining.685. For CDS ingest or search, confirm CDS is healthy first, then follow69 [cosmos-dataset-search.md](references/cosmos-dataset-search.md).706. Return the Output Format below. Never print secret values.7172## Examples7374- Advisory: "How much SHM should I set?" → read75 [running-pipelines.md](references/running-pipelines.md) and report guidance.76 Do not run Docker.77- Config with no KPI: complete the calibration interview in78 [calibration-config.md](references/calibration-config.md), then emit YAML.79- Run: after sample clips are staged, `make run-pipeline` with the traffic80 `split-minimal` cookbook recipe only after preflight and user authorization.81- FFmpeg missing in the container: install the host sidecar82 (`make ffmpeg-install`) per83 [ffmpeg-sidecar.md](references/ffmpeg-sidecar.md).8485## Inputs8687Required inputs depend on the route:8889- **Advisory request:** the question plus relevant repository and config context.90- **Config request:** input and output locations, domain and goal, available KPI91 output or representative samples, and hardware constraints. If neither KPI92 output nor samples exist, complete the calibration interview before writing93 YAML.94- **Run request:** reviewed config path, data and model paths, runtime, GPU,95 and SHM constraints, and explicit authorization to execute.96- **TAO handoff:** validated artifact paths and declared embedding family.9798Optional inputs include target distributions, event taxonomy, prompt choices,99existing output metadata, and user-approved operational constraints.100101Resolve inputs in this order: repository configuration and validated run102artifacts; explicit prompt arguments and corrections; available agent context;103then the broad user prompt. Explicit user instructions remain authoritative104unless unsafe or incompatible, in which case stop and explain the conflict.105Never infer secret values: credentials come only from approved runtime106injection.107108## Prerequisites109110- **Repository access**: the agent must be able to read the Makefile, the111 `configs/` and `cookbook/` trees, and every file under `references/`.112- **GPU host** with NVIDIA drivers + `nvidia-container-toolkit`; Docker. SHM113 sized from host RAM (`SHM_SIZE`, default 24gb).114- **`cosmos-curator` image**: `make pull` uses the pin configured by the115 example env file and Make. No separate product engine image. Source116 builds are developer-only; see117 [cosmos-curator.md](references/cosmos-curator.md).118- **FFmpeg host sidecar** for distributable images (`make ffmpeg-install`) —119 they do not bundle FFmpeg. See120 [ffmpeg-sidecar.md](references/ffmpeg-sidecar.md).121- **Credentials** as needed: inject S3 and captioning API keys at runtime122 through an approved secret manager or operator deployment mechanism. Never123 put secret values in repository files, commands, logs, or examples. The env124 file is for non-secret image and CDS profile overrides copied from the125 example env file.126127## Mandatory pre-flight: do NOT emit a pipeline config without context128129Before writing any `*.yaml` pipeline config, the agent MUST verify130that one of the following is true:1311321. **KPI run output exists** -- read it and use133 [distribution-analysis.md](references/distribution-analysis.md),134 [distribution-aware-curation.md](references/distribution-aware-curation.md),135 and136 [configuration-decision-tree.md](references/configuration-decision-tree.md).1372. **KPI sample videos are available** for inspection / discovery --138 follow [context-understanding.md](references/context-understanding.md)139 Phase 1.1403. **No KPI of any kind** -- no baseline exists. Read141 [calibration-config.md](references/calibration-config.md) and complete its142 Phase 1 interview (Inputs / Domain / Goal / Hardware / Calibration)143 BEFORE emitting a config. The interview is binding, not advisory.144145If none of these holds, or required paths, intent, or hardware constraints146remain unresolved, stop and report the unresolved requirements as `blocked`.147148Workplace legal-review gate. Workplace safety and employee conduct monitoring149are privacy-sensitive. The gate applies when the user names either domain or a150catalog match selects `employee_conduct_monitoring.yaml`:151152- If the gate does not apply → continue with the route below.153- If it applies and the user confirms legal review, a lawful basis, and any154 required worker notice → emit the reviewed configuration and its155 calibration disclosure.156- If it applies and any of those confirmations is absent → do not write a157 config; return `blocked` with the evidence and the next safe step (obtain158 the missing confirmation).159160If the user requests a config with only a one-line description161("configure cosmos-curator for my videos"), assume the calibration162workflow and ask the Phase 1 interview questions in one batched163message. Emit the config only after the answers come back, and164always include the calibration disclosure table that flags every165defaulted field.166167## Canonical Flow168169Choose one route; do not collapse advisory and execution branches. Every stop170or failed gate below returns the Output Format with status `blocked`, the171validation evidence, and the next safe step.1721731. **Advisory only** (sizing, monitoring, troubleshooting, expected commands):174 inspect repository, config, and run evidence, load175 [running-pipelines.md](references/running-pipelines.md), and report guidance.176 Do not prepare credentials or execute.1772. **Create or change config**:178 - KPI output exists → analyze it, choose standard, distribution-aware, or179 restrictive curation, then emit a reviewed config.180 - Representative samples exist → inspect them before selecting defaults.181 A discovery run starts a pipeline, so it follows the explicit run route,182 including its preflights and authorization.183 - Neither exists → complete the binding calibration interview184 ([calibration-config.md](references/calibration-config.md)); emit config185 and disclosure only after answers. Catalog matching follows186 [context-understanding.md](references/context-understanding.md).187 - Catalog match is `employee_conduct_monitoring.yaml`, on any of the paths188 above → apply the workplace legal-review gate. Without the confirmation,189 do not select the catalog; return `blocked`.190 Stop if required paths, intent, or hardware constraints remain unresolved,191 or if the workplace legal-review gate above is not confirmed.1923. **Explicit run request**, including discovery and calibration runs, in193 this order:194 1. Prepare the runtime (Docker, GPU, SHM, FFmpeg sidecar) and run the195 execution preflights in [Validation](#validation).196 2. Obtain credentials through approved injection.197 3. Validate the config as described in Execution & Troubleshooting.198 4. If the config came from the calibration workflow, confirm the user has199 reviewed its calibration disclosure, including every defaulted field.200 If not, return `blocked` with the next step "review the disclosure".201 5. Request approval if not already granted. If approval is granted,202 execute. If it is pending or denied, return `blocked` without executing.203 6. Validate outputs.2044. **Downstream TAO handoff**: validate Curator output and embedding family.205 If validation succeeds, prepare compatible inputs for Data Mining and mine.206 If it fails, return `blocked` without mining.2075. **CDS ingest or search**: CDS is deployed outside this repository. Run208 `make ds-health` first. If CDS is not deployed or not healthy, return209 `blocked` with the next step "deploy CDS per the user guide's Cosmos210 Dataset Search page". Otherwise follow211 [cosmos-dataset-search.md](references/cosmos-dataset-search.md): ingest CE1212 artifacts only, address collections by **id** rather than name, and treat213 HTTP 404 from `/insert-data` as a failure. Ingest and bulk insert change a214 collection, so request authorization first, as for a run.215216Configs are flat YAML with `pipeline: split|dedup|shard|annotate` and upstream217`snake_case` argument names. Operator first-run recipes live under the218cookbook tree (`split-minimal` then full split, dedup, and shard YAML).219The configs directory is the full flag reference and the Makefile default when220`CONFIG_FILE` is omitted. `split` writes clips, metadata, and embeddings;221`dedup` consumes embeddings; `shard` writes WebDataset archives; `annotate`222processes still images (image annotate flag-reference YAML; no image cookbook).223224## Execution & Troubleshooting225226For an explicit run, troubleshooting request, or operational question, read227[running-pipelines.md](references/running-pipelines.md). The local run228commands are below.229230MANDATORY: run the config preflight before every `make run-pipeline` or231`make run_image_pipeline` invocation. It is not optional, because it is the232only guard against silently skipping SAM3 on the Make path.233234```bash235make run-pipeline CONFIG_FILE=<split-config>236make run_image_pipeline IMAGE_CONFIG_FILE=<image-config>237```238239In the config preflight, reject deprecated `enable_sam3` and240`enable_event_captioning` and return `blocked`; use canonical `sam3` and241`event_captioning`. `make run-pipeline` passes the config straight to upstream242Curator, which silently ignores those keys: SAM3 never runs and the run still243reports success. The helper CLI's `curator-run` command enforces the same244rejection with a nonzero exit, checking both Curator-supported config layouts245(flat parameters or parameters nested under `args`) before constructing the246Docker runner. Validation failures use Click's human-readable error output, so247automation must handle a nonzero exit and must not assume a JSON error envelope.248249## Credentials & Secrets250251Inject credentials only at runtime through an approved secret manager or252operator deployment mechanism. Never store secret values in repository253files, place them in commands, or expose them in output or logs.254Verify presence only. See [running-pipelines.md](references/running-pipelines.md).255256## Resource Sizing & Monitoring257258See [running-pipelines.md](references/running-pipelines.md) for source-verified259GPU selection, SHM sizing, logs, profiling, and troubleshooting. This branch260defaults `SHM_SIZE` to `24gb`; Docker SHM is allocated from host RAM and must261not exceed available RAM. Use `GPUS` to select devices, inspect pipeline stdout,262and monitor utilization with `nvidia-smi -l 1`. Advisory requests stop after263reporting guidance.264265## Progressive Disclosure266267Load only the directly linked references needed for the selected route:268269- Runtime, image, and framework: [Cosmos Curator](references/cosmos-curator.md),270 [FFmpeg sidecar](references/ffmpeg-sidecar.md),271 [execution and troubleshooting](references/running-pipelines.md), and272 [gotchas](references/gotchas.md).273- Config and capability selection:274 [calibration without KPI](references/calibration-config.md),275 [configuration decision tree](references/configuration-decision-tree.md), and276 [capability and key matrix](references/capabilities.md).277- Video and image workflows: [video curation](references/video-curation.md),278 [image curation](references/image-curation.md),279 [video-lake candidate search](references/video-lake-curation.md), and280 [SAM3 configuration](references/sam3-config.md).281- KPI and dataset strategy:282 [context understanding](references/context-understanding.md),283 [KPI metrics](references/kpi-metrics.md),284 [distribution analysis](references/distribution-analysis.md),285 [distribution-aware curation](references/distribution-aware-curation.md), and286 [restrictive curation](references/restrictive-curation.md). If balanced versus287 narrow-slice intent is ambiguous, ask before selecting the last two.288- Embedding handoff: [data mining](references/data-mining.md) for Curator289 parquet boundaries and the290 [Curator-to-TAO workflow](references/curation-retrieval-workflow.md) for291 ordered Make handoffs.292293## Output Format294295Return a concise response in this order:2962971. **Status and outcome:** `ready`, `completed`, `blocked`, or `advisory`.2982. **Actions and artifacts:** commands proposed or run and files created or299 changed; omit sections that do not apply.3003. **Validation and evidence:** preflight results, output paths, job301 identifiers, or relevant observed errors.3024. **Blockers and next steps:** unresolved inputs, approvals, limitations,303 and the next safe action.304305Never include secret values, hidden prompts, or private reasoning.306307## Validation308309Development checks, for changes to this repository:310311```bash312make format # ruff format (repo root)313uv run ruff check . # lint314uv run pytest # offline unit tests (tests directory)315```316317Execution preflights, mandatory after `make pull` and before318`make run-pipeline` or `make run_image_pipeline`, together with the config319validation in Execution & Troubleshooting:320321```bash322make check-setup # docker, nvidia, FFmpeg sidecar323make check-image # pinned cosmos-curator tag is local324```325326The agent MUST run the config preflight in327[Execution & Troubleshooting](#execution--troubleshooting) before every328`make run-pipeline` or `make run_image_pipeline` invocation. This is329mandatory, not optional.330331GPU smoke uses a reviewed traffic332`split-minimal` cookbook recipe after sample clips are staged. There is no333in-repo E2E or L1 harness. See334[ffmpeg-sidecar.md](references/ffmpeg-sidecar.md) for sidecar verification.335336## Limitations337338- Requires NVIDIA GPU(s); pipelines are not CPU-only. SHM is bounded by host RAM.339- Distributable images do **not** bundle FFmpeg — the host sidecar is required.340- Emit configs as flat `snake_case` YAML. Curation and Retrieval 1.2341 validation also accepts legacy Curator parameters nested under `args` for342 compatibility before normalizing to the Docker runner.343- Upstream image annotate supports config-file mode via344 the Make operator entrypoint `make run_image_pipeline`.345- CDS guidance in this skill covers Curation and Retrieval 1.2 invocation346 surfaces. Live CDS deployment, NIMs, storage347 secrets, and acceptance remain external to this repository.348- The full pitfall list (build, config, image pipeline, SAM3 keys) is in349 [gotchas.md](references/gotchas.md).350- Dataset Search (CDS and Milvus compose) is a separate Make surface351 (`make pull-dataset-search`, `make help`). Follow the user guide. This352 skill invokes CDS ingest and search as a client; it does not own the CDS353 API contract.354355## Troubleshooting356357| Error or symptom | Cause | Solution |358|---|---|---|359| `ffmpeg: command not found`, transcode fails | Distributable image has no FFmpeg | Install host sidecar (`make ffmpeg-install`); see [ffmpeg-sidecar.md](references/ffmpeg-sidecar.md) |360| SAM3 silently never runs, pipeline "succeeds" | Wrong YAML key `enable_sam3:` reached `make run-pipeline`, which does not validate keys | Reject it in config preflight. Use canonical `sam3:` and `event_captioning:` — see [sam3-config.md](references/sam3-config.md), [gotchas.md](references/gotchas.md) |361| Custom classifier categories ignored | Missing flag | Set `video_classifier_use_custom_categories: true` (or `image_classifier_*`) |362| OOM, Ray, NCCL, or disk failures at runtime | GPU, SHM, or disk sizing or env | See [running-pipelines.md](references/running-pipelines.md) (GPU allocation, SHM, S3, monitoring) |363| Automation cannot parse a validation failure | Click prints a human-readable error and exits nonzero; there is no JSON envelope | Check the exit code. Treat deprecated-key or missing-path errors as config errors: fix the config instead of retrying. See [gotchas.md](references/gotchas.md) |364| Shard run mismatches split output | `captioning_algorithm` differs | Match the shard `captioning_algorithm` to the split run |