Isaac Sim Validator
Final gate before delivery. Reject outputs that fail the checks below; do not soften results.
Universal checklist (apply to every output)
| Check |
Pass |
On fail |
| Script path |
exists, readable |
reject: Script path invalid: <path> |
| File size |
> 1 KB |
reject: Script empty or corrupted |
isaacsim import |
uses isaacsim.*, not omni.isaac.core |
reject: Use isaacsim.* namespace; omni.isaac.core is deprecated |
SimulationApp config |
explicit width and height |
reject: SimulationApp without window args causes swapchain-capture size mismatch |
| Lights present |
DomeLight (intensity >= 100) and DistantLight (>= 500) |
reject: No lighting; render will be black |
| Render mode |
RayTracedLighting for iteration; PathTracing only for hero shots |
reject: PathTracing too slow for iteration; use RayTracedLighting |
| ACES tonemap |
/rtx/post/tonemap/op=4, filmIso 200 default / 600 deep-aisle / 400 aerial |
suggest: Enable ACES (op=4) for contrast |
| Final render |
>= 150 KB and mean_RGB > 30 |
reject: Render is black or low-energy; check lighting |
| User paths |
no /home/<user>/, C:\Users\<name>\, or per-agent home paths |
reject: Hardcoded user path; use $ISAAC_SIM_DIR, $ISAAC_LAB_DIR, $WORKSPACE_DIR, or argparse |
| Network endpoints |
no bare IPs (192.168.x.x, 10.x.x.x) or non-localhost hosts |
reject: Hardcoded endpoint; parameterize via env var or config |
| Output dir |
configurable (env var, CLI, or function param) |
reject: Output dir hardcoded; accept as parameter or read $OUTPUT_DIR |
| VRAM headroom |
nvidia-smi headroom > 20% after settle |
warn at >90% utilization; likely leak or scale issue |
pkill -f "kit/kit" used |
flag for kill+relaunch bug |
warn: Verify zero kit processes before next launch |
| Sim duration |
>= 3 s of physics |
reject: Sim ran < 3 s; physics has not settled |
File structure validation (USD scenes)
| Item |
Valid |
Invalid |
Action |
$ISAAC_LAB_DIR/scripts/ |
read-only reference |
writing outputs there |
redirect to $OUTPUT_DIR or $WORKSPACE_DIR/outputs/ |
get_assets_root_path() -> s3://... |
runtime OK |
baked into delivered output |
cache locally; reference via configurable path |
Melting / Splat MDL-only shaders |
OK in raw form |
MDL-only at export |
filter out; MDL-only causes black render |
MakeInstanceable for RL |
required (make_instanceable: true) |
absent |
reject |
usdPreviewSurface on surfaces |
required |
absent |
reject; suggest adding UsdPreviewSurface |
convex_decompose_mesh |
false (with clean_sdf_dc) |
true |
recommend false for stability |
mpu=0.01 |
OK for warehouse shell (13 m tiles) |
on a robot |
reject |
Render validation (frame analysis)
| Metric |
Good |
Bad |
Action |
| File size |
1–4 MB |
~82 KB |
82 KB = black, reject |
| Mean RGB |
80–160 |
< 20 |
reject: Too dark; add lights |
| Max RGB |
200–250 |
0 |
reject: No light in scene |
| Pixel variance |
> 15 |
< 5 |
flat = missing shadows; add accents |
| Color histogram |
multi-modal |
single peak |
overexposed or flat lighting; adjust intensity |
| Object visibility |
>= 3 robots or 50+ assets |
< 10 visible |
suggest: Scene too sparse; add set dressing |
| Shadow consistency |
consistent shadows |
random black blobs |
reject: Shadow artifacts; check light cone bounds |
RL training validation
| Issue |
Action |
AppLauncher not first |
reject: AppLauncher must precede any isaaclab imports |
Task __init__.py non-empty |
reject: Task __init__.py must be empty; do not import configs |
omni.isaac.lab import |
reject: Use isaaclab.*; omni.isaac.lab is deprecated |
| PhysX backend (when Newton applies) |
reject: Use Newton; it is the default on Kit 110 |
Missing PYTHONUNBUFFERED=1 |
warn: Set PYTHONUNBUFFERED=1 to see logs in real time |
| No GPU memory logging |
suggest: Log torch.cuda.memory_allocated()/1e6 to W&B |
| Batch size exceeds VRAM |
suggest: Reduce num_envs (try 2048) |
| Wrong checkpoint name |
reject: Must be model_XXXX.zip, not policy.pt |
| Single seed |
suggest: Run seeds 42, 123, 456 for valid comparison |
Asset & shell validation
| Input |
Check |
Action |
docker run command |
-v mount for /workspace/output |
reject: Output folder not mounted |
| Sub-agent spawn |
workdir=$WORKSPACE_DIR |
reject: Sub-agent workdir hardcoded; use $WORKSPACE_DIR |
| Blender USD export |
correct mpu, dual-shader (UsdPreviewSurface + MDL), correct export type |
reject if any missing |
| Scene complexity |
renders within 10 settle steps |
reject: Scene too complex; simplify assets |
| Ghost artifacts |
cleanup of /dev/shm/carb-* |
reject: Use workspace_cleanup.py to clear /dev/shm/carb-* |
Decision tree
graph TD
A[Received Output] --> B{Has script?}
B -->|No| C[Reject: "No script provided"]
B -->|Yes| D{File size >1KB?}
D -->|No| E[Reject: "Script empty or corrupted"]
D -->|Yes| F{Python?}
F -->|Yes| G{isaacsim.* imports?}
G -->|No| H[Reject: "Use isaacsim.* not omni.isaac.core"]
G -->|Yes| I{SimulationApp or --exec?}
I -->|Neither| J[Reject: "Must use SimulationApp with window args OR isaac-sim.sh --exec"]
I -->|SimulationApp| K{width and height set?}
K -->|No| L[Reject: "SimulationApp requires explicit width and height"]
K -->|Yes| M{Dome + Distant lights?}
M -->|No| N[Reject: "Add DomeLight and DistantLight (min intensity 100 and 500)"]
M -->|Yes| O{RayTracedLighting?}
O -->|No| P[Reject: "Use RayTracedLighting for iteration"]
O -->|Yes| Q{Render output exists?}
Q -->|No| R[Reject: "No render output"]
Q -->|Yes| S{>= 150 KB and mean_RGB > 30?}
S -->|No| T[Reject: "Render is black, too dark, or corrupted"]
S -->|Yes| U{Hardcoded user paths or bare IPs?}
U -->|Yes| V[Reject: "Replace hardcoded user paths/IPs with env vars or args"]
U -->|No| Y[VALID - deliver]
Training output
| Item |
Action |
policy_only_XXXX.pt |
accept |
model_XXXX.zip |
accept |
log.csv or wandb URL |
accept |
results.json with episode data |
accept |
| No reward curve / metrics |
reject: Log episode/reward_mean and train/value_loss |
Reply format
- Pass:
Valid: <one-line summary>
- Fail:
Invalid: <list of failed checks>
- Suggest (only when otherwise valid):
Could be improved: <tip>
Do not hedge ("I think", "maybe"). Do not deliver a black frame.
Input sources
isaac-sim-orchestrator, isaac-sim-rendering, other producer skills, and sub-agent outputs.- Invoked on: "validate this", "check this render", "review this code".
Feedback loop
- Rejections feed back into the producing skill's failure-mode catalog.
- Recurring failures become new validator rules.
1---2name: isaac-sim-validator3description: Validate Isaac Sim simulation scripts and outputs before delivery. Rejects black frames, deprecated `omni.isaac.*` imports, hardcoded user paths/IPs, missing lights, wrong render mode, undersized SimulationApp configs, and bad RL output formats. Use when a sub-agent produces a script, a render, a training artifact, or any USD scene that's about to be handed to the user.4---56# Isaac Sim Validator78Final gate before delivery. Reject outputs that fail the checks below; do not soften results.910## Universal checklist (apply to every output)1112| Check | Pass | On fail |13|---|---|---|14| Script path | exists, readable | reject: `Script path invalid: <path>` |15| File size | > 1 KB | reject: `Script empty or corrupted` |16| `isaacsim` import | uses `isaacsim.*`, not `omni.isaac.core` | reject: `Use isaacsim.* namespace; omni.isaac.core is deprecated` |17| `SimulationApp` config | explicit `width` and `height` | reject: `SimulationApp without window args causes swapchain-capture size mismatch` |18| Lights present | `DomeLight` (intensity >= 100) **and** `DistantLight` (>= 500) | reject: `No lighting; render will be black` |19| Render mode | `RayTracedLighting` for iteration; `PathTracing` only for hero shots | reject: `PathTracing too slow for iteration; use RayTracedLighting` |20| ACES tonemap | `/rtx/post/tonemap/op=4`, `filmIso` 200 default / 600 deep-aisle / 400 aerial | suggest: `Enable ACES (op=4) for contrast` |21| Final render | >= 150 KB and mean_RGB > 30 | reject: `Render is black or low-energy; check lighting` |22| User paths | no `/home/<user>/`, `C:\Users\<name>\`, or per-agent home paths | reject: `Hardcoded user path; use $ISAAC_SIM_DIR, $ISAAC_LAB_DIR, $WORKSPACE_DIR, or argparse` |23| Network endpoints | no bare IPs (`192.168.x.x`, `10.x.x.x`) or non-localhost hosts | reject: `Hardcoded endpoint; parameterize via env var or config` |24| Output dir | configurable (env var, CLI, or function param) | reject: `Output dir hardcoded; accept as parameter or read $OUTPUT_DIR` |25| VRAM headroom | `nvidia-smi` headroom > 20% after settle | warn at >90% utilization; likely leak or scale issue |26| `pkill -f "kit/kit"` used | flag for kill+relaunch bug | warn: `Verify zero kit processes before next launch` |27| Sim duration | >= 3 s of physics | reject: `Sim ran < 3 s; physics has not settled` |2829## File structure validation (USD scenes)3031| Item | Valid | Invalid | Action |32|---|---|---|---|33| `$ISAAC_LAB_DIR/scripts/` | read-only reference | writing outputs there | redirect to `$OUTPUT_DIR` or `$WORKSPACE_DIR/outputs/` |34| `get_assets_root_path() -> s3://...` | runtime OK | baked into delivered output | cache locally; reference via configurable path |35| `Melting` / `Splat` MDL-only shaders | OK in raw form | MDL-only at export | filter out; MDL-only causes black render |36| `MakeInstanceable` for RL | required (`make_instanceable: true`) | absent | reject |37| `usdPreviewSurface` on surfaces | required | absent | reject; suggest adding `UsdPreviewSurface` |38| `convex_decompose_mesh` | `false` (with `clean_sdf_dc`) | `true` | recommend `false` for stability |39| `mpu=0.01` | OK for warehouse shell (13 m tiles) | on a robot | reject |4041## Render validation (frame analysis)4243| Metric | Good | Bad | Action |44|---|---|---|---|45| File size | 1–4 MB | ~82 KB | 82 KB = black, reject |46| Mean RGB | 80–160 | < 20 | reject: `Too dark; add lights` |47| Max RGB | 200–250 | 0 | reject: `No light in scene` |48| Pixel variance | > 15 | < 5 | flat = missing shadows; add accents |49| Color histogram | multi-modal | single peak | overexposed or flat lighting; adjust intensity |50| Object visibility | >= 3 robots or 50+ assets | < 10 visible | suggest: `Scene too sparse; add set dressing` |51| Shadow consistency | consistent shadows | random black blobs | reject: `Shadow artifacts; check light cone bounds` |5253## RL training validation5455| Issue | Action |56|---|---|57| `AppLauncher` not first | reject: `AppLauncher must precede any isaaclab imports` |58| Task `__init__.py` non-empty | reject: `Task __init__.py must be empty; do not import configs` |59| `omni.isaac.lab` import | reject: `Use isaaclab.*; omni.isaac.lab is deprecated` |60| PhysX backend (when Newton applies) | reject: `Use Newton; it is the default on Kit 110` |61| Missing `PYTHONUNBUFFERED=1` | warn: `Set PYTHONUNBUFFERED=1 to see logs in real time` |62| No GPU memory logging | suggest: `Log torch.cuda.memory_allocated()/1e6 to W&B` |63| Batch size exceeds VRAM | suggest: `Reduce num_envs (try 2048)` |64| Wrong checkpoint name | reject: `Must be model_XXXX.zip, not policy.pt` |65| Single seed | suggest: `Run seeds 42, 123, 456 for valid comparison` |6667## Asset & shell validation6869| Input | Check | Action |70|---|---|---|71| `docker run` command | `-v` mount for `/workspace/output` | reject: `Output folder not mounted` |72| Sub-agent spawn | `workdir=$WORKSPACE_DIR` | reject: `Sub-agent workdir hardcoded; use $WORKSPACE_DIR` |73| Blender USD export | correct mpu, dual-shader (UsdPreviewSurface + MDL), correct export type | reject if any missing |74| Scene complexity | renders within 10 settle steps | reject: `Scene too complex; simplify assets` |75| Ghost artifacts | cleanup of `/dev/shm/carb-*` | reject: `Use workspace_cleanup.py to clear /dev/shm/carb-*` |7677## Decision tree7879```mermaid80graph TD81 A[Received Output] --> B{Has script?}82 B -->|No| C[Reject: "No script provided"]83 B -->|Yes| D{File size >1KB?}84 D -->|No| E[Reject: "Script empty or corrupted"]85 D -->|Yes| F{Python?}86 F -->|Yes| G{isaacsim.* imports?}87 G -->|No| H[Reject: "Use isaacsim.* not omni.isaac.core"]88 G -->|Yes| I{SimulationApp or --exec?}89 I -->|Neither| J[Reject: "Must use SimulationApp with window args OR isaac-sim.sh --exec"]90 I -->|SimulationApp| K{width and height set?}91 K -->|No| L[Reject: "SimulationApp requires explicit width and height"]92 K -->|Yes| M{Dome + Distant lights?}93 M -->|No| N[Reject: "Add DomeLight and DistantLight (min intensity 100 and 500)"]94 M -->|Yes| O{RayTracedLighting?}95 O -->|No| P[Reject: "Use RayTracedLighting for iteration"]96 O -->|Yes| Q{Render output exists?}97 Q -->|No| R[Reject: "No render output"]98 Q -->|Yes| S{>= 150 KB and mean_RGB > 30?}99 S -->|No| T[Reject: "Render is black, too dark, or corrupted"]100 S -->|Yes| U{Hardcoded user paths or bare IPs?}101 U -->|Yes| V[Reject: "Replace hardcoded user paths/IPs with env vars or args"]102 U -->|No| Y[VALID - deliver]103```104105## Training output106107| Item | Action |108|---|---|109| `policy_only_XXXX.pt` | accept |110| `model_XXXX.zip` | accept |111| `log.csv` or `wandb` URL | accept |112| `results.json` with episode data | accept |113| No reward curve / metrics | reject: `Log episode/reward_mean and train/value_loss` |114115## Reply format116117- Pass: `Valid: <one-line summary>`118- Fail: `Invalid: <list of failed checks>`119- Suggest (only when otherwise valid): `Could be improved: <tip>`120121Do not hedge ("I think", "maybe"). Do not deliver a black frame.122123## Input sources124125- `isaac-sim-orchestrator`, `isaac-sim-rendering`, other producer skills, and sub-agent outputs.126- Invoked on: "validate this", "check this render", "review this code".127128## Feedback loop129130- Rejections feed back into the producing skill's failure-mode catalog.131- Recurring failures become new validator rules.
isaac-sim