nvlabs
- 35 skills
- 0 followers
- 9 hours ago last updated
- ▌ Ace Rtl · nvlabs bundleConsolidated ACE-RTL skill for running an iterative generator-reflector-coordinator RTL agent on CVDP. Use for CVDP official repo setup with native or Docker environments, no-cheating benchmark runs, role-based external LLM routing, unique outputs logging, task-driven CVDP prompt guidance, and real pass-rate reporting.
- ▌ Eda Tool Setup · nvlabs bundleUse when CVDP or ACE-RTL native runs need simulator, synthesis, coverage, or license setup before classifying EDA-tool failures as INFRA_SETUP.
- ▌ Doc Reviewer · nvlabsReviews recent code changes and checks if documentation needs updates. Reads all MD files in root and docs/ to identify stale or missing documentation. Use when completing features, before pushing, or when asked to "check docs", "review documentation", or "doc-reviewer".
- ▌ Optimize Slurm Topology · nvlabs bundleOptimize AlpaSim Slurm topology throughput using persistent local Prometheus/Grafana telemetry and run artifacts. Use when tuning service GPU placement, replicas_per_container, runtime.nr_workers, endpoint n_concurrent_rollouts, NRE/physics cache sizes, or Slurm experiment batches for full-duration rollout throughput.
- ▌ Robolab Taskgen · nvlabs bundleGenerate RoboLab task files from natural language descriptions of robot manipulation goals. Use this skill when a user wants to create, write, or generate a new task definition, or asks how to define success conditions, terminations, subtasks, or instructions for a robot manipulation task.
- ▌ Robolab Scenegen · nvlabs bundleGenerate USD scene files for RoboLab from natural language descriptions. Use this skill when a user wants to create a new scene with objects on a table, or asks to arrange objects for a robot manipulation task.
- ▌ Alpamayo1 Sft · nvlabs bundleRun end-to-end supervised fine-tuning (SFT) of the Alpamayo-1 VLM action model on the Physical AI AV (PAI) dataset. The agent collects a small set of choices from the user up front (PAI chunk range, CoC reasoning toggle, W&B preference, which stage(s) to run, dataset/checkpoint paths), then drives the whole pipeline. Use when an agent must run Stage 1 VLM SFT, Stage 2 trajectory-diffusion expert, or both on a multi-GPU host or cluster; when setting up the `a1_sft` uv venv from scratch; when overriding Hydra config (cameras, chunk IDs, learning rate, DeepSpeed); when evaluating a Stage-2 checkpoint against `val/metric/min_ade`; when diagnosing common SFT failures (flash-attn dtype warnings, wandb 403, DeepSpeed grad-accum mismatch, `use_cache` checkpointing warnings). Trigger keywords: alpamayo, alpamayo-1, alpamayo1, sft, post-train, post-training, fine-tune, finetune, vlm, action expert, trajectory diffusion, reasoning vla, qwen3-vl, qwen3, pai, physical_ai_av, physicalai-autonomous-vehicles, hydra, omegacon
- ▌ Alpamayo1 X Rl · nvlabs bundleRun end-to-end open-loop RL post-training of the Alpamayo VLM action model (Alpamayo 1 or 1.5) on the Physical AI AV (PAI) dataset using Cosmos-RL + GRPO. The agent collects a small set of choices from the user up front (Alpamayo version, reward mode — motion-only vs joint reasoning+motion, W&B preference, single-node vs multi-node, dataset/checkpoint paths), then drives the whole pipeline. Use when an agent must convert a released Alpamayo checkpoint into a training-ready one, curate a PAI mini subset, launch policy + rollout replicas via `cosmos-rl`, and export the resulting policy checkpoint back to a HuggingFace directory; when setting up the `a1x_rl` uv venv from scratch; when overriding TOML config (parallelism, replicas, reward weights, prefetch, optimizer); when diagnosing common RL failures (rollout-too-fast buffer growth, weight sync lag, prefetch misconfiguration, vLLM OOM, GRPO group collapse). Trigger keywords: alpamayo, alpamayo-1, alpamayo-1.5, alpamayo1, alpamayo1_5, rl, post-train, post-train
- ▌ Alpamayo1 5 Sft · nvlabs bundleRun end-to-end supervised fine-tuning (SFT) of the Alpamayo-1.5 VLM action model on PAI (with navigation conditioning) or LingoQA (for visual question answering). The agent collects a small set of choices from the user up front (Stage-1 task — nav / vqa / default trajectory, W&B preference, dataset and checkpoint paths, which stage(s) to run), then drives the whole pipeline. Use when an agent must convert the released `nvidia/Alpamayo-1.5-10B` checkpoint into A1-format with `convert_checkpoint.py to-a1`, prepare PAI + nav annotations or LingoQA Scenery, run Stage-1 VLM SFT on the chosen task, optionally continue into Stage-2 trajectory-diffusion-expert training (for trajectory tasks), and evaluate; when setting up the `a1_5_sft` uv venv from scratch; when overriding Hydra config (`vla_processor` variant, `chunk_ids`, learning rate, DeepSpeed); when diagnosing common SFT failures (flash-attn dtype warnings, wandb 403, DeepSpeed grad-accum mismatch, `use_cache` checkpointing warnings, hardcoded-path slip-ups in
- ▌ Compass · nvlabs bundleFront-door for COMPASS — training, evaluation, SAGE scene workflows (search / USD conversion / scene registration), and OSMO cloud submission. Use whenever the user mentions training a policy, adding a SAGE scene, evaluating a checkpoint, or running COMPASS in general. For debug / onboarding-a-new-robot, see the specialty siblings: compass-doctor, compass-newembodiment.
- ▌ Compass Doctor · nvlabs bundleDiagnose why COMPASS isn't working: container, GPU, activated shell, assets, Isaac Sim init, checkpoint validity. Use whenever the user reports vague COMPASS errors — "training won't start", "something's wrong", "why is this failing" — even without the word "debug". Make sure to use this when the user mentions COMPASS isn't behaving and doesn't have a more specific intent.
- ▌ Compass Newembodiment · nvlabsOnboard a new robot platform to COMPASS: generate robot ArticulationCfg + per-embodiment env_cfg, register in EmbodimentEnvCfgMap, smoke-test the spawn with --num_envs 1. Use whenever the user wants to add a new robot, register an embodiment, or extend COMPASS with their platform. NOT for switching between built-in embodiments (use --embodiment) or for adding a new scene (the `compass` skill covers SAGE → register).
- ▌ Pr · nvlabsDraft and submit a GitHub Pull Request following the project PR template. Reads recent commits, fills out description/motivation/test sections, checks the checklist interactively, then runs gh pr create.
- ▌ Enpire · nvlabsInspect, test, and extend ENPIRE robot tools, tasks, calibration, and PLD workflows.
- ▌ Sync Docs · nvlabsSynchronize docs/ with codebase: update stale docs, discover undocumented features, rebuild CLAUDE.md index. Use after implementation work or periodically.
- ▌ Libero Fix Loop · nvlabs bundleRun the baseline-free LIBERO-Pro Fix Loop: inspect one initial observed scene, generate task-level code, debug failures on seeds 51–65 using traces and keyframes, validate on seeds 1–50, and promote reusable patterns.
- ▌ Libero Evosearch · nvlabs bundleRun the LIBERO-Pro Fix Loop + Evolutionary Search experiment: iterative candidate search for low-performing tasks, validation selection, and final held-out evaluation.
- ▌
- ▌ Robosuite Fix Loop Skill · nvlabs bundleMaster reference for ASPIRE Robosuite experiments. Covers system overview, 7 tasks, setup, running experiments, debugging, full API reference, and all pipeline modes (Fix Loop, Baseline).
- ▌ Robosuite Evosearch · nvlabs bundleRun Robosuite Fix Loop + Evolutionary Search with K=8 candidate search on seeds 101-125 and selected-code evaluation on seeds 1-100.
- ▌ Yam Geometry · nvlabs bundleUse for YAM DOF, coordinate directions, tabletop/rack/trash geometry, camera frames, gripper geometry, and contact/placement offsets.
- ▌ Yam Full Demo · nvlabsUse for the current YAM full demo, yam_demo.sh, bottle rack, KitKat handover/trash, bowl/can commands, and canonical saved-script commands.
- ▌ Yam Transport · nvlabsUse for YAM held-object transport after pickup, rack/bin approach waypoint ordering, object-relative placement, avoiding shelf/rack collisions, and choosing staged axis moves.
- ▌ Robosuite Training Law Skill · nvlabs bundleMaster reference for ASPIRE Robosuite experiments. Covers system overview, 7 tasks, setup, running experiments, debugging, full API reference, and all pipeline modes (Fix Loop, Baseline).
- ▌ Yam Grasp Pickup · nvlabsUse for YAM object detection, grasp candidate generation, side/top-down/rim pickup choices, staged gripper close, contact evidence, lift-only checks, and pickup failures.
- ▌ Yam Server Setup · nvlabsUse for YAM arm, camera, perception, AnyGrasp, BundleSDF, SAM3, cuRobo, PyRoki, and provider server checks/restarts.
- ▌ Libero Zeroshot Transfer · nvlabs bundleRun the LIBERO-90 build and zero-shot transfer experiment: build a skill library on LIBERO-90, tag snapshots, then evaluate transfer to LIBERO-Long-Pro.
- ▌ Yam Motion Planner · nvlabsUse for YAM cuRobo planning/IK failures, no-motion vs real-robot planning mismatch, waypoint robustness sweeps, perturbation testing, and recovery after motion planner failures.
- ▌ Yam Robot Debugging · nvlabsUse for YAM real-robot debugging: saved-script edits, physical runs, artifact inspection, autonomous patch-and-retry loops, and compact runtime context updates.
- ▌ Libero Library Size Scaling · nvlabs bundleRun the LIBERO library-size scaling experiment: evaluate frozen snapshot skill libraries on LIBERO-Long-Pro and generate success-rate scaling tables/plots.
- ▌ Yam Retreat Recovery · nvlabsUse for YAM post-release retreat, gripper-open confirmation, home/open recovery, retreat after rack/bin placement, and safe recovery while holding an object after a failed run.
- ▌ Yam Runtime Artifacts · nvlabsUse when inspecting YAM logs, videos, SAM3 overlays, observations, planner previews, function-call JSON, result files, or failure evidence.
- ▌ Libero Inference Time Scaling · nvlabs bundleRun the LIBERO inference-time scaling experiment: debug Long-Pro tasks at frozen library snapshots, evaluate token-budget checkpoints, and plot Pareto curves.
- ▌ Yam Simulation Transfer · nvlabs bundleUse when borrowing LIBERO/robosuite strategy ideas for YAM grasping, handover, pouring, placement, rack, or bin tasks.
- ▌ Graspgenx · nvlabs bundleRun and operate GraspGenX — a cross-embodiment foundation model for 6-DOF grasp generation. Use when installing GraspGenX (uv or Docker), running inference demos (segmented object point cloud, scene point cloud, or object mesh), choosing a gripper or planner (graspmoe vs diffusion), starting or calling the ZMQ client-server, driving grasping through the MCP server, integrating a new gripper (config wizard / sweep-volume params), or wiring grasps into an end-to-end cuRobo + Newton pick-and-place pipeline. Also covers checkpoint/asset setup and the depth-quality requirement for real-world use.