NVIDIA-TAO

@nvidia-tao source repo

75 skills
0 followers
1 week ago last updated

▌
Tao Train Ocrnet · nvidia-tao bundle
OCRNet for scene text recognition. Recognizes text content from cropped text-region images and supports CTC and attention-based decoders. Use when training, evaluating, exporting, pruning, quantizing, retraining, or running inference for a TAO OCRNet model. Trigger phrases include "train OCRNet", "scene text recognition", "OCR cropped text", "CTC / attention text decoder".

0
installs
▌
Tao Train Rtdetr · nvidia-tao bundle
RT-DETR (Real-Time DEtection TRansformer) for 2D object detection. Designed for real-time inference with competitive accuracy and supports distillation and quantization for deployment optimization. Use when training, evaluating, distilling, quantizing, exporting, or running inference for a TAO RT-DETR model. Trigger phrases include "train RT-DETR", "real-time DETR", "low-latency object detection", "RT-DETR distillation / quantization".

0
installs
▌
Tao Launch Workflow · nvidia-tao bundle
The mandatory pre-launch gate and four-verb execution contract for every TAO workflow or action. Invoke BEFORE launching anything side-effecting — AutoML, train, evaluate, inference, export, TensorRT engine generation, or DEFT/application workflows — on any execution platform. Covers platform selection, credentials, image confirmation, dataset intake, preflight, the launch review, job records, monitoring, and failure/retry classification. Trigger phrases include "train this model", "run AutoML", "launch on SLURM/docker/k8s/brev/virtualenv", "evaluate my checkpoint", "start a TAO job".

0
installs
▌
Tao Mine Aoi Images · nvidia-tao bundle
Runs the DEFT embed-then-mine workflow for VCN AOI iterations — embeds the gap-analysis target parquet, embeds a source pool, and mines nearest-neighbour source images for downstream augmentation. Use as the immediate next step after `tao-route-visual-changenet-samples` when expanding a real-image augmentation queue from the mining subset.

0
installs
▌
Tao Finetune Clip · nvidia-tao bundle
CLIP vision-language model for image-text retrieval, zero-shot classification, embedding extraction, ONNX export, and TensorRT deployment. Use when fine-tuning or training CLIP, running zero-shot classification, computing image embeddings, or deploying CLIP to ONNX/TensorRT. This is a single-action model skill; do not use it for an iterative weak-attribute improvement loop that keeps retraining and evaluating until progress stops, which belongs to tao-run-deft-pas.

0
installs
▌
Tao Run On Brev · nvidia-tao bundle
Run a TAO training/evaluation/inference container on an NVIDIA Brev GPU instance. Instance provisioning (create/search/stop/delete/login) is delegated to the official brev-cli agent skill or the Brev MCP server; this skill covers only the TAO-specific part — running the container over `brev exec` via the four-verb docker contract. Trigger phrases include "run on Brev", "Brev GPU instance", "TAO on Brev", "submit job to Brev".

0
installs
▌
Paidf Cosmos Predict · nvidia-tao bundle
Prepare and run PAIDF Cosmos Predict video generation for DEFT media samples.

0
installs
▌
Tao Train Nvdinov2 · nvidia-tao bundle
NVDINOv2 for self-supervised visual representation learning. Trains vision transformers via self-distillation (teacher-student) without labels and produces general-purpose visual features. Use when training, exporting, or running inference for a TAO NVDINOv2 backbone. Trigger phrases include "train NVDINOv2", "self-supervised ViT pretraining", "DINOv2 backbone", "visual representation learning".

0
installs
▌
Tao Artifacts · nvidia-tao bundle
The contract home for TAO's SDK-free execution pipeline — authoritative JSON Schemas for the four typed artifacts (spec-bundle, job-record, results_dir layout, best_rec) plus the fixed job-status vocabulary and the nested-not-dotted spec rule. Use when authoring or validating a spec-bundle before submit, writing or reading a .tao/jobs job-record, resolving where results land, or consuming AutoML's best_rec.json. Trigger phrases include "validate the spec bundle", "job record schema", "status vocabulary", "results_dir layout", "best_rec schema".

0
installs
▌
Tao Data Io · nvidia-tao bundle
The data-mover for TAO jobs — decides the storage tier (A pre-positioned mount with zero fetch / B volume-from-S3 / C ephemeral in-compute fetch), stages inputs (bulk + annotation-selective + archive extract + HF/NGC PTM), maps credentials to env, routes outputs 3-way with upload-excludes, and runs the compute-frame verify gate. A support skill other platform skills (docker, kubernetes, slurm, brev, virtualenv) call to get data to and from the compute container without the TAO SDK. Trigger phrases include "stage inputs", "mount the dataset", "upload TAO results", "download only referenced files", "resolve results_dir", "verify the container can read the data".

0
installs
▌
Tao Train Dino · nvidia-tao bundle
DINO (DETR with Improved DeNoising Anchor Boxes) for 2D object detection. Transformer-based detector with denoising training, multi-scale features, and optional distillation support. Use when training, evaluating, exporting, distilling, quantizing, or running inference for a TAO DINO detector. Trigger phrases include "train DINO", "DETR object detection", "TAO 2D detection", "DINO with distillation".

0
installs
▌
Tao Train Reid · nvidia-tao bundle
Person re-identification (ReID). Learns discriminative embeddings to match the same person across different camera views, based on metric learning. Use when training, evaluating, exporting, or running inference for a TAO person re-identification model. Trigger phrases include "train ReID", "person re-identification", "cross-camera person matching", "ReID embeddings", "person re-id".

0
installs
▌
Replace With Skill Name · nvidia-tao bundle
One-to-three-sentence description of what this data transformation does. Use when the user asks to "REPLACE-WITH-INTENT", or mentions REPLACE-WITH-DOMAIN-TERMS. Include literal trigger phrases the user is likely to say.

0
installs
▌
Tao Mine Od Images · nvidia-tao bundle
Run TAO Data Services TMM unique-neighbor matching mining from embedding parquet files for object detection workflows. Use when an object detection workflow needs to mine a bijectively-assigned set of unique source images closest to target samples. Use global allocation when mining without class constraints. Use class_stratified when rare classes are specified.

0
installs
▌
Tao Train Codetr · nvidia-tao bundle
Co-DETR (CoDINO) for object detection. A DETR-family detector with collaborative hybrid assignment — auxiliary one-to-many heads supervise the encoder during training, giving strong closed-set accuracy at high inference cost. Use when training, evaluating, or running inference for a TAO Co-DETR model. Trigger phrases include "train Co-DETR", "run CoDINO", "codetr inference", "collaborative DETR", "autolabel detections with Co-DETR".

0
installs
▌
Tao Train Dinov3 · nvidia-tao bundle
DINOv3 continual self-supervised pre-training. Domain-adapts public DINOv3 ViT backbones on unlabeled images via teacher-student self-distillation (DINO + iBOT + KoLeo, optional Gram anchoring) and converts the EMA teacher into a timm-format backbone for downstream tasks. Trigger phrases include "train DINOv3", "DINOv3 SSL", "domain-adapt a foundation backbone", "continual pretraining", "self-supervised finetune DINOv3".

0
installs
▌
Tao Train Ocdnet · nvidia-tao bundle
OCDNet for scene text detection. Detects arbitrary-oriented text regions in natural images using a differentiable binarization approach. Use when training, evaluating, exporting, pruning, quantizing, retraining, or running inference for a TAO OCDNet model. Trigger phrases include "train OCDNet", "scene text detection", "arbitrary-oriented text boxes", "differentiable binarization detector".

0
installs
▌
Tao Setup · nvidia-tao bundle
One-time session setup and orchestration map for the TAO skill bank. Run this first when the TAO skills were installed individually (e.g. from a public skills catalog) so the session gets the cross-skill discovery flow, credential checks, and host preflight that the bundled plugin hook would otherwise inject automatically. Trigger phrases include "set up TAO skills", "TAO session setup", "prepare TAO environment", "TAO getting started".

0
installs
▌
Tao Finetune Nv Tesseract Ad Diffusion · nvidia-tao bundle
NV-Tesseract AD Diffusion — diffusion-based anomaly detection and fine-tuning for multivariate time series. Use when the user asks to "fine-tune NV-Tesseract", "run AD diffusion inference", "detect anomalies with diffusion", "time series anomaly detection", "finetune ad-diffusion", "use perform_anomaly_analysis_with_diffusion", "automl ad-diffusion", "hyperparameter search ad-diffusion", "hyperparameter optimization" or mentions "curriculum_medium.yaml", "final_model.pth", "nv-tesseract-ad-diffusion", "ad_diffusion", or "TSDiffuser_Generic".

0
installs
▌
Tao Run Automl Deft Pipeline · nvidia-tao bundle
Run the canonical NVIDIA AOI three-phase training pipeline — Phase 1 AutoML baseline (HPO), Phase 2 DEFT loop (RCA → SDG → mining → plain-train retrain), Phase 3 AutoML refinement on the DEFT-augmented dataset. Use when the user asks to "run the AOI workflow", "fine-tune my PCB AOI model end-to-end", "improve my AOI ChangeNet model", or "AOI workflow with AutoML" request — route here instead of tao-run-deft-aoi directly unless the user explicitly asks for the DEFT loop ONLY (e.g. "run JUST the DEFT loop", "skip AutoML, only DEFT"). Also handles the same three-phase pattern for non-AOI DEFT applications — AutoML baseline then DEFT loop warm-started from AutoML's winning HPs then post-DEFT AutoML refinement on the iteration-augmented dataset. Trigger phrases include "run the AOI workflow", "AOI end-to-end", "AutoML + DEFT", "AutoML then DEFT", "tune hyperparameters then DEFT", "DEFT with AutoML at both ends", "warm-start DEFT", "improve my AOI model".

0
installs
▌
Tao Validate Recipe Transfer · nvidia-tao bundle
Port a published computer vision paper's official code and training recipe onto a customer's own dataset, or diagnose why such a transfer produced bad numbers. Use this whenever someone wants to reproduce a CV paper, run a paper's repo on their own images, fine-tune a published detection/segmentation/classification/keypoint model on customer data, adapt a training recipe to a new dataset, or figure out why a fine-tuned vision model scores well on validation but fails in production. Also use for post-mortems on any failed or disappointing CV training run, and whenever a user mentions mAP that looks too good, a model that "worked in training but not in deployment", or transferring hyperparameters from a paper to their own data. Trigger even if the user only says "train a model on my dataset" and a published architecture or repo is involved.

0
installs
▌
Tao Run Deft Object Detection · nvidia-tao bundle
Run the full DEFT smart-data-augmentation loop for NVIDIA TAO Grounding DINO object detection: zero-shot baseline inference, KPI analysis, per-class gap analysis, SigLIP embedding of weak images, unique-neighbor mining against a source pool, ODVG dataset staging, and retraining — repeated for a fixed number of iterations. Also prepares the source pool the loop mines from, as a separate run: Co-DETR pseudo-labeling, folding to the target classes, KITTI→COCO→ODVG conversion, and embedding. Use for prompts like "run the DEFT OD loop", "run smart data augmentation for grounding dino", "mine and retrain my detection model", "improve OD mAP with gap analysis and mining", "prep the source pool", or "pseudo-label my unlabeled images for mining"; do not use for standalone TAO training, one-off inference, or gap analysis alone.

0
installs
▌
Tao Finetune Huggingface Model · nvidia-tao bundle
Fine-tune any HuggingFace CV / VLM / LLM model on local NVIDIA GPUs inside an NGC PyTorch container when no dedicated TAO model skill matches. Use when the user wants to fine-tune a HuggingFace model (full or LoRA), train a vision / VLM / LLM model end-to-end, generate a reproducible HF training pipeline, smoke-test a HuggingFace model locally before scale-up, push a fine-tuned model to the HF Hub with a model card, or emit a self-contained rerun skill for an existing HuggingFace finetune. Supports image classification, object detection, semantic / instance / panoptic segmentation, depth estimation, image-text-to-text VLM (SFT / LoRA), and LLM SFT / DPO / GRPO. Six-step workflow: inspect and qualify, hardware and NGC image, research, generate and smoke, train + eval + infer, push and emit rerun skill. Do not use for any Hugging Face model ID claimed by a dedicated `skills/models/*` skill; the model skill and its declared execution environment take precedence.

0
installs
▌
Tao Finetune Nv Tesseract Forecasting · nvidia-tao bundle
NV-Tesseract Forecasting — transformer-based multivariate time series forecasting with DARR (context-enhanced kNN retrieval), interpretability, and fine-tuning. Use when the user asks to "forecast with NV-Tesseract", "run forecasting inference", "use perform_forecasting", "DARR mode", "context-enhanced forecasting", "lag horizon attribution", "interpretability", "fine-tune forecasting", "fine-tune forecasting with automl", "hyper-parameter optimization with forecasting", or or mentions "nv-tesseract-forecasting", "moment_head_512_6hr", or "run8_best_model_cr".

0
installs
▌
Tao Train Metric Learning Recognition · nvidia-tao bundle
Metric-learning recognition (ml-recog) for fine-grained visual recognition. Learns embeddings for retrieval-based matching (e.g., retail product recognition) using triplet / contrastive losses. Use when training, evaluating, exporting, or running inference for a TAO metric-learning recognition model. Trigger phrases include "train metric learning", "ml-recog", "retrieval embeddings", "triplet loss recognition", "fine-grained matching".

0
installs
▌
Tao Train Image Classification · nvidia-tao bundle
PyTorch-based TAO image classification. Supports a wide range of backbones (FAN, EfficientNet, ResNet, etc.) with distillation and quantization for deployment. Use when training, evaluating, distilling, quantizing, exporting, or running inference for a TAO image-classification (PyT) model. Trigger phrases include "train image classifier", "TAO classification", "ResNet/EfficientNet/FAN backbone classifier", "classification-pyt".

0
installs
▌
Tao Run Inference Service · nvidia-tao bundle
Start, query, and stop a network-specific TAO inference microservice ({network_arch}-inference-microservice) by delegating container execution to the appropriate platform skill. Handles container image resolution, job-payload JSON construction, and the service registry. Use when the user wants to run inference on a TAO model checkpoint using a microservice container, deploy a TAO inference endpoint, or stop a running inference container.

0
installs
▌
Tao Analyze Gaps Visual Changenet · nvidia-tao bundle
Performs gap analysis on NVIDIA TAO VCN Classify (Visual Component Net) experiments by invoking the pinned TAO data-services container directly via `docker run … gap_analysis vcn_aoi …` — picks the optimal decision threshold, ranks per-sample weakness, and emits a top-K weakest parquet expanded per-lighting for downstream augmentation. Use when analyzing VCN classification failures, picking SDA augmentation targets, or auditing PASS/NO_PASS boundary cases.

0
installs
▌
Tao Run Deft Cr Its Mining · nvidia-tao bundle
Run the mining-based DEFT improvement workflow for ITS Cosmos-Reason binary video questions, focused on the non-reasoning classification/evaluation path. Use when the user asks for a DEFT CR ITS mining workflow, traffic-camera Cosmos Reason improvement loop, collision-identification workflow with data mining, or iterative Cosmos-RL refinement driven by gap analysis.

0
installs
▌
Tao Generate Referring Expressions · nvidia-tao bundle
Four-step image referring-expression pipeline: turns images plus KITTI bounding-box labels into region descriptions, scene captions, grounded referring expressions, and (optionally) verified expressions via VLM distillation. Use when the user wants to generate referring-expression annotations from images with KITTI labels, build region descriptions, produce grouped grounding phrases tied to bboxes, run a double-check verification pass on grounding expressions, auto-label traffic / scene images for referring datasets, or run the image_referring_expression pipeline. Triggers include 'referring expression', 'region description', 'KITTI labels', 'spatial relationship annotation', 'auto-label image referring expression', 'image_referring_expression'.

0
installs
▌
Tao Route Visual Changenet Samples · nvidia-tao bundle
Routes the weakest VCN samples (output of `tao-analyze-gaps-visual-changenet`) into per-augmentation-module subsets based on each module's label eligibility. Use when the user asks to "route VCN gap samples", "split AOI gaps for k-NN mining and AnomalyGen", or prepare the immediate next step after DEFT gap analysis in a VCN AOI SDA iteration.

0
installs
▌
Tao Setup Nvidia Gpu Host · nvidia-tao bundle
Host setup for TAO GPU backends. Checks and, after user approval, installs minimum-compatible NVIDIA driver, CUDA Toolkit, and NVIDIA Container Toolkit versions for Docker/local-Docker and Kubernetes GPU worker hosts. TAO-wide defaults can be overridden by the selected model's runtime profile. The `--check-only` path works on any Linux distribution; `--install` automates debian-family (Ubuntu/Debian/Pop!_OS/Mint/Zorin/Raspbian), rhel-family (Fedora/RHEL/Rocky/AlmaLinux), and suse-family (openSUSE/SLES) hosts, and prints actionable manual-install steps for everything else. Use when the user asks to "set up an NVIDIA GPU host", "check TAO Docker GPU runtime", or prepare a Kubernetes GPU worker for TAO.

0
installs
▌
Tao Train Action Recognition · nvidia-tao bundle
Action recognition from video sequences. Supports RGB, optical flow, and joint (multi-stream) input types for classifying temporal actions in video clips. Use when training, evaluating, exporting, or running inference on a TAO action-recognition model. Trigger phrases include "train action recognition", "video action classification", "RGB + optical flow action model", "TAO ActionRecognition".

0
installs
▌
Tao Train Optical Inspection · nvidia-tao bundle
Optical Inspection for defect detection using Siamese networks. Compares image pairs to detect manufacturing defects, anomalies, or quality issues. Use when training, evaluating, exporting, or running inference for a TAO Optical Inspection model on AOI / quality-control data. Trigger phrases include "train optical inspection", "AOI defect detection", "Siamese defect classifier", "PCB / manufacturing inspection".

0
installs
▌
Tao Train Mask Grounding Dino · nvidia-tao bundle
Mask Grounding DINO for grounded instance segmentation. Extends Grounding DINO with a mask-prediction head for open-set segmentation guided by text prompts. Use when training, evaluating, exporting, quantizing, or running inference for a TAO Mask-Grounding-DINO model. Trigger phrases include "train Mask Grounding DINO", "open-vocabulary segmentation", "text-prompted instance segmentation", "grounded mask DETR".

0
installs
▌
Tao Train Pose Classification · nvidia-tao bundle
Pose classification using ST-GCN (Spatial Temporal Graph Convolutional Network). Classifies skeleton sequences into action categories from pose-keypoint data. Use when training, evaluating, exporting, or running inference for a TAO pose-classification model. Trigger phrases include "train pose classification", "skeleton action recognition", "ST-GCN", "keypoint sequence classifier".

0
installs
▌
Tao Run Deft Aoi Cosmos3 · nvidia-tao bundle
Run the disk-backed DEFT AOI improvement loop for NVIDIA Cosmos Reason 3 / Cosmos3 models, using Nano by default and Edge or Super when explicitly requested: evaluate the base model on Proxy and frozen Benchmark splits, mine real image pairs from Proxy gaps, generate AnomalyGen synthetic NG pairs, assemble a per-iteration Train JSON from both producers, train with Cosmos Framework LoRA SFT, and repeat through the selected platform's submit/status/logs/cancel contract. This migration supports bare labels only: the assistant response must be exactly OK or NG. Use for "run Cosmos3 DEFT AOI", "CR3 AOI loop", or "improve Cosmos3 PCB inspection with bare OK/NG"; do not use for rich/reasoning annotation, one-off Cosmos training, or generic anomaly generation.

0
installs
▌
Tao Train Visual Changenet · nvidia-tao bundle
Visual ChangeNet for binary image classification and segmentation in AOI defect detection. Use when training, evaluating, exporting, or running inference for PCB defect detection or visual inspection, comparing image pairs for PASS/NO_PASS classification, or producing change-segmentation masks. Trigger phrases include "train Visual ChangeNet", "ChangeNet classify", "ChangeNet segment", "AOI defect detection", "PCB inspection model".

0
installs
▌
Tao Train Single Step · nvidia-tao bundle
Standard single-step train/eval/export workflow for any TAO model. Use when training a TAO model on a dataset without iterative data augmentation, AutoML, or DEFT loops. Trigger phrases include "single train run", "train then evaluate then export", "plain TAO training", "normal training", "no AutoML", "skip the loop". Routes through the per-model SKILL.md for action specifics and through `tao-launch-workflow` for platform/credentials/dataset intake.

0
installs
▌
Tao Generate Image Embeddings · nvidia-tao bundle
Run TAO Data Services image embedding to turn a parquet of image filepaths into an embedding parquet using CLIP, SigLIP, or a TAO checkpoint. Use when a workflow needs embeddings before nearest-neighbor or unique-neighbor mining, or when the user asks to "embed images", "compute image embeddings", or "generate SigLIP embeddings".

0
installs
▌
Tao Train Depth Anything V2 · nvidia-tao bundle
Monocular depth estimation using Metric Depth Anything v2 or Relative Depth Anything architectures. Predicts per-pixel depth from single RGB images. Use when training, evaluating, exporting, or running inference for a TAO monocular depth model. Trigger phrases include "train monocular depth", "DepthAnything v2", "metric depth from single image", "monocular depth estimation".

0
installs
▌
Tao Train Foundation Stereo · nvidia-tao bundle
Stereo depth estimation using FoundationStereo. Predicts disparity maps from stereo image pairs for 3D reconstruction. Use when training, evaluating, exporting, or running inference for a TAO FoundationStereo model. Trigger phrases include "train stereo depth", "FoundationStereo", "stereo disparity estimation", "3D reconstruction from stereo".

0
installs
▌
Tao Train Mask Auto Encoder · nvidia-tao bundle
Masked Auto-Encoder (MAE) for self-supervised pretraining and fine-tuning. Masks random patches and reconstructs them to learn visual representations; supports pretrain and finetune stages. Use when training, evaluating, exporting, or running inference for a TAO MAE backbone. Trigger phrases include "pretrain MAE", "self-supervised vision pretraining", "Masked Autoencoder", "Mask Auto-Encoder", "MAE fine-tune".

0
installs
▌
Tao Run On Virtualenv · nvidia-tao bundle
Run a Python training/eval script directly in an existing local virtualenv — no docker, no container. Implements the four-verb consumer contract (submit/status/logs/cancel) over a vendored process-lifecycle runner with durable on-disk state, PID-reuse-safe identity, and process-group cleanup. Use for docker-free local execution, plain-Python model scripts, fast HPO/AutoML trial smokes, or hosts where containers are unavailable. Trigger phrases include "run in my venv", "no docker", "virtualenv execution", "local python training", "run this training script directly".

0
installs
▌
Tao Convert Dataset Format · nvidia-tao bundle
Run `tao-daft convert` to convert NVIDIA TAO DAFT datasets between supported formats. Do not use for non-DAFT data. Use when the user asks to convert a DAFT dataset, change DAFT format, change a TAO dataset format, or run `tao-daft convert`.

0
installs
▌
Tao Mine Nearest Neighbors · nvidia-tao bundle
Run TAO Data Services TMM nearest-neighbor mining from embedding parquet files. Use when a workflow needs to mine source samples closest to target samples.

0
installs
▌
Tao Train Grounding Dino · nvidia-tao bundle
Grounding DINO for open-set object detection. Combines DINO-style detection with a BERT text encoder for language-guided detection — detects objects described by text prompts without a fixed class vocabulary. Use when training, evaluating, exporting, quantizing, or running inference for a TAO Grounding DINO model. Trigger phrases include "train Grounding DINO", "open-vocabulary detection", "text-prompted detector", "language-guided object detection".

0
installs
▌
Tao Validate Dataset Format · nvidia-tao bundle
Run `tao-daft validate` to check NVIDIA TAO DAFT datasets for structure, schema, and cross-reference errors. Do not use for non-DAFT formats. Use when the user asks to validate a DAFT dataset, check DAFT schema, validate a TAO dataset format, or run `tao-daft validate`.

0
installs
▌
Tao Finetune Cosmos Embed · nvidia-tao bundle
Cosmos-Embed1 video-text embedding for text-to-video retrieval, video-to-video search, semantic deduplication, and fine-tuning. Use when the user asks to "fine-tune Cosmos-Embed1", "run cosmos-embed inference", "export Cosmos-Embed1", "embed videos", or "search videos with text".

0
installs
▌
Tao Train Deformable Detr · nvidia-tao bundle
Deformable DETR for 2D object detection. Uses deformable attention for efficient multi-scale feature processing, lighter than DINO with competitive accuracy. Use when training, evaluating, exporting, quantizing, or running inference for a TAO Deformable-DETR model. Trigger phrases include "train deformable-detr", "Deformable DETR object detection", "lightweight DETR detector".

0
installs
▌
Tao Train Mask Auto Label · nvidia-tao bundle
MAL (Mask Auto-Label) for weakly-supervised segmentation. Produces segmentation masks from minimal annotations (point or box annotations) using a ViT-MAE backbone. Use when training, evaluating, or running inference for a TAO MAL model. Trigger phrases include "train MAL", "Mask Auto-Label", "weakly-supervised segmentation", "box-prompted segmentation", "minimal-annotation mask prediction".

0
installs
▌
Tao Generate Image Grounding · nvidia-tao bundle
Two-step image grounding pipeline: extracts referring expressions from (image, caption) pairs and grounds them to pixel-space bounding boxes via a VLM. Use when the user wants to ground captions to bboxes, generate phrase-grounded annotations, auto-label images for grounding, or run the image_grounding pipeline. Triggers include 'image grounding', 'phrase grounding', 'ground captions', 'auto-label image grounding', 'image_grounding'.

0
installs
▌
Tao Finetune Cosmos Reason · nvidia-tao bundle
Shared Cosmos3 frontend that explicitly routes Cosmos Framework and Cosmos-RL, validates runtime model/video-dataset/SLURM inputs, consumes an SQSH or packaged backend image, optionally plans explicit clean source builds, prepares checkpoints, validates the first update in-process, and returns token-weighted losses and task-aware accuracy.

0
installs
▌
Tao Analyze Gaps Vlm Bcq · nvidia-tao bundle
Extract false-positive and false-negative gaps from VLM binary-classification-question (BCQ, yes/no) predictions. Use when the user asks to "analyze VLM BCQ gaps", "extract VLM false positives and false negatives", or identify failure cases from a predictions JSON for DEFT root-cause analysis on a binary-classification VLM workflow.

0
installs
▌
Tao Train Nvpanoptix3d · nvidia-tao bundle
NVPanoptix3D for panoptic 3D scene reconstruction from posed RGB images. Produces 3D panoptic segmentation (semantic, instance, and panoptic masks) with occupancy completion. Built on a VGGT backbone with a Mask2Former-style head and 3D frustum reconstruction. Use when training, evaluating, exporting, or running inference for a TAO NVPanoptix3D model. Trigger phrases include "train NVPanoptix3D", "panoptic 3D reconstruction", "3D scene segmentation", "occupancy completion".

0
installs
▌
Tao Train Pointpillars · nvidia-tao bundle
PointPillars for 3D object detection from LiDAR point clouds. Encodes point clouds into a pseudo-image via a pillar-based representation, then applies 2D detection — used in autonomous driving and robotics. Use when training, evaluating, exporting, pruning, retraining, or running inference for a TAO PointPillars model. Trigger phrases include "train PointPillars", "LiDAR 3D detection", "point-cloud object detection", "pillar-based 3D detector".

0
installs
▌
Tao Analyze Detection Kpi · nvidia-tao bundle
Run TAO Data Services KPI analysis for object detection, comparing inference annotations against ground truth to compute per-class TP/FP/FN/TN, precision, recall, accuracy, and AP at a fixed IoU. Use when an object detection workflow needs per-class mAP reported after inference, or when the user asks to "run KPI analyze", "compute detection mAP", or "score my OD predictions against ground truth".

0
installs
▌
Tao Finetune Video Clip · nvidia-tao bundle
InternVideo2-CLIP L14 (TAO video_clip) for video-text retrieval, zero-shot classification, embedding extraction, LoRA fine-tuning, ONNX export, and TensorRT deployment. Use when the user asks to "fine-tune IV2CLIP", "run video_clip train/evaluate/inference/export", "build a Video-CLIP TensorRT engine", "InternVideo2-CLIP on KPI chunks", or "TAO video_clip on vadr1_chunks JSON".

0
installs
▌
Tao Run On Kubernetes · nvidia-tao bundle
Kubernetes execution platform — submits TAO container jobs as k8s Jobs with NVIDIA GPU scheduling; single-pod for one node, Indexed Jobs for multi-node distributed training. Use when running on EKS / GKE / AKS / on-prem clusters with the NVIDIA GPU Operator installed, or when integrating TAO into an existing k8s-native ML platform.

0
installs
▌
Tao Generate Anomalies · nvidia-tao bundle
Full PAIDF AnomalyGen pipeline — fine-tune on a new anomaly dataset, generate synthetic anomaly images (SDG), evaluate quality (nn_score), and search per-sample (guidance, crop_ratio) parameters. Three modes: full (Phase 0→7: finetune then generate), finetune_only (Phase 0→1: train only), inference_only (Phase 0, 2→7: generate from an existing checkpoint). Use when the user asks to "fine-tune AnomalyGen", "generate anomaly images", "run PAIDF SDG", "evaluate SDG output quality", "run per-sample search", or run any part of the AnomalyGen pipeline, even if they only mention one phase.

0
installs
▌
Tao Train Centerpose · nvidia-tao bundle
CenterPose for keypoint / pose estimation. Detects object centers and regresses keypoint locations for 6-DoF object pose estimation. Use when training, evaluating, exporting, or running inference for a TAO CenterPose model. Trigger phrases include "train CenterPose", "6-DoF object pose", "keypoint estimation", "object pose regression".

0
installs
▌
Tao Analyze Gaps Od Map · nvidia-tao bundle
Run TAO Data Services object-detection gap analysis from ground-truth and inference annotations. Use when an object detection workflow needs to identify weak images by comparing model predictions against ground truth using per-class recall, precision, and AP50 thresholds. Use when the user asks to "analyze OD gaps", "find weak OD images", or "run mAP gap analysis".

0
installs
▌
Tao Train Mask2former · nvidia-tao bundle
Mask2Former for universal image segmentation (panoptic, instance, and semantic). Transformer-based with masked attention for high-quality segmentation results. Use when training, evaluating, exporting, quantizing, or running inference for a TAO Mask2Former model. Trigger phrases include "train Mask2Former", "universal segmentation", "panoptic / instance / semantic segmentation", "masked-attention transformer segmenter".

0
installs
▌
Tao Run Deft Aoi · nvidia-tao bundle
Run the full DEFT AOI improvement loop for NVIDIA TAO VisualChangeNet / ChangeNet PCB inspection models: baseline evaluate, RCA, Cosmos AnomalyGen / AMP synthetic defects, k-NN mining, retraining, and deployment gating against a customer-defined primary metric and optional constraints. Use only when the request identifies an AOI / automated-optical-inspection, PCB-defect, VisualChangeNet, or ChangeNet workflow. Supports air-gapped/offline runs with pre-staged assets. Never infer AOI from generic iterative-improvement language. Do not use for CLIP / SigLIP image retrieval, attribute-labelled image data, standalone TAO training, one-off inference, generic anomaly generation, or RCA-only analysis.

0
installs
▌
Tao Run Deft Pas · nvidia-tao bundle
Run iterative improvement for NVIDIA TAO CLIP / SigLIP image-text retrieval on attribute-labelled data. Use when a request combines retrieval evaluation, weak-attribute or caption-pair mining, repeated retraining, and a stopping condition based on a retrieval KPI, validation plateau, or iteration budget; the customer need not know the DEFT or People Attribute Search (PAS) names. The self-contained workflow performs dataset preparation, zero-shot evaluation, attribute gap analysis, caption-space k-NN mining, history-aware selection, retraining, and re-evaluation. Treat `tao-deft-pas` as shorthand for this canonical `tao-run-deft-pas` workflow. Do not use for standalone CLIP training, one-off evaluation or embedding, generic k-NN mining, or AOI/ChangeNet DEFT workflows.

0
installs
▌
Tao Train Sparse4d · nvidia-tao bundle
Sparse4D for multi-camera temporal 3D object detection and tracking. Uses sparse queries with deformable attention across camera views and time for end-to-end 3D perception, with an instance bank for temporal tracking. Use when training, evaluating, exporting, quantizing, or running inference for a TAO Sparse4D model. Trigger phrases include "train Sparse4D", "multi-camera 3D detection", "temporal 3D tracker", "sparse query 3D perception".

0
installs
▌
Tao Run On Slurm · nvidia-tao bundle
Remote SLURM GPU cluster execution over SSH with sbatch/srun, Pyxis/Enroot containers, and Lustre-backed results. Use when running TAO training/eval/inference jobs on an on-prem or DGX SLURM cluster. Trigger phrases include "run on SLURM", "submit sbatch", "DGX SLURM cluster", "Pyxis/Enroot container", "Lustre dataset".

0
installs
▌
Tao List Capabilities · nvidia-tao bundle
Answer what the TAO Skill Bank plugin can do by generating the response from packaged application, data, model, AutoML, and platform manifests. Use when the user asks "what can TAO Skill Bank do", "list TAO models", "which TAO workflows are available", or "what supports AutoML".

0
installs
▌
Tao Train Bevfusion · nvidia-tao bundle
BEVFusion for multi-sensor 3D object detection. Fuses LiDAR point clouds and camera images in bird's-eye-view (BEV) space, used in autonomous driving for robust 3D perception. Use when training, evaluating, or running inference for a TAO BEVFusion model. Trigger phrases include "train BEVFusion", "LiDAR + camera fusion", "BEV 3D detection", "multi-sensor 3D perception".

0
installs
▌
Tao Train Oneformer · nvidia-tao bundle
OneFormer for universal image segmentation. Unifies panoptic, instance, and semantic segmentation with a single architecture using task-conditioned queries. Use when training, evaluating, exporting, quantizing, or running inference for a TAO OneFormer model. Trigger phrases include "train OneFormer", "universal segmentation", "task-conditioned segmentation", "panoptic / instance / semantic in one model".

0
installs
▌
Tao Train Segformer · nvidia-tao bundle
SegFormer for semantic segmentation. Lightweight transformer-based architecture with hierarchical feature extraction, efficient for real-time segmentation tasks. Use when training, evaluating, exporting, quantizing, or running inference for a TAO SegFormer model. Trigger phrases include "train SegFormer", "semantic segmentation", "lightweight transformer segmenter", "real-time semantic segmentation".

0
installs
▌
Tao Run On Docker · nvidia-tao bundle
The Docker execution platform for TAO jobs — a local daemon or a remote GPU box via DOCKER_HOST=ssh://user@host. Implements the four-verb consumer contract (submit/status/logs/cancel) over the docker CLI, wired to the job-record, tao-data-io staging, and the redact lint, on top of the underlying docker conventions (--gpus, mounts, NGC auth, inspection, data-root relocation, error modes). Use to run any single-node TAO container action on Docker without the SDK. Trigger keywords — docker, docker run, run on docker, DOCKER_HOST, remote docker, nvcr.io, --gpus, single-node GPU job.

0
installs
▌
Replace With Platform Name · nvidia-tao
Where and how GPU jobs run on this platform. One-to-three-sentence summary. Use when the user asks to "deploy on REPLACE-PLATFORM", "run on REPLACE-PLATFORM", or mentions the platform's distinctive concepts (e.g., resource shape, instance, node group).

0
installs
▌
Replace With Workflow Name · nvidia-tao bundle
Top-level workflow orchestrator for <use case>. Runs an end-to-end pipeline of <N> stages: <stage 1> → <stage 2> → … → <stage N>. Use when the user asks to "REPLACE-WITH-INTENT-1", "REPLACE-WITH-INTENT-2", or mentions REPLACE-WITH-DOMAIN-TERMS. Include literal trigger phrases.

0
installs
▌
Tao Run Automl · nvidia-tao bundle
Run container-backed AutoML / hyperparameter optimization (HPO) for NVIDIA TAO networks using AutoMLRunner. Handles algorithm selection (bayesian, hyperband, asha, bohb, llm, hybrid, autoresearch), WandB experiment tracking, job execution on any TAO SDK platform, result interpretation, and per-rec custom evaluation hooks. Use when the user mentions TAO AutoML, hyperparameter optimization, HPO, automl, automl_settings, AutoMLRunner, tao_automl, bayesian search, hyperband, ASHA, LLM-guided search, autoresearch, or wants to tune train/evaluate/inference/distill/prune/quantize for a TAO network. Model actions use the resolved image; venv training requires an explicit request. Platform-agnostic — runs on any SDK (Brev, SLURM, Kubernetes, Docker). Do not use generic "keep improving" language alone to override a matching domain-specific DEFT workflow; attribute-labelled CLIP / SigLIP image-retrieval loops belong to tao-run-deft-pas unless HPO is explicit.

0
installs

75 published skills