Results for “tensorrt-llm”

Train, evaluate, export, and run inference for monocular depth estimation models using Metric Depth Anything v2 or Relative Depth Anything architectures via the TAO toolkit.

Train, evaluate, distill, quantize, export, and run inference for RT-DETR object detection models using NVIDIA TAO.

Train, evaluate, export, quantize, and run inference for a TAO OneFormer model that performs panoptic, instance, and semantic segmentation using task-conditioned queries.

Trains, evaluates, exports, and runs inference for Siamese-network-based optical inspection models to detect manufacturing defects and quality issues in image pairs.

Submit and manage TAO training, evaluation, and inference jobs on SLURM GPU clusters over SSH with sbatch/srun, Pyxis/Enroot containers, and Lustre-backed storage.

Benchmark Jetson LLM/VLM serving performance across vLLM, llama.cpp, and Ollama with structured JSON output.

Train, evaluate, export, and run inference for CenterPose models used in 6-DoF object pose estimation with keypoint regression.

Reduce per-token latency on Jetson vLLM servers by appending speculative decoding configuration, with guidance on when to enable and how to benchmark the improvement.

Trains, evaluates, exports, quantizes, and runs inference for a Mask Grounding DINO model for open-set instance segmentation guided by text prompts.

Train, evaluate, export, prune, and run inference for PointPillars 3D object detection models from LiDAR point clouds using NVIDIA TAO.

Lint and format Python code for Megatron-LM using ruff, black, isort, pylint, and mypy, with commands for autoformatting and import ordering.

Train, evaluate, export, and run inference for Masked Auto-Encoder (MAE) models for self-supervised pretraining and fine-tuning of visual representations.

Launch distributed Megatron-LM training jobs on a SLURM cluster with a minimal sbatch skeleton, environment-variable setup for torch.distributed.run, CUDA_DEVICE_MAX_CONNECTIONS rules, container conventions, monitoring, and per-rank failure diagnosis.

Train, evaluate, distill, quantize, export, and run inference for PyTorch-based TAO image classification models with support for multiple backbones.