Results for “triton-inference-server”

12 skills

vss-deploy-video-embedding

Deploy and operate the VSS 3.2 GA RT-Embed Video Embedding microservice using Docker Compose, covering GPU prerequisites, REST API usage for file uploads, text/video embeddings, live RTSP streams, Redis/Kafka/OTel integration, and troubleshooting.

More results

jetson-llm-serve

Serve LLMs and VLMs on NVIDIA Jetson devices using vLLM or SGLang with optimized Docker containers and quantization presets.

tao-run-inference-service

Start, query, and stop a TAO inference microservice for a specific network architecture by delegating container execution to the appropriate platform skill.

Build video analytics pipelines using NVIDIA DeepStream SDK 9.0 with Python pyservicemaker API, including GStreamer-based video processing, TensorRT inference integration, object detection/tracking, and Kafka/message broker integration.

jetson-flash-image

Flash a promoted BSP image to a Jetson device in RCM mode using NVIDIA's flash.sh or l4t_initrd_flash.sh toolchain.

jetson-inference-mem-tune

Recommends an inference runtime and memory-related launch flags for LLM/VLM workloads on NVIDIA Jetson devices, based on a live memory audit snapshot.

Build, deploy, evaluate, debug, and measure latency for a GPU-accelerated FastAPI service that detects whether operators perform assembly-line steps in order via event boundary detection and VLM classification.

jetson-speculative-decoding

Reduce per-token latency on Jetson vLLM servers by appending speculative decoding configuration, with guidance on when to enable and how to benchmark the improvement.

Deploys and optimizes AI/ML inference workloads on GKE, using GPUs, TPUs, and model servers.

jetson-customize-camera

Enable MIPI/GMSL camera sensors on a Jetson Thor or Orin custom carrier by rendering a kernel-DT overlay from the in-tree sensor DTSI.

nv-generate-mr-brain-finetune

Finetunes the NV-Generate-CTMR MR-brain diffusion UNet from user-supplied NIfTI training volumes using a wrapper that stages configs and delegates to upstream scripts.

Selects Jetson-compatible containers, vLLM runtime images, and Jetson AI Lab PyPI indexes based on Orin SM 8.7 vs Thor SM 11.0 and JetPack version.