Results for “triton-inference-server”

5 skills

Vss Deploy Video Embedding

Deploy and operate the VSS 3.2 GA RT-Embed Video Embedding microservice using Docker Compose, covering GPU prerequisites, REST API usage for file uploads, text/video embeddings, live RTSP streams, Redis/Kafka/OTel integration, and troubleshooting.

More results

Jetson LLM Serve

Serve LLMs and VLMs on NVIDIA Jetson devices using vLLM or SGLang with optimized Docker containers and quantization presets.

Tao Run Inference Service

Start, query, and stop a TAO inference microservice for a specific network architecture by delegating container execution to the appropriate platform skill.

Deploys and optimizes AI/ML inference workloads on GKE, using GPUs, TPUs, and model servers.

Selects Jetson-compatible containers, vLLM runtime images, and Jetson AI Lab PyPI indexes based on Orin SM 8.7 vs Thor SM 11.0 and JetPack version.