Results for “vlm-serving”

Deploy a trained model to serving with versioning, shadow or canary rollout, and a tested rollback path.

Deploy a standalone RT-VLM dense-captioning microservice and exercise its REST API endpoints for file upload, caption generation, streaming, chat completions, and Kafka integration.

You are an expert in vLLM, the high-throughput LLM serving engine. You help developers deploy open-source models (Llama, Mistral, Qwen, Phi, Gemma) with PagedAttention for efficient memory management, continuous batching, tensor parallelism for multi-GPU, OpenAI-compatible API, and quantization support — achieving 2-24x higher throughput than HuggingFace Transformers for production LLM serving.

Deploy and serve LLMs in production with vLLM, Ollama, TGI, and llama.cpp, including quantization and GPU optimization.

Deploy and operate the RTVI-CV-3D microservice for multi-camera 3D detection and tracking, supporting sample datasets, custom videos, and RTSP streams.

Recommends an inference runtime and memory-related launch flags for LLM/VLM workloads on NVIDIA Jetson devices, based on a live memory audit snapshot.

Build, deploy, evaluate, debug, and measure latency for a GPU-accelerated FastAPI service that detects whether operators perform assembly-line steps in order via event boundary detection and VLM classification.

Deploy and operate the VSS 3.2 GA RT-Embed Video Embedding microservice using Docker Compose, covering GPU prerequisites, REST API usage for file uploads, text/video embeddings, live RTSP streams, Redis/Kafka/OTel integration, and troubleshooting.

Deploy, debug, and operate the RTVI-CV 2D detection/tracking microservice and call its REST API for stream management, health checks, and metrics.

ML pipeline design — data versioning, experiment tracking, deployment patterns, drift monitoring. Use when building an ML pipeline from data to deployment, setting up MLOps tooling (DVC, MLflow, model registry), choosing deployment patterns (shadow, canary, A/B), or designing monitoring for drift and degradation.