Results for “vllm”
17 skills
nvidia jetson-llm-benchmark
Benchmark Jetson LLM/VLM serving performance across vLLM, llama.cpp, and Ollama with structured JSON output.
2.2k · bundle
nvidia jetson-llm-serve
Serve LLMs and VLMs on NVIDIA Jetson devices using vLLM or SGLang with optimized Docker containers and quantization presets.
2.2k · bundle
huggingface huggingface-community-evals
Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware, with backend selection between vLLM, Transformers, and accelerate.
10.8k · bundle
nvidia jetson-package
Selects Jetson-compatible containers, vLLM runtime images, and Jetson AI Lab PyPI indexes based on Orin SM 8.7 vs Thor SM 11.0 and JetPack version.
2.2k · bundle
More results
nvidia jetson-speculative-decoding
Reduce per-token latency on Jetson vLLM servers by appending speculative decoding configuration, with guidance on when to enable and how to benchmark the improvement.
2.2k · bundle
google gke-inference
Deploys and optimizes AI/ML inference workloads on GKE, using GPUs, TPUs, and model servers.
14.4k
nvidia vss-summarize-video
Summarize recorded video clips using the LVS microservice with a VLM fallback, producing a narrative summary with timestamped events.
2.2k · bundle
nvidia tao-analyze-gaps-vlm-bcq
Extract false-positive and false-negative gaps from VLM binary-classification-question predictions by comparing model responses against ground truth, producing a structured JSONL file and summary report for downstream root-cause analysis.
2.2k · bundle
nvidia vss-deploy-dense-captioning
Deploy a standalone RT-VLM dense-captioning microservice and exercise its REST API endpoints for file upload, caption generation, streaming, chat completions, and Kafka integration.
2.2k · bundle
github mvvm-toolkit-di
Wire CommunityToolkit.Mvvm ViewModels into Microsoft.Extensions.DependencyInjection with constructor injection, service lifetimes, IMessenger registration, and testing seams for WPF, WinUI 3, MAUI, Uno, and Avalonia apps.
36.2k · bundle
nvidia jetson-inference-mem-tune
Recommends an inference runtime and memory-related launch flags for LLM/VLM workloads on NVIDIA Jetson devices, based on a live memory audit snapshot.
2.2k · bundle
github mvvm-toolkit
Build .NET MVVM ViewModels with CommunityToolkit.Mvvm source generators, base classes, and commands for WPF, WinUI, MAUI, Uno, and Avalonia.
36.2k · bundle
nvidia vss-deploy-profile
Selects, configures, deploys, verifies, debugs, or tears down a VSS profile (base, search, lvs, warehouse, edge) for NVIDIA's video search and summarization stack.
2.2k · bundle
nvidia vss-ask-video
Ask visual questions about video clips using a VSS agent's video_understanding tool, requiring a fresh look at frames rather than prior metadata or search results.
2.2k · bundle
huggingface hf-cloud-sagemaker-production-defaults
Creates SageMaker endpoints (real-time or async) with autoscaling, CloudWatch alarms, and tagging enabled by default, then smoke-tests them before declaring success.
10.8k · bundle
huggingface hf-cloud-serving-image-selection
Selects the correct SageMaker serving container image URI for HuggingFace model deployments, prioritizing HuggingFace-curated Deep Learning Containers over generic alternatives.
10.8k · bundle
nvidia deepstream-sop
Build, deploy, evaluate, debug, and measure latency for a GPU-accelerated FastAPI service that detects whether operators perform assembly-line steps in order via event boundary detection and VLM classification.
2.2k · bundle