Vllm

Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.

graniet Updated 213 repo stars

File contents

graniet/kheish commit 682d2f6d0d

Frequently asked questions

npx get-skill add graniet/vllm