Vllm Caching

vLLM tiered KV cache configuration for production H100/H200 clusters. Native CPU offload, LMCache (CPU+NVMe+GDS), NixlConnector (disaggregated prefill), MooncakeConnector (RDMA), MultiConnector composition. Version gates, sizing math (flag total across TP, not per-GPU — opposite of SGLang), KV-vs-weights offload distinction operators most often get wrong.

air-gapped Updated 6 repo stars

File contents

air-gapped/skills commit 7c41f7428a

Frequently asked questions

npx get-skill add air-gapped/vllm-caching