Vllm

Use when self-hosting an OpenAI-compatible LLM endpoint with vLLM 0.33 — vllm serve, offline LLM.generate / llm.chat, PagedAttention with continuous batching and prefix caching, --tensor-parallel-size / --pipeline-parallel-size, AWQ/GPTQ/FP8 quantization, or wiring LangChain/LlamaIndex clients at http://localhost:8000/v1; also when the server OOMs on long context, chat requests ignore your sampling params because generation_config.json overrides them, or engine startup dies compiling kernels.

znlgis Updated 79 repo stars

File contents

znlgis/opengis-skills commit 2d2d211af5

Frequently asked questions

npx get-skill add znlgis/vllm