Agentsop Vllm

Decision SOP for serving LLMs with vLLM. Covers PagedAttention mental model, quantization/parallelism/batching tradeoffs, OOM triage, and when NOT to use vLLM. Activates when a coder-agent is choosing or tuning an inference engine, debugging vLLM throughput/latency/OOM, or comparing vLLM against TGI/SGLang/TensorRT-LLM/llama.cpp.

agentsope Updated 483 repo stars

File contents

agentsope/SkillAlchemy commit cb2852e68b

Frequently asked questions

npx get-skill add agentsope/agentsop-vllm