Vllm Speculative Decoding

Pick, configure, tune, monitor vLLM speculative decoding in production. Thirteen SpeculativeMethod options (ngram, ngram_gpu, medusa, mlp_speculator, draft_model, suffix, custom_class, eagle, eagle3, dflash, dspark, mtp, extract_hidden_states), `--speculative-config` JSON schema, which methods pair with which target model family, Prometheus acceptance metric surface, version gates (v0.11.1 EAGLE-3 preamble fix, v0.16 parallel drafting, v0.18 ngram_gpu, v0.19 dflash and zero-bubble), composability with chunked prefill / PP / LoRA / FP8 / structured outputs, Arctic Inference plugin, where spec-dec stops paying at high batch.

air-gapped Updated 6 repo stars

File contents

air-gapped/skills commit c060e3ec42

Frequently asked questions

npx get-skill add air-gapped/vllm-speculative-decoding