Speculative Decoding

Accelerate LLM inference using speculative decoding, Medusa multiple heads, and lookahead decoding techniques for 1.5-3.6× speedup without quality loss.

Orchestra Research Updated 14.5k repo stars
AI Research Skills Demo

File contents

Orchestra-Research/AI-Research-SKILLs commit 773a52944b

Frequently asked questions

npx get-skill add orchestra-research/speculative-decoding