Vocabtrim Speculative Decoding

Accelerate speculative decoding by pruning drafter vocabulary to high-frequency tokens. Achieves 16% speedup in memory-bound settings by eliminating unused vocabulary entries without retraining.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit 0036b2f0d2

Frequently asked questions

npx get-skill add adu2021/vocabtrim-speculative-decoding