Lookaheadkv Predictive Cache Eviction

Evict unnecessary KV cache entries using trainable lookahead tokens and LoRA modules that predict token importance without drafting responses. Achieve 14.5× speedup over draft-based methods with 2% eviction overhead.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit 4fd01bbc83

Frequently asked questions

npx get-skill add adu2021/lookaheadkv-predictive-cache-eviction