Lychee Decode Sparse Kv Sharing

Classify attention heads into retrieval (full attention) and sparse (token-selected) roles using HardKuma distribution for differentiable discrete optimization. Sparse heads reuse KV pairs from retrieval heads, reducing cache by 90% while maintaining quality through joint training.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit d622013f8b

Frequently asked questions

npx get-skill add adu2021/lychee-decode-sparse-kv-sharing