Leank Kv Cache Pruning

Learning-based approach to reduce key-value cache memory during inference through static channel-wise sparsity masks. Achieves 70% K cache and 16-18% V cache memory reduction with 1.3x attention speedup.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit 4e7804dc54

Frequently asked questions

npx get-skill add adu2021/leank-kv-cache-pruning