Fasa Frequency Aware Sparse Attention

Identify dominant frequency components in RoPE embeddings to determine which attention dimensions are essential, reducing KV cache by 81% while maintaining performance. Training-free approach integrates with existing compression methods for 2.56x speedup on long-context reasoning.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit 0d25a95e4c

Frequently asked questions

npx get-skill add adu2021/fasa-frequency-aware-sparse-attention