Sparse Attention Reasoning

Learn sparse attention patterns for reasoning model decoding via self-distilled gating, achieving 9x speedup at 90% sparsity while maintaining reasoning quality.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit 22aa6f6a16

Frequently asked questions

npx get-skill add adu2021/sparse-attention-reasoning