Sparser Block Attention

Accelerates self-attention by reordering tokens to concentrate relevant tokens within fewer blocks. Achieves 2.75x speedup on long-context prefilling by permuting tokens so semantically related information clusters together, enabling aggressive block sparsity without accuracy loss.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit b0a15ad4ff

Frequently asked questions

npx get-skill add adu2021/sparser-block-attention