Diagonal Batching Recurrent Transformers

Unlocks parallelism in recurrent memory transformers through diagonal batching of the layers-segments grid, achieving 3.3x speedup on 131K-token sequences without model retraining.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit beee138a05

Frequently asked questions

npx get-skill add adu2021/diagonal-batching-recurrent-transformers