Efficient Long Context Reasoning

Achieve 10× higher decoding throughput on long prompts by replacing 50% of cross-attention layers with gated memory units (GMUs) combining SSMs and attention. Maintains reasoning capability while reducing memory I/O bottleneck from O(d_kv·N) to O(d_h).

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit dc0ec83ed9

Frequently asked questions

npx get-skill add adu2021/efficient-long-context-reasoning