Higher Order Linear Attention Mechanism

Enable data-dependent higher-order interactions in attention using prefix-sufficient statistics that maintain linear time and constant state, replacing quadratic dot-product attention while preserving expressivity through compact matrix operations.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit 275447e8e5

Frequently asked questions

npx get-skill add adu2021/higher-order-linear-attention-mechanism