Memory LLM Ffn Decoupling

Decouple feed-forward networks from self-attention by training FFNs on context-free token embeddings instead of residual streams. Enables pre-computation of FFN outputs as static lookup tables for inference efficiency and improved interpretability.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit 0c1a680db7

Frequently asked questions

npx get-skill add adu2021/memory-llm-ffn-decoupling