Moe Sparsity Reasoning

Determine optimal MoE sparsity by separating memorization and reasoning trade-offs: active FLOPs improve reasoning while total parameters improve memorization, requiring joint optimization

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit 5c3d8f04cc

Frequently asked questions

npx get-skill add adu2021/moe-sparsity-reasoning