Standing Committee Moe

Reveal that Mixture-of-Experts models harbor a 'Standing Committee' of consistent expert coalitions handling majority computational load across domains. Challenges specialization assumptions and suggests training approaches like load-balancing losses may work against natural optimization.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit ec4738ba74

Frequently asked questions

npx get-skill add adu2021/standing-committee-moe