Mobe Mixture Basis Experts

Compresses MoE language models through shared basis factorization of expert weight matrices, achieving 24-30% parameter reduction with minimal accuracy loss.

adu2021 Updated 7 repo stars

File contents

adu2021/skillxiv commit 12c8b92492

Frequently asked questions

npx get-skill add adu2021/mobe-mixture-basis-experts