Add Moe Model

Add a new HF MoE model to the transformers_modeling_backend experiment. Use when the user provides a HuggingFace model ID and wants to integrate it with Titan's MoE, or invokes /add_moe_model.

pytorch Updated
torchtitan

File contents

pytorch/torchtitan commit 0a259d495c

Frequently asked questions

npx get-skill add pytorch/add-moe-model