Mcore Migrate Gpt To Hybrid

Migration guide for moving Megatron Core GPTModel checkpoints, model providers, training commands, and layer mappings to HybridModel, including the mechanical steps for transferring an existing pretrain_gpt.py launch script.

NVIDIA Updated 23k repo stars

File contents

NVIDIA/Megatron-LM commit 0622e976bf

Frequently asked questions

npx get-skill add nvidia/mcore-migrate-gpt-to-hybrid