Results for “momentum-burst”

Configure fault tolerance, straggler detection, preemption, in-process restart, and re-run state machine for Megatron Bridge training jobs.

Reduces peak GPU memory in Megatron Bridge training by applying expandable segments, parallelism resizing, activation recompute, and CPU offloading constraints.

Provides guidance for training Mixture-of-Experts models with long context windows, covering context parallelism sizing, selective recomputation, dispatcher choices, and practical patterns from recent experiments.