Results for “node-bridge”

Convert single-node PyTorch distributed scripts into multi-node Slurm sbatch jobs and debug common multi-node failures, covering srun-native and torch.distributed approaches, container setup, NCCL timeouts, and interactive allocation.

Validate and use CUDA graph capture in Megatron Bridge, including local full-iteration graphs and Transformer Engine scoped graphs for attention, MLP, and MoE modules.

Enables Megatron Fully Sharded Data Parallel in Megatron-Bridge with configuration overrides, code anchors, pitfalls, and verification steps.

Configure fault tolerance, straggler detection, preemption, in-process restart, and re-run state machine for Megatron Bridge training jobs.