Results for “bridges”

Validate and use CUDA graph capture in Megatron Bridge, including local full-iteration graphs and Transformer Engine scoped graphs for attention, MLP, and MoE modules.

Run Megatron-LM (MLM) and Megatron Bridge training with mock or real data, covering correlation testing, available recipes, and multi-GPU examples.

Configure fault tolerance, straggler detection, preemption, in-process restart, and re-run state machine for Megatron Bridge training jobs.

Enables Megatron Fully Sharded Data Parallel in Megatron-Bridge with configuration overrides, code anchors, pitfalls, and verification steps.

Reduces peak GPU memory in Megatron Bridge training by applying expandable segments, parallelism resizing, activation recompute, and CPU offloading constraints.

Configure and validate CPU offloading for Megatron Bridge training, including activation offloading and optimizer state offloading with HybridDeviceOptimizer.

Validate and configure packed sequences and long-context training in Megatron-Bridge, distinguishing offline packed SFT for LLMs from in-batch packing for VLMs with correct context parallelism constraints.

Indexes Megatron Bridge recipes and recommends the best starting config based on model, GPU count, and training goal.

Convert single-node PyTorch distributed scripts into multi-node Slurm sbatch jobs and debug common multi-node failures, covering srun-native and torch.distributed approaches, container setup, NCCL timeouts, and interactive allocation.