Results for “mbr”

Optimizes MoE expert-parallel communication overlap in Megatron Bridge, covering dispatch/combine overlap, flex dispatcher backends, and expert wgrad scheduling.

Provides practical guidance for training Mixture-of-Experts Vision-Language Models in Megatron Bridge, comparing FSDP and 3D-parallel approaches with lessons from recent multimodal experiments.

Validate and use CUDA graph capture in Megatron Bridge, including local full-iteration graphs and Transformer Engine scoped graphs for attention, MLP, and MoE modules.

Generates synthetic brain MRI volumes using NVIDIA's NV-Generate-CTMR workflow, with configurable modality and random seed.

Generates synthetic body MRI volumes using NVIDIA's NV-Generate-CTMR rflow-mr model. Wraps the upstream diffusion inference pipeline with config staging, output validation, and NIfTI volume summarization.

Finetunes the NV-Generate-CTMR MR-brain diffusion UNet from user-supplied NIfTI training volumes using a wrapper that stages configs and delegates to upstream scripts.

Reduces peak GPU memory in Megatron Bridge training by applying expandable segments, parallelism resizing, activation recompute, and CPU offloading constraints.

Manage NVIDIA Brev GPU instances for TAO training, evaluation, and inference using the Brev CLI and Docker.

Provides guidance for training Mixture-of-Experts models with long context windows, covering context parallelism sizing, selective recomputation, dispatcher choices, and practical patterns from recent experiments.

Validate and configure packed sequences and long-context training in Megatron-Bridge, distinguishing offline packed SFT for LLMs from in-batch packing for VLMs with correct context parallelism constraints.

Convert single-node PyTorch distributed scripts into multi-node Slurm sbatch jobs and debug common multi-node failures, covering srun-native and torch.distributed approaches, container setup, NCCL timeouts, and interactive allocation.