Results for “pipeline-parallelism”

Validate and configure packed sequences and long-context training in Megatron-Bridge, distinguishing offline packed SFT for LLMs from in-batch packing for VLMs with correct context parallelism constraints.

Designs and implements production-grade ML pipeline infrastructure: configures experiment tracking, creates orchestration DAGs, builds feature store schemas, deploys model registries, and automates retraining and validation workflows.

Provides expert guidance for distributed training with DeepSpeed, covering ZeRO optimization stages, pipeline parallelism, FP16/BF16/FP8, 1-bit Adam, and sparse attention.