Results for “moe”

Guides implementation of new model architectures in NeMo AutoModel through five phases: discovery, implementation, registration, validation, and testing.

Provides guidance for training Mixture-of-Experts models with long context windows, covering context parallelism sizing, selective recomputation, dispatcher choices, and practical patterns from recent experiments.

Launch, monitor, stop, and debug NeMo-RL recipes on a Kubernetes cluster using the nrl-k8s CLI, supporting ephemeral and long-lived RayCluster modes.

Train, evaluate, export, and run inference for Masked Auto-Encoder (MAE) models for self-supervised pretraining and fine-tuning of visual representations.

Maintain durable session memory across agent disconnects by writing structured checkpoints to the repo's session directory, enabling context recovery.

Guides agents through the full lifecycle of NeMo-RL experiments: understanding recipes, launching reproducible runs, analyzing results, and preserving human oversight with git and TSV logs.

Run evaluation tasks against a NeMo Platform server using the Evaluator plugin CLI and Python SDK.

Validate and use CUDA graph capture in Megatron Bridge, including local full-iteration graphs and Transformer Engine scoped graphs for attention, MLP, and MoE modules.