Nemo Mbridge Perf Activation Recompute

Validate and use selective and full activation recompute in Megatron Bridge to reduce GPU memory usage at the cost of extra compute. Use for activation memory OOMs or regressions involving recompute_granularity, recompute_num_layers, recompute_modules, recompute_method, selective recompute, full recompute, or activation checkpointing.

NVIDIA-NeMo Updated 986 repo stars
image

File contents

NVIDIA-NeMo/Megatron-Bridge commit c1a76e48e7

Frequently asked questions

npx get-skill add nvidia-nemo-megatron-bridge/nemo-mbridge-perf-activation-recompute