Numerics Debugging

Capture and compare per-op activations between two TorchTitan runs to spot numerics divergence (eager vs aot_fx_trace, FSDP vs no-FSDP, before vs after a refactor). Use when the user wants to debug bitwise / numeric drift in training, or invokes /numerics_debugging.

pytorch Updated
torchtitan

File contents

pytorch/torchtitan commit 6c4e00d62c

Frequently asked questions

npx get-skill add pytorch/numerics-debugging