Perturb

Reproduce a flaky/timing-dependent kernel failure (suspected undocumented HW race) by injecting NOPs/delays to shift inter-thread / inter-kernel / inter-core timing until the failure becomes frequent enough to isolate and minimize into a deterministic reproducer. Sweeps NOP count × injection position × actor, records the max-error scenario PER actor to a report file (rewritten after each actor finishes so you can stop early), then (on request) minimizes the test around a chosen scenario. Works for tt-metal ttnn op tests and tt-llk kernel tests.

tenstorrent Updated 125 repo stars

File contents

tenstorrent/tt-metal commit 76fbd32218

Frequently asked questions

npx get-skill add tenstorrent/perturb