Validate that a slow, opaque integration test (upgrade-suite scenarios, cluster/simtest) actually catches the failure it claims to: inject a minimal fault that should trip the property, confirm the predicted log evidence appears, then revert. A green run alone proves nothing — the catch is often recoverable, so the test passes either way and the log line is the real assertion.
Target: $ARGUMENTS (the property/test to validate, e.g. "upgrade-suite state continuity" or "honest validators reject a faulty one"). If empty, ask which test and which property.
Full methodology — the two outcome shapes (hard-fail vs recoverable), the
7-step loop, which side to inject, the specificity control, and two worked
ika examples (historical: cross_binary state continuity; cross-binary malicious-party
detection) — lives in dev-docs/playbooks/test-testing.md.
Read it and follow it; don't reinvent the methodology ad hoc.
The non-negotiables:
- The log is the assertion, not the exit code. Predict the exact log
string and which node's log it lands in before running; gate on grepping
for it at
$UPGRADE_TEST_DIR/validator-*/node.log. - Pick the injection side deliberately — producer output vs a party's outgoing round message are not interchangeable.
- Check specificity — only the faulty output/party should be rejected while the rest stays healthy; blanket failure means you're measuring collateral breakage, not the property.
- Revert every fault artifact — mark edits
// TEST-TESTING FAULT (temporary — DO NOT MERGE); afterwardsgit statusis clean andgrep -r "TEST-TESTING FAULT"is empty.