AeonDave Testing Reliability
This skill focuses on test trustworthiness, not syntax. A test suite that passes for the wrong reason is worse than no test.
When to activate
- Flaky, timing-sensitive, or order-dependent tests.
- Heavy mocks, unclear patch targets, or fakes that do not match real behavior.
- Production code changed only to make tests easier.
- Coverage increased without stronger assertions.
- CI failures that cannot be reproduced locally.
Reliability workflow
- Reproduce the failure or false confidence with the smallest test selection.
- Identify the hidden dependency: time, random, filesystem, network, process state, global state, locale, order, or concurrency.
- Replace sleeps with observable conditions or fake clocks/events.
- Move mocks to trust boundaries; prefer fakes for stateful collaborators.
- Remove test-only production hooks unless they represent a real public seam.
- Strengthen assertions around behavior and invariants, not implementation trivia.
- Run focused, repeated, and then broader checks.
Anti-patterns to remove
- Testing mock expectations instead of product behavior.
- Partial mocks that silently omit important side effects.
- Sleeping for async behavior instead of waiting on a condition.
- Shared mutable fixtures that couple tests by order.
- Golden outputs updated without reviewing semantic changes.
- Coverage-only tests that assert nothing meaningful.
Offensive testing focus
- Exploit harnesses must prove primitives, not just crash/no-crash.
- Fuzz reproducer tests must pin seed/input and minimized artifact.
- Network tooling tests should use captured transcripts, local test servers, or protocol fakes.
- Evasion tests must distinguish telemetry absence from collection failure.
Resources
Load on demand:
references/testing-anti-patterns.md— detailed anti-patterns and safer replacements.
Scripts:
scripts/find_polluter.py— run tests one by one to identify which test creates unwanted files or state.