Testing Discipline
Trigger phrases: "write a test", "run the tests", "coverage", "are the tests green", "unit test", "integration test", "tests pass locally", "flaky test", "test is flaky"
Goal: behavior correctness — test real behavior without breaking product code just to make a test pass.
Principles
- Pyramid: many unit, fewer integration, few end-to-end (e2e). Limit e2e to critical flows.
- AAA: Arrange-Act-Assert; one test = one behavior.
- Isolation & determinism: external dependencies are mocked/faked; time and randomness are fixed; test order is independent.
- Risk-coverage: risk, not metrics. Critical path + boundary + negative + authorization (IDOR/404) scenarios.
- Naming:
what_it_tests_under_which_condition_what_it_expects — on failure it is clear what broke.
- Red-green: first a failing test, then the implementation.
Watch out
- Flaky = bug: an occasionally failing test is not tolerated; it is fixed at the root.
- In snapshot/golden-file tests, avoid needless brittleness (assert only the meaningful output).
Tests that cannot fail
Green is evidence only if the test could have gone red. Shapes that check nothing:
- Assertion restates the code: it recomputes the implementation's own expression, so it moves with every change.
- Setup guarantees the result: arrange plants the value the assert reads back; it holds even if the act never ran.
- Expected value derived the way the code derives it: same formula, query or parser — the same wrong assumption on
both sides still matches.
- A double asserted against its own stub: the fake is told to return X and the test asserts X; only the fake runs.
- No assertion: it passes because nothing threw. If not throwing is the behavior under test, assert that.
Prove it can fail: break what the test covers — invert a condition, return a wrong constant — confirm it goes red,
then restore. Still green means the test is documentation, not a gate: fix it or delete it, never count it as coverage.
Flaky triage
An intermittent failure gets exactly one outcome, decided the day it is seen: fix it, quarantine it (out of
the gating lane, never out of the suite, with an owner and a date), or escalate it as the product defect it
signals. Infrastructure flakes may get a bounded, logged retry; product flakes never do. Classifying them, the retry
rule, and what a quarantine entry must carry: references/flaky-triage.md.
DoD (this skill's contribution)
- The project's own test command is green — read it off the manifest/CI (
dotnet test · npm test · pytest ·
go test ./...), don't assume one; critical paths are covered; no empty/meaningless tests.
1---2name: testing-33description: |4 The how of testing: pyramid, AAA, isolation, risk coverage, determinism. Guarantees the DoD's "tests are5 green". crew-test-expert applies it.6 Use when writing or changing tests, or when a suite is flaky, slow or green for the wrong reason.7metadata:8 source: https://skillmd.com/skills/byerlikaya/testing-39---1011# Testing Discipline1213<!-- routing-eval reads the next line; why it sits in the body: AGENT_TEMPLATE.md -->14Trigger phrases: "write a test", "run the tests", "coverage", "are the tests green", "unit test", "integration test", "tests pass locally", "flaky test", "test is flaky"1516Goal: **behavior correctness** — test real behavior without breaking product code just to make a test pass.1718## Principles19- **Pyramid:** many unit, fewer integration, few end-to-end (e2e). Limit e2e to critical flows.20- **AAA:** Arrange-Act-Assert; **one test = one behavior**.21- **Isolation & determinism:** external dependencies are mocked/faked; time and randomness are fixed; test order is independent.22- **Risk-coverage:** risk, not metrics. Critical path + **boundary** + **negative** + **authorization (IDOR/404)** scenarios.23- **Naming:** `what_it_tests_under_which_condition_what_it_expects` — on failure it is clear what broke.24- **Red-green:** first a failing test, then the implementation.2526## Watch out27- **Flaky = bug:** an occasionally failing test is not tolerated; it is fixed at the root.28- In snapshot/golden-file tests, avoid needless brittleness (assert only the meaningful output).2930## Tests that cannot fail3132Green is evidence only if the test could have gone red. Shapes that check nothing:33- **Assertion restates the code:** it recomputes the implementation's own expression, so it moves with every change.34- **Setup guarantees the result:** arrange plants the value the assert reads back; it holds even if the act never ran.35- **Expected value derived the way the code derives it:** same formula, query or parser — the same wrong assumption on36 both sides still matches.37- **A double asserted against its own stub:** the fake is told to return X and the test asserts X; only the fake runs.38- **No assertion:** it passes because nothing threw. If *not throwing* is the behavior under test, assert that.3940**Prove it can fail:** break what the test covers — invert a condition, return a wrong constant — confirm it goes red,41then restore. Still green means the test is documentation, not a gate: fix it or delete it, never count it as coverage.4243## Flaky triage44An intermittent failure gets exactly one outcome, decided the day it is seen: **fix** it, **quarantine** it (out of45the gating lane, never out of the suite, with an owner and a date), or **escalate** it as the product defect it46signals. Infrastructure flakes may get a bounded, logged retry; product flakes never do. Classifying them, the retry47rule, and what a quarantine entry must carry: **`references/flaky-triage.md`**.4849## DoD (this skill's contribution)50- The project's own test command is green — read it off the manifest/CI (`dotnet test` · `npm test` · `pytest` ·51 `go test ./...`), don't assume one; critical paths are covered; no empty/meaningless tests.