# Test Discipline > Write isolated order-independent automated tests that assert on observable behavior rather than private internals. Use when writing reviewing or debugging any test (unit integration or end-to-end) in any language especially when tests share fixtures touch global or filesystem state flake depending on run order or reach into implementation details. - Skill: `niksavis/test-discipline` (Agent Skill) - Install (CLI): `npx skillmds@latest add niksavis/test-discipline` - Raw SKILL.md: https://api.skillmd.com/api/skills/niksavis/test-discipline/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Integrations & APIs - Author: niksavis (https://skillmd.com/u/niksavis) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/niksavis/test-discipline --- # Test Discipline Keep a test suite trustworthy as it grows. These are judgment rules, not lint — the deterministic floor (formatters, type checkers, the test runner itself) cannot catch a test that passes today but lies. Apply them when authoring or reviewing any automated test, in any language. ## Isolation and ownership - A test owns its inputs. Build the fixtures it needs and tear them down; never read or assert on state that another test, a previous run, or the developer's machine happened to leave behind — a test that reads state it did not create is broken even when it passes today. - A test must pass alone, in any order, and in parallel: no shared mutable state, no reliance on execution order, no hidden dependency on a sibling running first. ## Assert on behavior, not internals - Assert on observable behavior and public contracts — return values, emitted output, externally visible side effects — not private fields, call counts, or internal structure a refactor should be free to change. - A test that breaks on a pure refactor is testing the implementation, not the contract. Prefer the coarsest assertion that still proves the behavior. ## Determinism - Remove nondeterminism at the source: pin the clock, seed randomness, stub the network and the filesystem. A flaky test is a broken test, not a "sometimes" test — fix or quarantine it, never mask it with a blind retry. - When you fix a bug, add the regression test that would have caught it first, and watch it fail before the fix so you know it exercises the defect. - Wait for the condition, never for a duration: poll the predicate with a generous timeout so the assertion says the thing happened, not when. Where elapsed time is the claim, measure it on a monotonic clock, and set any wall-clock cap to clear the slowest runner in the CI matrix by 2x — a 3.0 s cap measured 0.48 s locally and 3.1 s on windows-latest, and was a flake generator, not a flake (basicly-7aler3). ## Set up the condition the code actually reads - When the code under test loads its own configuration, settings, or environment, put those values where that code reads them from — the file, the directory, the environment variable. A config object built in the test and handed to a helper never reaches a function that calls its own loader, so the test exercises the defaults and passes whatever you changed. - Then prove the condition is live rather than merely arranged: assert that the input you set up actually moves the value the code branches on. A threshold left at its default, a window too small to trigger, a flag read from somewhere else — each leaves a test that stages a scenario the code never sees. - The tell is a test that passes with the fix removed. Whenever a test is written after the fix it pins, delete the fix and watch it fail for the stated reason before trusting it. ## Mutate the fix in both directions, not just off - Deleting the fix proves the tests notice it is gone. It does not prove they describe what it *does*. So run the second mutation too: make the check fire on everything — always warn, always refuse, always report the same verdict — and watch a control fail. If nothing fails, the suite pins "it reacts" and not "it discriminates", and a check that fires on every input passes every test you wrote while being useless. - This is the direction that finds the missing test, and it has done so every time it was run. A per-lane report that labelled each lane refused passed the whole file, because every case in it was a refused lane; the over-trigger mutation is what showed the admitted-lane control was never written. - Say which direction found what when recording the result, so the next reader can tell a check that was verified from one that was merely exercised. - **Restore from a copy, never `git checkout -- `.** A mutation is applied to a working tree that usually holds the very change being verified, and `git checkout` restores from the index — silently discarding all of it. Copy the file aside first and copy it back. (Recovering from this mid-session cost a full re-apply of an uncommitted module.) ## A zero result needs a positive control - A search, scan or query that finds nothing proves nothing by itself. It is evidence only if the same method, run the same way, finds a control you already know is present; without that control the honest finding is "the method was inconclusive", not "the thing is absent". - The incident: `strings` over the copilot CLI binary found no hook identifiers, and that was reported as the capability being absent. A control retracted it — strings the CLI had been directly observed emitting (`allow-all-tools`, `premiumRequests`) also returned zero matches, because the payload is a compressed JS bundle. - The same trap in a test: an assertion that something is *absent* passes just as happily when the locator is wrong, so pin the locator with a case that must match before trusting the case that must not. ## A seam is not duplication - A module-level alias or thin wrapper around an external dependency — a CLI, a clock, an HTTP client — is usually a deliberate test seam: it is the thing tests replace. Several modules each holding their own is not accidental repetition. - So before consolidating duplicated access behind one shared helper, look at what the tests patch. Moving the call behind a new module silently bypasses every patch of the old alias, and the suite fails en masse for a reason unrelated to the refactor's intent. - If the consolidation is still right, migrate the seam deliberately: move the patch points in the same commit, and keep one seam per module rather than one for the whole codebase.