Testing & Verification
Completion means the requested behavior and the relevant invariants are supported by evidence, not merely that the edit is finished. Evidence is the command, its meaningful result, and its scope.
Truth policy
- Never fabricate test output, benchmark numbers, or passing checks.
- Never claim success without a shown run. "Tests pass" requires the runner output.
- Never hide or downplay a failure. Never weaken, delete, skip, or mock away a failing test, lint rule, or type check to get green. If a test is wrong, fix it honestly and say why.
- When uncertain, say so and state how to resolve it.
Risk-based ladder
Run the cheapest check that can disprove the change, then expand as risk warrants:
- Static or structural check on edited files (lint, type check, syntax).
- One focused test that exercises the changed behavior or the regression.
- Neighboring unit or integration tests across the affected boundary.
- Build, type check, lint, migration validation, or end-to-end journey when the change can affect them.
- Security, performance, compatibility, or rollback checks for high-impact paths.
Match evidence to risk: public contract or schema → compatibility and consumer checks; auth, payment, privacy, or destructive path → denied cases, audit behavior, rollback; concurrency or retry → ordering, cancellation, idempotency, race-sensitive tests; UI → loading, empty, error, keyboard, responsive, accessibility states; configuration or deployment → parse, dry-run, environment differences, safe rollback.
Design strong tests
- Test behavior at the lowest level that observes the real failure mechanism. Use integration or contract tests when serialization, persistence, framework wiring, or service boundaries matter. Keep a small end-to-end set for critical journeys.
- Cover the happy path plus meaningful validation, authorization, boundary, concurrency, and failure cases according to risk.
- Prefer real behavior over excessive mocking; mock only unstable or external boundaries, and only with a mock that can fail like the real dependency.
- Deterministic clocks, identifiers, randomness, and schedulers over sleeps and retries.
- Avoid tests that mirror implementation, snapshots with unreviewed noise, or assertions that cannot fail.
- For a bug: reproduce first, then add a regression test that fails on the original mechanism.
Flaky suites
Reproduce and classify the cause: timing, order, shared state, network, resource, or environment. Fix isolation or synchronization. Quarantine only with an owner, an issue, evidence, and an expiry.
Completion gate
Run every applicable repository gate once implementation is complete, with evidence shown. Skip a gate only when it genuinely does not apply, and say which and why. On any failure the task is not done: report it, fix the cause, re-run. Procedure, per-stack commands, and what counts as evidence: references/completion-gate.md.
End with an honest status line: what passed (with output), what was skipped and why, what remains.
1---2name: testing-verification3description: Select tests by risk, design strong regression and boundary tests, handle flaky suites, and run the completion gate with evidence: exact command, result, scope, and every skipped or blocked check. Use when choosing what to test, judging whether work is done, or producing release evidence. Never weaken a failing test to get green.4license: MIT5---67# Testing & Verification89Completion means the requested behavior and the relevant invariants are supported by evidence, not merely that the edit is finished. Evidence is the command, its meaningful result, and its scope.1011## Truth policy1213- Never fabricate test output, benchmark numbers, or passing checks.14- Never claim success without a shown run. "Tests pass" requires the runner output.15- Never hide or downplay a failure. Never weaken, delete, skip, or mock away a failing test, lint rule, or type check to get green. If a test is wrong, fix it honestly and say why.16- When uncertain, say so and state how to resolve it.1718## Risk-based ladder1920Run the cheapest check that can disprove the change, then expand as risk warrants:21221. Static or structural check on edited files (lint, type check, syntax).232. One focused test that exercises the changed behavior or the regression.243. Neighboring unit or integration tests across the affected boundary.254. Build, type check, lint, migration validation, or end-to-end journey when the change can affect them.265. Security, performance, compatibility, or rollback checks for high-impact paths.2728Match evidence to risk: public contract or schema → compatibility and consumer checks; auth, payment, privacy, or destructive path → denied cases, audit behavior, rollback; concurrency or retry → ordering, cancellation, idempotency, race-sensitive tests; UI → loading, empty, error, keyboard, responsive, accessibility states; configuration or deployment → parse, dry-run, environment differences, safe rollback.2930## Design strong tests3132- Test behavior at the lowest level that observes the real failure mechanism. Use integration or contract tests when serialization, persistence, framework wiring, or service boundaries matter. Keep a small end-to-end set for critical journeys.33- Cover the happy path plus meaningful validation, authorization, boundary, concurrency, and failure cases according to risk.34- Prefer real behavior over excessive mocking; mock only unstable or external boundaries, and only with a mock that can fail like the real dependency.35- Deterministic clocks, identifiers, randomness, and schedulers over sleeps and retries.36- Avoid tests that mirror implementation, snapshots with unreviewed noise, or assertions that cannot fail.37- For a bug: reproduce first, then add a regression test that fails on the original mechanism.3839## Flaky suites4041Reproduce and classify the cause: timing, order, shared state, network, resource, or environment. Fix isolation or synchronization. Quarantine only with an owner, an issue, evidence, and an expiry.4243## Completion gate4445Run every applicable repository gate once implementation is complete, with evidence shown. Skip a gate only when it genuinely does not apply, and say which and why. On any failure the task is not done: report it, fix the cause, re-run. Procedure, per-stack commands, and what counts as evidence: `references/completion-gate.md`.4647End with an honest status line: what passed (with output), what was skipped and why, what remains.
AL-JANEF