grimaldost Verification Before Completion
Evidence before claims — a rigid skill. The bright line: a completion claim is made only after the command that proves it has run in this session and its output has been read.
The gate
Before stating any status — done, fixed, passing, ready, complete:
- Identify the command that would prove the claim.
- Run it, fresh and in full — not a remembered result or partial check.
- Read the output: exit code, counts, failures, warnings.
(
references/non-vacuity.md: reading an exit code after a pipe.) - State the result that the output supports. Confirmed → the claim, with the evidence. Refuted → the actual state, with the output. A true "still failing" is a better deliverable than a false "done".
What claims require
| Claim | Requires | Not sufficient |
|---|---|---|
| Tests pass | This session's run: zero failures | A previous run, "should pass" |
| Linter clean | Linter output: zero errors | A partial check, extrapolation |
| Build succeeds | Build command: exit 0 | Linter passing, logs looking fine |
| Bug fixed | Symptom re-tested gone and a regression test that red-greens | Symptom gone, no test left behind |
| Regression test works | Red-green verified (below) | The test passing once |
| A check ran | The count of units it saw, non-zero | Exit 0 with no output |
| Data output correct | A hard case named and its expected value written down before the fix, then recomputed by a path that does not reuse the transform | Totals that reconcile; a clean run |
| Doc/report claim accurate | The cited span read whole — the file, not a line range | A file:lo-hi citation ending mid-structure |
| Delegated work done | The diff inspected, checks re-run | The agent's success report |
| Requirements met | Line-by-line check against the plan | Tests passing |
| Artifact ships right (wheel, image, bundle) | The built artifact inspected directly | A green editable/CI run — it may never build it |
Regression tests are red-green verified
Write the test → it passes → revert the fix → the test fails → restore the fix → it passes again. A regression test never seen failing against the bug proves nothing about it.
A bug fix is not done until that test exists — even a one-line fix, even when you judged the full test-driven-development cycle not worth loading (its exceptions still hold: throwaway spikes, generated code, pure config — agreed with the user, not self-granted). "The fix is obvious" is how a fixed bug comes back later; shipping without the test is an unverified durability claim, not a smaller scope.
A verifier is trusted green only after it has been seen red, over real input
The regression-test rule above is one case of a principle governing any
verifier — a gate, a parity diff, a contract check, an eval assertion. A check
seen only green is indistinguishable from one that tests nothing, and a green
over an empty scan is worse than a red one because it gets quoted as
evidence. Confirm both halves: that it read the input you think it did — count
the files, rows, or cases it saw — and that it can fail. Plant a known
violation, confirm the catch, then remove the plant by the inverse edit,
never a checkout. (references/non-vacuity.md.)
Delegated work
A subagent's "success" is a claim, not evidence. Inspect the diff, run the verification yourself, and report the state you observed, including any gap between report and diff.
Finishing a change
Before committing, opening a PR, or moving on: the full test command ran this
session with zero failures; requirements re-read and checked off
individually; the output is pristine — no stray errors, warnings, or a jumped
runtime riding along. Recurring checks belong in pre-commit or CI, not memory —
mechanism outlasts intention. When the suite carries irreducible pre-existing
failures, the honest gate is "zero net regression" against a baseline, not an
absolute zero (references/non-vacuity.md).
Wording that signals an unverified claim
"Should work", "probably passes", "seems fixed", and satisfaction expressed before the verification ran — each marks a claim outrunning its evidence: run the proving command or say plainly that verification hasn't happened.
Boundaries
Designing what to verify — suite shape, coverage strategy — is test-strategy work. This skill governs the moment of claiming. The reproducing-test cycle for a fix belongs to test-driven-development; this skill takes the evidence from there — and, when that cycle wasn't loaded, still refuses a fix's completion claim without its cheap core, a red-green regression test.