grimaldost Verification Before Completion
Evidence before claims — a rigid skill. The bright line: a completion claim is made only after the command that proves it has run in this session and its output has been read.
The gate
Before stating any status — done, fixed, passing, ready, complete:
- Identify the command that would prove the claim.
- Run it, fresh and in full — not a remembered result or partial check.
- Read the output: exit code, counts, failures, warnings. (
$?is the LAST command's exit code — read it right after the bare command, never after a pipe; capture output to a file instead.) - State the result that the output supports. Confirmed → the claim, with the evidence. Refuted → the actual state, with the output. A true "still failing" is a better deliverable than a false "done".
What claims require
| Claim | Requires | Not sufficient |
|---|---|---|
| Tests pass | This session's run: zero failures | A previous run, "should pass" |
| Linter clean | Linter output: zero errors | A partial check, extrapolation |
| Build succeeds | Build command: exit 0 | Linter passing, logs looking fine |
| Bug fixed | Symptom re-tested gone and a regression test that red-greens | Symptom gone, no test left behind |
| Regression test works | Red-green verified (below) | The test passing once |
| Delegated work done | The diff inspected, checks re-run | The agent's success report |
| Requirements met | Line-by-line check against the plan | Tests passing |
| Artifact ships right (wheel, image, bundle) | The built artifact inspected directly | A green editable/CI run — it may never build it |
Regression tests are red-green verified
Write the test → it passes → revert the fix → the test fails → restore the fix → it passes again. A regression test never seen failing against the bug proves nothing about it.
A bug fix is not done until that test exists — even a one-line fix, even when you judged the full test-driven-development cycle not worth loading (its exceptions still hold: throwaway spikes, generated code, pure config — agreed with the user, not self-granted). "The fix is obvious" is how a fixed bug comes back later; shipping without the test is an unverified durability claim, not a smaller scope.
A verifier is trusted green only after it has been seen red, over real input
The regression-test rule above is one case of a principle governing any
verifier — a gate, a parity diff, a contract check, an eval assertion. A check
seen only green is indistinguishable from one that tests nothing (a typo'd join
key, a tolerance nothing trips, a fixture that hits a fallback). Confirm both
halves: that it read the input you think it did — count
the files, rows, or cases it saw — and that it can fail. Plant a known
violation, confirm the catch, then remove the plant by the inverse edit; a
checkout also discards whatever else was in that file, and only a byte-precise
restore is verifiable. references/evidence-fabrication.md catalogues the four
ways evidence itself goes wrong: invented events, confabulated anchors, silence
read as finished, a gate green because it never ran.
(data-engineering-discipline, when installed, carries the canonical
non-vacuity matrix: its scenarios.md, Step 8.2.)
Delegated work
A subagent's "success" is a claim, not evidence. Inspect the diff, run the verification yourself, and report the state you observed, including any gap between report and diff.
Finishing a change
Before committing, opening a PR, or moving on: the full test command ran this session with zero failures; requirements re-read and checked off individually; the output is pristine — no stray errors or warnings riding along. Recurring checks belong in pre-commit or CI, not memory — mechanism outlasts intention. When the suite carries irreducible pre-existing failures, the honest gate is "zero net regression" against a baseline, not an absolute zero: diff this run's failure set against the baseline's and require the difference to be empty — capture the baseline by stashing the change and running the suite, or by running it at the base commit.
Wording that signals an unverified claim
"Should work", "probably passes", "seems fixed", and satisfaction expressed before the verification ran — each marks a claim outrunning its evidence: run the proving command or say plainly that verification hasn't happened.
Boundaries
Designing what to verify — suite shape, coverage strategy — is test-strategy work. This skill governs the moment of claiming. The reproducing-test cycle for a fix belongs to test-driven-development; this skill takes the evidence from there — and, when that cycle wasn't loaded, still refuses a fix's completion claim without its cheap core, a red-green regression test.