crew-verify: evidence before claims
The words "done", "fixed", "works", "passing", "verified" are claims about the
world, and reading code is not observing the world. This skill is the gate
between finishing the work and describing the work.
The method: verification is claim-shaped
You don't "verify the code"; you verify claims. So start from the words,
not the diff:
- Write down the claims you are about to make, as bullet points, in the
words you'd actually use ("the flag parser now accepts
--target",
"the crash is fixed", "docs match the new behavior").
- For each claim, name the cheapest observation that would prove it. Then
go make that observation, this session.
- Anything you can't or didn't observe gets reported as exactly that:
"changed, not verified", with the reason. Under-claiming costs a
sentence; over-claiming costs the reader's trust and a broken deploy.
Minimum evidence per claim
| Claim |
Floor of evidence (observed this session) |
| "it compiles / builds" |
the build command's actual output, exit 0 |
| "tests pass" |
test run output with counts, and the tests exercise the change |
| "the bug is fixed" |
repro failing before the fix, passing after (both outputs) |
| "feature works" |
the real entry point exercised end-to-end, not only a unit harness |
| "UI looks right" |
rendered output actually seen (screenshot, curl of the page) |
| "faster now" |
before/after numbers, same machine, same workload |
| "docs are updated" |
every changed command re-run as written |
| "nothing else broke" |
the affected test suite run, not asserted |
Verify the verifier
The most seductive false pass: tests that pass because they never touched the
change. When a test run is your evidence, confirm the connection. Break the
change deliberately for one run (or check coverage/logs) and watch the test
fail, then restore. Green that can't go red is not evidence. Same for the
environment: confirm you ran the built artifact, current branch, fresh build.
Anything "verified" against a stale binary is the classic self-own.
The claims your change just falsified
Everything above checks whether the claims you are making are true. The other
half is the claims already written down that your change just made false. These
never fail a build, no test covers them, and they are read by the people you
most want to trust you.
They are also the most common way a verified change still ships a lie. A README
that says the page loads no JavaScript, written before you added a toggle. A
count in a doc, correct until you added the fourth item. A screenshot showing
the old copy. A comment describing the branch you just deleted.
Between finishing and reporting, ask: what did this change make untrue
somewhere else? Then grep for it rather than trying to remember.
| You changed |
Go grep for |
| Added, removed or renamed a thing in a set |
the count of that set, spelled out and numeric, and any list naming the members |
| Behavior a doc describes |
that doc's claim, and the quickstart that walks through it |
| Anything user-visible |
screenshots, recorded sessions, sample output, social-card images |
| A flag, command or path |
every place it appears outside code: README, help text, comments, other repos |
| A default |
the sentence somewhere that says what the default is |
Two rules that make this cheap:
- Grep for the claim, not the code. The stale sentence rarely contains the
identifier you changed. Search the number, the adjective, the old name.
- Fix it in the same commit. A follow-up commit for stale docs is a commit
that does not get made, and the gap between them is when someone reads it.
The trap worth naming: fixing a stale claim is itself a change, so it can
falsify a neighbouring claim in turn. Correcting a count, then leaving the same
count wrong two paragraphs down, is the usual shape. Re-grep after the fix.
Failure modes
| Trap |
Reality |
| "The change is trivial, no need to run it" |
Trivial changes have the least-reviewed blast radius; run it anyway |
| "I verified the change, so I'm done" |
You verified your claims; you did not check the claims your change falsified elsewhere |
| Verifying via the harness only |
Users run the entry point; the harness skips the wiring where bugs live |
| "Tests pass" (they didn't run the new path) |
See verify the verifier |
| Declaring victory from logs you expected |
Search for the failure signal too, not just the success line |
| Verifying once, then "one last tweak" |
Any edit after verification voids it; re-run the cheapest relevant check |
Definition of done
The claims list exists; every claim carries its observation (command + real
output, quoted, not summarized); everything unobserved is explicitly labeled
unverified; and the claims this change falsified elsewhere are found and fixed
in the same commit. Then, and only then, write the report or the commit
message.
1---2name: crew-verify3description: Use before claiming any work is done, fixed, working, or passing, and before committing or opening a PR. Converts each claim you're about to make into observed evidence from this session, and downgrades anything unproven to "changed but unverified".4---56# crew-verify: evidence before claims78The words "done", "fixed", "works", "passing", "verified" are claims about the9world, and reading code is not observing the world. This skill is the gate10between finishing the work and describing the work.1112## The method: verification is claim-shaped1314You don't "verify the code"; you verify *claims*. So start from the words,15not the diff:16171. Write down the claims you are about to make, as bullet points, in the18 words you'd actually use ("the flag parser now accepts `--target`",19 "the crash is fixed", "docs match the new behavior").202. For each claim, name the cheapest observation that would prove it. Then21 go make that observation, this session.223. Anything you can't or didn't observe gets reported as exactly that:23 "changed, not verified", with the reason. Under-claiming costs a24 sentence; over-claiming costs the reader's trust and a broken deploy.2526## Minimum evidence per claim2728| Claim | Floor of evidence (observed this session) |29|---|---|30| "it compiles / builds" | the build command's actual output, exit 0 |31| "tests pass" | test run output with counts, and the tests exercise the change |32| "the bug is fixed" | repro failing before the fix, passing after (both outputs) |33| "feature works" | the real entry point exercised end-to-end, not only a unit harness |34| "UI looks right" | rendered output actually seen (screenshot, curl of the page) |35| "faster now" | before/after numbers, same machine, same workload |36| "docs are updated" | every changed command re-run as written |37| "nothing else broke" | the affected test suite run, not asserted |3839## Verify the verifier4041The most seductive false pass: tests that pass because they never touched the42change. When a test run is your evidence, confirm the connection. Break the43change deliberately for one run (or check coverage/logs) and watch the test44fail, then restore. Green that can't go red is not evidence. Same for the45environment: confirm you ran the built artifact, current branch, fresh build.46Anything "verified" against a stale binary is the classic self-own.4748## The claims your change just falsified4950Everything above checks whether the claims you are *making* are true. The other51half is the claims already written down that your change just made false. These52never fail a build, no test covers them, and they are read by the people you53most want to trust you.5455They are also the most common way a verified change still ships a lie. A README56that says the page loads no JavaScript, written before you added a toggle. A57count in a doc, correct until you added the fourth item. A screenshot showing58the old copy. A comment describing the branch you just deleted.5960Between finishing and reporting, ask: **what did this change make untrue61somewhere else?** Then grep for it rather than trying to remember.6263| You changed | Go grep for |64|---|---|65| Added, removed or renamed a thing in a set | the count of that set, spelled out and numeric, and any list naming the members |66| Behavior a doc describes | that doc's claim, and the quickstart that walks through it |67| Anything user-visible | screenshots, recorded sessions, sample output, social-card images |68| A flag, command or path | every place it appears outside code: README, help text, comments, other repos |69| A default | the sentence somewhere that says what the default is |7071Two rules that make this cheap:7273- **Grep for the claim, not the code.** The stale sentence rarely contains the74 identifier you changed. Search the number, the adjective, the old name.75- **Fix it in the same commit.** A follow-up commit for stale docs is a commit76 that does not get made, and the gap between them is when someone reads it.7778The trap worth naming: fixing a stale claim is itself a change, so it can79falsify a neighbouring claim in turn. Correcting a count, then leaving the same80count wrong two paragraphs down, is the usual shape. Re-grep after the fix.8182## Failure modes8384| Trap | Reality |85|---|---|86| "The change is trivial, no need to run it" | Trivial changes have the least-reviewed blast radius; run it anyway |87| "I verified the change, so I'm done" | You verified your claims; you did not check the claims your change falsified elsewhere |88| Verifying via the harness only | Users run the entry point; the harness skips the wiring where bugs live |89| "Tests pass" (they didn't run the new path) | See *verify the verifier* |90| Declaring victory from logs you expected | Search for the failure signal too, not just the success line |91| Verifying once, then "one last tweak" | Any edit after verification voids it; re-run the cheapest relevant check |9293## Definition of done9495The claims list exists; every claim carries its observation (command + real96output, quoted, not summarized); everything unobserved is explicitly labeled97unverified; and the claims this change falsified elsewhere are found and fixed98in the same commit. Then, and only then, write the report or the commit99message.
Honorboxx