oimiragieo Verify the claim, not the label
The rule: a SUCCESS / DONE / RESOLVED / PASSING / CACHED / READY label is a HYPOTHESIS, not ground truth. Verify it against the bytes, the per-sample data, or the real code before trusting it — including claims you just made yourself.
This extends the SILENT-SUCCESS REFLEX (CLAUDE.md, 2026-06-25 arc) to the hardest case: catching your OWN "it works" conclusions before they propagate.
Why labels mislead
A label encodes what was believed at the moment it was written, not what the system does. Tests pass against what the author expected, not the full behavior space. Docs drift the moment code changes. A green CI check can prove nothing about the path that actually fails in production (siliconopera.com, 2026). The longer a label has been trusted, the more dangerous: "a test passing for 18 months on a stale assumption is confidence you never paid for" (dev.to/sophielane). The trap deepens when the same agent writes implementation and verification — both encode the same misunderstanding (plumblinehq.ai, beingagent.org).
Red-flag table
| Label you see | What it actually guarantees | What to check instead |
|---|---|---|
"exit 0" / "DONE" |
The process did not crash | Was a real artifact produced? (count files, check size > 0) |
"tests passed" |
The tests that exist passed | Do those tests exercise the code path that could fail? |
"RESOLVED / FIXED" |
Someone believed it was fixed at commit time | Run the failing scenario against the actual fix in the actual env |
"CACHED / READY" |
A cache entry exists | Is the cached file non-zero and uncorrupted? Load it |
"extraction floor (must be harness bug)" |
A narrative that sounds mechanistically complete | Pull per_item data; distinguish real floor from under-elicitation |
"it completes" |
The happy path ran | Did ALL the sub-tasks complete? (chunked gate, multi-bench suite) |
| Comments / docs / CLAUDE.md lore | State of the system at time of writing | grep the ACTUAL running code; lore has a shelf life |
Verification protocol (3 questions)
- What concrete artifact proves this is done? (non-zero file, specific stdout line, per-item score distribution — not just an exit code)
- Would I have seen a different label if it were broken? If not, the label is decorative (plumblinehq.ai: "a green is only evidence if a broken system would have gone red").
- Am I reading the code that runs, or the doc that describes what was intended to run? Code is ground truth; comments rot (beingagent.org).
Illustrative cases — where this discipline pays off
A "resolved" bug that was actually a 3-layer wrong-lore chain. A CI gate was believed "BROKEN" (exit crash). Layer 1 diagnosis blamed one dependency. Layer 2 blamed another. Both wrong — the real cause was a cumulative resource issue that only triggers when the full file set runs together. None of the three layers was verified against the running code before being recorded; each was accepted on narrative fit alone. Lesson: a plausible mechanism story is not a verified mechanism. Grep the real crash stack; test the hypothesis before banking it as lore.
A cache said READY but was corrupt. An artifact existed on disk with a "cached" marker file next to it. Assumed: usable. Reality: the write had been interrupted and the artifact was truncated to zero bytes — the marker was written, the payload wasn't. Lesson: presence of a cache entry is not integrity. Check file size and actually load the thing before trusting the marker.
A generated report exists but is empty. A report-generation step exited 0 and produced a file at the expected path. First read: "the report generated successfully." Opening the file showed zero rows — the generator had silently no-op'd on an empty input set upstream. Lesson: "the file exists" and "the file has content" are different claims; check both.
CI shard 1 passed but shards 2-3 never ran. After shard 1 of a 3-shard CI run finished green, the claim "CI passed" was written before shards 2 and 3 had even been dispatched. The full suite was required to establish the real outcome. Lesson: a partial run completing is not "it completes." Count the sub-tasks; confirm the final artifact against the expected total.
Applying it to your OWN claims
The CLAUDE.md SILENT-SUCCESS REFLEX originally targeted external signals (DONE-with-empty- harvest, gate-exit-0-but-crashed). This skill extends it inward: your own live check is also a hypothesis. Before writing "I ran it and it worked," confirm the output file is non-zero, stdout contains the expected terminal line, and per-sample data matches the interpretation. Trust comes from the evidence, not the label.
Cross-references
- CLAUDE.md "SILENT-SUCCESS REFLEX" — the parent discipline