# Verify The Claim Not The Label

> Use when a status label (DONE / RESOLVED / PASSING / CACHED / READY / SUCCESS / "it completes") is the only evidence that something works. Triggers on: "the gate is green", "that bug is resolved", "the cache is ready", "I ran it and it passed", "the test completed", "a generated report exists but is empty", "CI shard 1 passed but shards 2-3 never ran", "a cache said READY but was corrupt". The discipline: a SUCCESS label is a HYPOTHESIS until the bytes / per-sample data / real code confirm it. Extends SILENT-SUCCESS REFLEX (already in CLAUDE.md) to the agent's OWN claims and OWN live checks, not just external signals.

- Skill: `oimiragieo/verify-the-claim-not-the-label` (Agent Skill)
- Install (CLI): `npx skillmds@latest add oimiragieo/verify-the-claim-not-the-label`
- Raw SKILL.md: https://api.skillmd.com/api/skills/oimiragieo/verify-the-claim-not-the-label/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: oimiragieo (https://skillmd.com/u/oimiragieo)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/oimiragieo/verify-the-claim-not-the-label

---


# Verify the claim, not the label

**The rule:** a SUCCESS / DONE / RESOLVED / PASSING / CACHED / READY label is a
**HYPOTHESIS**, not ground truth. Verify it against the bytes, the per-sample data,
or the real code before trusting it — including claims you just made yourself.

This extends the SILENT-SUCCESS REFLEX (CLAUDE.md, 2026-06-25 arc) to the hardest
case: catching your OWN "it works" conclusions before they propagate.

## Why labels mislead

A label encodes what *was believed* at the moment it was written, not what the system
does. Tests pass against what the author expected, not the full behavior space. Docs
drift the moment code changes. A green CI check can prove nothing about the path that
actually fails in production (siliconopera.com, 2026). The longer a label has been
trusted, the more dangerous: "a test passing for 18 months on a stale assumption is
confidence you never paid for" (dev.to/sophielane). The trap deepens when the same
agent writes implementation *and* verification — both encode the same misunderstanding
(plumblinehq.ai, beingagent.org).

## Red-flag table

| Label you see | What it actually guarantees | What to check instead |
|---|---|---|
| `"exit 0"` / `"DONE"` | The process did not crash | Was a real artifact produced? (count files, check size > 0) |
| `"tests passed"` | The tests that exist passed | Do those tests exercise the code path that could fail? |
| `"RESOLVED / FIXED"` | Someone believed it was fixed at commit time | Run the failing scenario against the actual fix in the actual env |
| `"CACHED / READY"` | A cache entry exists | Is the cached file non-zero and uncorrupted? Load it |
| `"extraction floor (must be harness bug)"` | A narrative that sounds mechanistically complete | Pull per_item data; distinguish real floor from under-elicitation |
| `"it completes"` | The happy path ran | Did ALL the sub-tasks complete? (chunked gate, multi-bench suite) |
| Comments / docs / CLAUDE.md lore | State of the system at time of writing | grep the ACTUAL running code; lore has a shelf life |

## Verification protocol (3 questions)

1. **What concrete artifact proves this is done?** (non-zero file, specific stdout line,
   per-item score distribution — not just an exit code)
2. **Would I have seen a different label if it were broken?** If not, the label is
   decorative (plumblinehq.ai: "a green is only evidence if a broken system would have
   gone red").
3. **Am I reading the code that runs, or the doc that describes what was intended to
   run?** Code is ground truth; comments rot (beingagent.org).

## Illustrative cases — where this discipline pays off

**A "resolved" bug that was actually a 3-layer wrong-lore chain.** A CI gate was
believed "BROKEN" (exit crash). Layer 1 diagnosis blamed one dependency. Layer 2
blamed another. Both wrong — the real cause was a cumulative resource issue that only
triggers when the full file set runs together. None of the three layers was verified
against the running code before being recorded; each was accepted on narrative fit
alone. **Lesson:** a plausible mechanism story is not a verified mechanism. Grep the
real crash stack; test the hypothesis before banking it as lore.

**A cache said READY but was corrupt.** An artifact existed on disk with a "cached"
marker file next to it. Assumed: usable. Reality: the write had been interrupted and
the artifact was truncated to zero bytes — the marker was written, the payload wasn't.
**Lesson:** presence of a cache entry is not integrity. Check file size and actually
load the thing before trusting the marker.

**A generated report exists but is empty.** A report-generation step exited 0 and
produced a file at the expected path. First read: "the report generated successfully."
Opening the file showed zero rows — the generator had silently no-op'd on an empty
input set upstream. **Lesson:** "the file exists" and "the file has content" are
different claims; check both.

**CI shard 1 passed but shards 2-3 never ran.** After shard 1 of a 3-shard CI run
finished green, the claim "CI passed" was written before shards 2 and 3 had even been
dispatched. The full suite was required to establish the real outcome. **Lesson:** a
partial run completing is not "it completes." Count the sub-tasks; confirm the final
artifact against the expected total.

## Applying it to your OWN claims

The CLAUDE.md SILENT-SUCCESS REFLEX originally targeted external signals (DONE-with-empty-
harvest, gate-exit-0-but-crashed). This skill extends it inward: your own live check is
also a hypothesis. Before writing "I ran it and it worked," confirm the output file is
non-zero, stdout contains the expected terminal line, and per-sample data matches the
interpretation. **Trust comes from the evidence, not the label.**

## Cross-references

- CLAUDE.md "SILENT-SUCCESS REFLEX" — the parent discipline