oimiragieo Verify Claims Against Execution
Overview
An agent's claim of success is a hypothesis until external state confirms it — an exit code, a file:line citation that resolves, a delivery receipt, a liveness probe. Most reward-hack episodes ship a confident rationale framing hollow output as legitimate, so reading the claim tells you nothing. Check the world, not the words.
When to use
- Marking any task complete / fixed / passing / "succeeded".
- Accepting a subagent's self-reported result or score.
- Trusting a grader / oracle / judge / auto-rater — before you run it on a real batch.
- A result "looks right" but you haven't watched it run.
The technique
- Bidirectional oracle gate (do this FIRST). Before trusting any grader, feed it a known-good input (must PASS) and a known-empty/wrong stub (must FAIL). If the empty stub passes, the oracle is broken — halt and fix the oracle, not the agent. A grader you've only ever run against correct answers is unvalidated.
- Resolve, don't read. Demand evidence that resolves to external state: an exit code from the real test, a
file:linethat opens to a real path, a delivery-receipt row. A claim with no resolvable citation is unverified. - Isolate + recompute. Run the grader OUTSIDE the agent's sandbox; recompute expected output from first principles; give the agent no read/write to fixtures or pass/fail metadata.
- Hybrid signal. Combine execution-correctness + process-coherence + an independent judge. High coherence + low correctness = a reward-hack flag.
Common mistakes
- Trusting a grader that was never run against a wrong answer (the #1 trap).
- Blending an LLM-judge score INTO the oracle's pass/fail (let it advise, never gate).
- Write-only-at-end evaluation that loses the checkpoint on crash.
- Reading the agent's confident rationale as evidence.
Real-world impact
Live receipt: an agent audit run was marked succeeded while the worker wrote a stub emitting an empty report — the oracle only checked that 4 JSON keys existed, not that they had content. A bidirectional gate (an empty stub MUST fail) catches it instantly. Separately, a UI eval lane read as a capability gap was actually a broken oracle (a subprocess invocation silently failing on one OS) — always validate the oracle before blaming the agent.
Sources: arXiv 2605.02964 (Reward Hacking Benchmark), 2511.21654 (EvilGenie). Eval/CUJ-specific sibling: trustworthy-cuj-scoring.