zavelinski chain-of-verification
Draft, then interrogate your own draft. Generate verification questions about each claim, answer them independently, and rewrite the answer to match the verified facts. Catches confident-but-wrong details before they ship.
Why this exists (evidence)
- Chain-of-Verification (Dhuliawala et al., arXiv:2309.11495): a draft -> plan verification questions -> answer them independently -> revise loop measurably reduces hallucination across list, QA, and long-form generation vs answering in one pass.
- Key mechanism: answering the verification questions WITHOUT looking at the original draft prevents the model from rubber-stamping its own mistake.
When to use
- Multi-claim answers: research summaries, comparisons, "list all X", audits, post-mortems.
- Any answer where one wrong fact (a version, a number, an API, a name) poisons the whole.
- NOT for trivial single-fact answers (use cite-guard) or pure opinion.
The loop
- Draft the answer normally.
- Plan checks: extract each discrete factual claim and write one verification question per claim ("Does function X take arg Y?", "Was the release date Z?").
- Answer independently: answer each verification question on its own, ideally by going to the source (read the code, fetch the doc, run it), NOT by re-reading the draft. Independence is the whole point.
- Revise: correct any claim the checks contradict. Drop claims you cannot verify or mark them unverified.
- Ship the revised answer.
How to run it
- Inline for a handful of claims.
- Scaled: a Workflow stage that emits the claim list, a parallel stage that verifies each independently, then a synthesis stage that rewrites. Route verification to a fresh context so it does not see the draft.
Composes with
cite-guard: CoVe verifies claims by re-deriving them; cite-guard attaches the source. Use together: verify, then cite.self-consistency: vote across answers; CoVe checks one answer's claims. Different axes, combine for high-stakes.adversarial-verify: the code-behavior analogue.
Honest limits
- Verification questions answered from the same stale knowledge can still be wrong; prefer first-party sources for the checks.
- Costs an extra pass (draft + verify + revise). Worth it when a wrong detail is expensive, not for everything.
- Reported gains are on the paper's benchmarks; measure your own.