# Source Verify

> Use when evidence records exist with verification_status pending and need to be checked before synthesis — either because the user asks to "verify these sources", "check if this quote actually supports the claim", "audit this evidence", or because the orchestrator has just finished a collection pass and is handing off to verification before synthesis. This is the mandatory station between any source skill and the synthesizer — no source skill's evidence reaches findings.jsonl without passing through here first. Do NOT use this to collect new evidence (that's the individual source skills) or to compute finding status (that's the deterministic validator, scripts/validate_run.py — this skill only resolves evidence-level verification_status).

- Skill: `ali-demirbas/source-verify` (Agent Skill)
- Install (CLI): `npx skillmds@latest add ali-demirbas/source-verify`
- Raw SKILL.md: https://api.skillmd.com/api/skills/ali-demirbas/source-verify/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Security
- Author: ali-demirbas (https://skillmd.com/u/ali-demirbas)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/ali-demirbas/source-verify

---


# Source Verify

## Overview

The mandatory checkpoint every evidence record passes through before it can be cited by any finding (constitution rule 6). No source skill verifies its own output — the actor that collects evidence never grades it. This skill's only job: turn `verification_status: pending` into `verified`, `rejected`, or `unreachable`, honestly, one record at a time.

**Layer rule:** this is where network-dependent verification happens (constitution rule 8) — archive liveness checks, re-fetching URLs, confirming a page still says what the record claims. `scripts/validate_run.py` never touches the network; everything network-dependent about "is this evidence real" happens here, once, before the validator ever runs.

## The two checks (constitution rule 6)

Every `pending` record gets both checks, in order. Both must pass for `verified`.

### Check (a) — Mechanical: does the quote exist?

Fetch `source.url`. Search for the `evidence.quote` text, whitespace-normalized (collapse multiple spaces/newlines, ignore leading/trailing whitespace) but otherwise exact — no fuzzy matching, no "close enough."

**Multi-part quotes.** A quote trimmed with `[...]` is split on that marker into parts. Every part must appear at the source, **in the same order**, with no more than ~2000 characters between consecutive parts. Without the order and distance constraints, "exact quote" means something different in every implementation, and a quote could be assembled from fragments scattered across an entire page.

**Fall through to the archive on either failure mode, not just one:**

1. Live URL loads and contains the quote → check (a) passes, `checked_via: live`.
2. Live URL loads but the quote is **not** there → **try the archive anyway.** A page edited after collection is the normal case here, not an exception: the evidence was real when collected, and the snapshot is what proves it. Found in the archive → passes, `checked_via: archive`, and `limitations` records that the live page has since changed. Treating a content-drifted page as a rejection would discard true evidence and quietly punish sources that update.
3. Live URL fails to load → try the archive. Found → passes, `checked_via: archive`.
4. Not found in either → **rejected** (`mechanical: not-found`). Do not proceed to check (b).
5. Neither loads at all → **unreachable** (`mechanical: unreachable`), not `rejected` — an availability failure is not evidence the claim is false.

**`archive-failed:` is not a URL.** When `archive_url` starts with `archive-failed:`, there is no archive to try — skip straight to the outcome the live check produced. Never issue a fetch against that marker.

### Check (b) — Semantic: does the quote, in context, support the claim?

Read the passage surrounding the quote — not just the quote in isolation. Ask: standing in its actual context, does this passage assert what `claim` says, at the scope `claim` states (same version, region, date range, magnitude)?

**How much context.** "Surrounding" needs a definition or the same record verifies differently on different days. Start with the quote's own paragraph, expand to the paragraph before and after, and expand once more to the enclosing section only if the claim is still ambiguous. Per source type: HTML → paragraph, then section; PDF → paragraph, then page; GitHub code → the enclosing function or block, then the file's surrounding context; GitHub issues and community threads → the parent comment plus the thread's opening post, since a reply's meaning frequently lives in what it is replying to.

- **Full support** → check (b) passes → `verified`.
- **No support** (context contradicts, or the quote was cherry-picked to imply something the passage doesn't actually say) → **rejected**.
- **Partial support** (the passage supports part of the claim but not all of it — a narrower scope, a smaller number, a qualified rather than absolute statement) → **not a rejection.** Run the shrink procedure below.
- **The passage itself attributes the claim to another source** ("according to X", "as reported by", wire-service framing, a citation to a paper) → do not verify this record as-is. Route back to engine/origin-tracing.md to resolve the actual root, then re-run both checks against the root's own record. The original record, if kept at all, is marked with `root_origin` pointing at the resolved root and is not independently `verified` as if it originated the claim.

## Shrink → re-verify (constitution rule 6, "partial is not a status")

When check (b) finds partial support:

1. Rewrite `claim` to state exactly what the quote supports — no more. ("Revenue grew 40% globally" → "Revenue grew 40% in the enterprise segment" if that's what the passage actually says.)
2. Append to `limitations`: `"claim narrowed from: <original claim text>"` — the narrowing is never silent.
3. **Increment `claim_revision`.** This is what makes the narrowing visible downstream: any finding citing this record must carry a matching `acknowledged_revisions` entry, or validation fails (engine/synthesis-contract.md). Without this, a finding written against the original broad claim would silently outlive the evidence that no longer supports it — the validator cannot compare a finding's sentence to a claim semantically, so the acknowledgment is what forces a human/synthesizer to look again.
4. Re-run check (b) against the **new** claim. It should now pass (the claim was built from what the quote supports) — if it somehow still doesn't, treat as `rejected` rather than looping.
5. Set `verification_status: verified` on the narrowed record.

This record is never discarded for being narrower than first collected — a correctly-scoped `verified` record is more valuable than a broad `rejected` one. Log the narrowing in the run ledger too, so the synthesizer knows which findings need re-examination.

## Procedure

1. Pull every `pending` record from the run's `evidence.jsonl`.
2. For each: run check (a), then check (b) if (a) passed. Apply shrink if partial. Apply the origin-tracing routing if the quote is itself a relay.
3. Write the resolved `verification_status` back to the record, **together with the `verification` block** (`mechanical`, `semantic`, `checked_via`, `checked_at`). Recording only the final status loses the two things an auditor most wants later: which of the two checks decided it, and whether the quote was confirmed against the live source or an archived snapshot. This station is the only one allowed to write either field after collection.
4. **Never delete a `rejected` or `unreachable` record.** They stay in `evidence.jsonl` as a permanent trail of what was checked and why it didn't hold up — `rejected` evidence simply becomes uncitable (engine/synthesis-contract.md: no finding may cite it).
5. Log a short verification summary per run in the ledger: counts of verified/rejected/unreachable/narrowed, and which records were rerouted to origin-tracing.

## What this skill does not do

- Does not decide finding `status` (`confirmed`/`probable`/`contested`/`insufficient`) — that ladder is computed only by `scripts/validate_run.py`, deterministically, from verified evidence. This skill's output (`verification_status`) is one of that computation's inputs, nothing more.
- Does not synthesize, compare across records, or write findings.
- Does not re-collect evidence — if a claim needs different/better evidence, that goes back to the relevant source skill, not to this station.

## Common mistakes

| Mistake | Fix |
|---|---|
| Treating "URL is down" as `rejected` | That's `unreachable` — a different failure mode with different downstream handling |
| Discarding a partially-supported record | Shrink the claim and re-verify; never discard for being narrower |
| Shrinking a claim without bumping `claim_revision` | The bump is what forces findings to be re-checked against the narrowed claim — skipping it reintroduces the exact overclaim the shrink just fixed |
| Verifying a record whose quote attributes the claim elsewhere | Route to origin-tracing first, verify the resolved root instead |
| Checking the quote against the live URL only, skipping the archive on a fetch failure | Always try the archive before concluding `unreachable` |
| Rejecting a record because the live page no longer contains the quote | Try the archive first — an edited page is the normal case, and the snapshot is what the evidence was collected against |
| Fetching `archive-failed:<reason>` as if it were a URL | It is a marker, not an address; there is no archive to check |
| Letting the collecting source skill mark its own evidence `verified` | This station is separate from collection precisely so that never happens |