# Autoverifier

> Rigorously verify the claims in a scientific or technical paper (or any research artifact) using the AutoVerifier six-layer protocol — corpus building, claim-triple extraction with provenance, intra-document consistency and overclaim detection, cross-source triangulation with contradiction root-cause analysis, external/commercial signal corroboration, and a final hypothesis matrix with a technology-maturity rating. Use this skill whenever the user wants to fact-check, audit, stress-test, peer-review, or assess the validity, reproducibility, novelty, or maturity of a paper, preprint, technical report, or "breakthrough" announcement, or asks whether a result is overclaimed, genuinely novel, independently corroborated, or commercially motivated — even if they never say the word "verify". Also trigger for literature triangulation, conflict-of-interest checks on authors, or technology-readiness / S&TI (scientific & technical intelligence) assessment.

- Skill: `yuntaod/autoverifier` (Agent Skill, multi-file: 5 files)
- Install (CLI): `npx skillmds@latest add yuntaod/autoverifier`
- Raw SKILL.md: https://api.skillmd.com/api/skills/yuntaod/autoverifier/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Security
- Author: yuntaod (https://skillmd.com/u/yuntaod)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/yuntaod/autoverifier

---


# AutoVerifier

A structured, agentic protocol for verifying the claims in technical literature. It turns a raw paper plus a question ("does this result actually hold up?") into a traceable, evidence-linked assessment, by forcing the analysis through six progressive layers instead of trusting free-text inference.

The core idea: decompose every assertion into **claim triples** `(Subject, Predicate, Object)`, attach **provenance** and **evidence links** to each one, and then progressively test those triples against (a) the document's own evidence, (b) the independent literature, and (c) real-world commercial/financial signals — aggregating the result into a scored **hypothesis matrix**. The discipline is the point: it catches overclaims, citation distortion, and undisclosed conflicts that a one-shot "summarize this paper" pass would repeat verbatim.

## What this skill is — and is not

- **It is a verification *protocol*, not an oracle.** It encodes how a careful, skeptical analyst with no domain expertise should investigate a claim. Output quality depends on the evidence actually retrieved and the reasoning applied at each step. Treat every verdict as "best-supported assessment given what was found," not ground truth.
- **Do not let the verifier overclaim.** This tool's whole purpose is to flag conclusions that outrun their evidence. Hold the output to the same standard: every verdict must cite the evidence behind it, confidence must be hedged, and "not found" / "insufficient evidence" is a valid and important result. A confident verdict with no traceable support is itself the failure mode this skill exists to catch.
- **Named individuals and organizations require extra care** (see Guardrails). Conflict-of-interest and financial findings (Layer 5) must be tied to a citable public record and stated as "potential / per public filings," never asserted as fact.

## Tools this skill expects

The protocol assumes an agent with: **web search and page fetch** (to find rebuttals, benchmarks, filings, news), the ability to **read uploaded documents** (PDF/HTML), and optionally **code execution** (for embeddings, a small vector store, or metric math on a large corpus). It runs end-to-end as a single agent calling these tools; no special infrastructure is required for a handful of papers. A real vector database only earns its keep on large corpora or in the reproducible multi-model variant — for a single target paper plus its critics, search + the agent's own context is enough.

## Step 0 — Scope the task

Before running anything, fix three things:

1. **Target artifact(s).** The paper / preprint / report / claim under examination. If the user uploaded a file, that's the seed. If they named a claim or a "breakthrough," find the primary source first.
2. **Target query.** Phrase the actual question as a falsifiable statement to test — e.g. *"Does method X achieve the [speedup / accuracy / efficacy] it claims?"* rather than *"tell me about X."* The query defines what "supported / refuted" means downstream.
3. **Domain calibration.** This protocol is field-neutral and must be tuned to the artifact's field before extraction begins — **domain expertise enters the pipeline here.** The field's core concepts and terminology, what counts as strong evidence, its standard performance metrics and how they're defined, the reference benchmarks and current state of the art a claim must be measured against, what a genuinely strong baseline is, how comparisons get gamed, and where financial/COI signals live all differ by field. Run the calibration in `references/domain_calibration.md` first; the resulting domain profile provisions every downstream layer and is carried through the whole pipeline.

Then pick a **mode** matching the stakes and the user's appetite for effort. State which mode you're running and why.

| Mode | Layers | Use when |
|---|---|---|
| **Quick scan** | 1–3 | "Is this paper internally sound / is it overclaiming?" Single-document audit. Fast. |
| **Standard** | 1–4 | "Does the literature corroborate this claim?" Adds independent triangulation. Good default. |
| **Full assessment** | 1–6 | High-stakes, contested, or commercially-tied claims. Adds external signals + scored hypothesis matrix. This is the case-study mode. |

Layers are modular and ordered: each consumes the structured output of the previous one. You may also run a single layer in isolation if the user asks (e.g. "just do an overclaim check" → Layer 3 on the uploaded paper).

## The pipeline

Run the layers in order for the chosen mode. Each layer has a dedicated reference file with the detailed protocol, prompts, and output schema — **read the reference file for a layer before executing it.**

1. **Corpus Construction & Ingestion** → `references/layer1_corpus.md`
   Collect the target plus related sources (rebuttals, follow-ups, benchmarks, surveys, author profiles). Score sources for quality/bias. Extract clean text with section structure, process figures, capture metadata. Produces the evidence base.

2. **Entity & Claim Extraction** → `references/layer2_extraction.md`
   Extract entities and decompose every assertion into `(Subject, Predicate, Object)` triples. Tag each with a 5-level **provenance** label and normalize quantitative metrics to comparable units. Produces a claim knowledge graph.

3. **Intra-Document Verification** → `references/layer3_intradoc.md`
   For each triple, locate in-document evidence and label it supports / contradicts / neutral (NLI-style). Check methodology→result coherence (scope, baseline fairness, reproducibility). Flag overclaims (abstract/conclusion exceeding the body). Produces a per-document consistency score.

4. **Cross-Source Verification** → `references/layer4_crosssource.md`
   Discover related claims via citation networks, semantic similarity, and shared-entity traversal; route every new source back through Layers 1–3. Align claim pairs, check citation fidelity, run contradiction root-cause analysis, and weight by source independence. Produces cross-source consensus labels.

5. **External Signal Corroboration** → `references/layer5_external_signals.md`
   Build a financial/COI profile per entity (filings, funding, officer records), map supply-chain dependencies via multi-hop reasoning, and assemble a strategic-event timeline. Distinguishes real R&D from announcement-driven positioning. *(Optional; highest care required.)*

6. **Hypothesis Matrix & Reporting** → `references/layer6_hypothesis_matrix.md`
   Aggregate provenance (L2) + consistency (L3) + consensus (L4) + signals (L5) into testable hypotheses, generate adversarial counter-hypotheses, estimate confidence from answer-stability + multi-model agreement, and assign each a label of **Supported / Needs Review / Likely Hallucination**, plus an overall **technology-readiness** rating and **alpha-signal** flag.

## Output

**The report must present a detailed, clearly-labeled output for every layer that was run — not only the final verdict.** Each layer produces a real deliverable (the corpus inventory, the entities and claim-triple table, the intra-document verdicts and overclaim table with the consistency score, the cross-source contradiction analysis with independence weighting, the per-entity signal profiles, and the hypothesis matrix), and **each gets its own section in the report**, so the chain of reasoning is auditable end to end and any final conclusion can be traced back through the intermediate evidence that produced it. A report that shows only the bottom line is a failure — the traceability *is* the product. Use the per-layer tables in `references/output_templates.md` and the section-per-layer skeleton in `assets/report_template.md`.

Bookend the per-layer detail with an executive frame: **open with a bottom-line answer** that directly answers the target query — stating the **alpha-signal determination (present / absent) first**, then a concrete, consolidated summary of the reasons collected from every layer that ran, and a one-line split of *what holds up* vs *what does not* (e.g. "the underlying system demonstrably works and that finding is high-confidence; the headline performance claim does not survive independent evaluation") — so the reader gets the full, traceable verdict before the detail; then **close with a plain-language recommendation**. Conflating "the thing exists / runs" with "the headline claim is true" in that summary is the single most common error in technical assessment.

Include a short summary of the Step 0 **domain profile** up front so the reader can see which field-specific lens was applied. Match the report's depth to the mode: a Quick scan reports Layers 1–3, Standard reports 1–4, and the Full assessment reports all six.

## Guardrails

- **Evidence-link everything.** No verdict without a source-text link or citation. If a claim can't be tied to retrieved evidence, label it "unverified," not "false."
- **Layer 5 / conflicts of interest.** State COI, financial, or supply-chain findings only against a citable public record (filings, company pages, news), and phrase them as *potential* and *per [source]*. Never assert that a named person has a conflict, stake, or motive as established fact. When public records are thin, say so and stop — do not infer motive.
- **Calibrate confidence, don't manufacture it.** Use Low / Medium / High and tie the level to agreement and answer-stability. Disagreement among sources or among generations is a finding to report, not noise to suppress.
- **Provenance honesty.** A projection ("we expect...") is Level 5 even when it sounds like a result. A single best-case number is not a typical result; report the distribution (median/range), not just the maximum.
- **Stay domain-agnostic.** The protocol carries no field-specific assumptions. When a field has an accepted external rubric for what makes this *class* of claim credible, retrieve it during domain calibration and inject it as a Layer-4 evaluation framework rather than hard-coding judgments — see `references/domain_calibration.md` and `references/layer4_crosssource.md`.

## Customizing the pipeline for a new domain

Each layer is a self-contained module with declarative prompts and a structured deliverable, so analysts can select, reorder, or extend layers. The protocol itself is field-neutral; everything domain-specific is established once, up front, by the Step 0 calibration in `references/domain_calibration.md`, which feeds a domain profile into the layers — what counts as experimental vs simulated vs theoretical evidence (Layer 2), the field's credibility rubric (Layer 4), its characteristic ways of gaming comparisons (Layer 3), and its relevant external-signal sources (Layer 5, e.g. trial registries and regulators in biomedicine instead of securities filings in commercial tech). The provenance taxonomy and the hypothesis-matrix labels are domain-independent and should not be changed.