AI Agents External Claims

ai-agents External Claims

This repo runs verification-based governance: a claim is true when a primary source, gate, or command output says it is, not when a vendor blog, a summary, or an agent asserts it. This skill is the front-gate for one specific claim class: external, vendor, and third-party assertions that arrive from outside the repo and are about to land inside a repo artifact (an ADR, a retro, a memory, a PR description, a session log) or an external deliverable. A wrong number un-cited from a published artifact costs more than the one or two fetches that would have caught it in the same pass.

Audience: a zero-context contributor (human or model) holding an external claim and deciding whether it may enter a durable artifact.

This skill is the discipline for verifying claims that originate OUTSIDE the repo. For proving an idea that originates inside the repo (hypothesis, probe, eval baseline), use ai-agents-research-methodology. For what counts as test evidence for code, use ai-agents-validation-and-qa. This skill hardens the general-purpose gstack claim-verification-before-ingest pattern into the repo's own artifact-ingest path.

Triggers

• verify an external claim
• check a vendor number
• is this stat real
• validate a third-party citation

Scope Boundaries

When This Skill Fires

Any of these conditions in a claim about to enter a durable artifact:

• Numerical headline claim: "over 1000 X", "N+ stars", "X% of teams", "$Y in spend", "Z% accuracy improvement". Round numbers ending in 100, 1000, or 10000 are usually rhetorical rounding, often inflated above the real, precise, non-round count.
• Named-authority attribution: "Gartner says", "an MIT study found", "Anthropic disclosed", "per the docs". The named authority is a claim about a source, not the source.
• Stake-holding author: vendor self-promotion, a founder pitch, an advocacy post quoting a competitor's unfavorable stat. The author benefits from the framing, so the framing is suspect until the primary source confirms it.
• Stat-of-a-stat chain: source A cites source B citing source C. The deeper the chain, the higher the drift risk. Follow it to the origin.
• Downstream commitment: the claim will land in an ADR, a retro, a Serena memory, a PR body, or an external artifact where "I will verify it later" does not survive the round trip.

If none of these fire, this skill does not apply. The front-gate has a real cost (one or two fetches); reserve it for claims where a wrong number would actually bite.

Process

Phase 1: Identify the claim and its primary source

Name the exact assertion in one sentence and the ONE authoritative source that could confirm or refute it. The primary source is the origin, not a summary of the origin:

Repo caution, verified in ai-agents-research-methodology: vendor docs alone are not a primary source for runtime behavior. This repo was burned twice by wrong-by-omission docs (the #2205 and #2290 retros in .agents/retrospective/). When the claim is about how an external tool behaves, the primary source is an empirical probe at a pinned version, not the doc. Route that through ai-agents-empirical-probe-toolkit.

Phase 2: Fetch and compare against the primary source

• Retrieve the primary source directly (host API, registry, paper, the party's own words). Prefer the API or raw source over a rendered page.
• Compare the artifact-bound claim against the source value: exact number, exact attribution, exact scope. A claim that rounds 987 up to "over 1000" is inflated, not confirmed.
• Follow any stat-of-a-stat chain to its origin. Confirm the origin actually says what the chain claims; drift compounds at every hop.
• For a repo-internal cross-claim, open the cited file and quote the load-bearing line verbatim. Paraphrase is the failure mode .claude/rules/canonical-source-mirror.md exists to stop (FM-9; PR #1887 paid 7 fix commits for paraphrased contracts).

Record what you fetched, when, and the version or commit, so the next reader can re-run the same comparison.

Phase 3: Weigh the source's stake

A source with a stake in the framing is not disqualified, but its self-favorable claims carry a lower prior until a neutral primary source confirms them:

• Vendor self-promotion and founder pitches: verify the headline number against a neutral registry or the raw artifact before it enters any repo doc.
• Advocacy quoting a competitor's unfavorable stat: find the competitor's own primary source; advocacy selects and rounds.
• A named authority invoked second-hand ("Gartner says"): the primary source is the named authority's own publication, not the party invoking it.

State the stake explicitly in the artifact when it is material, so a reader can weigh it too. This is the .claude/rules/voice.md "Ownership: See Something, Say Something" discipline applied to provenance: a flag with a hypothesis, not a silent pass.

Phase 4: Walk the gate, or file conservative

This is the load-bearing discipline. When a claim is not yet verified, you have exactly two honest moves. Naming a verification you did not run, then shipping anyway, is neither:

• Walk the gate. Run the verification now (the fetch is typically under two minutes), then state the claim at the confidence the primary source supports, with the source cited inline.
• File conservative. State only what you can defend without the unrun verification, drop the unverifiable number or attribution, and do not name the gate you skipped inside the artifact.

The anti-pattern this closes is "Reporting Without Acting": writing "would be N if X corroborates" or "likely Y pending source Z" into an artifact instead of running X or Z. That hands the reader a TODO dressed as a finding. The repo skill avoiding-manufactured-work names the same reward-bias root cause on the produce-a-deliverable side; this phase is its provenance-side twin. The gstack claim-verification-before-ingest pattern is the general form.

• Every external number, attribution, or structural claim in the artifact is either verified against its primary source with an inline citation, or removed.
• No conditional-caveat sentence ("would be X if", "likely Y pending", "could confirm with Z") names a verification path that was left unrun.

Anti-Patterns

Verification

Before a claim from an external source enters a durable artifact:

• The exact assertion and its single authoritative primary source are named.
• The claim was fetched from and compared against that primary source, with version or date recorded.
• Any stat-of-a-stat chain was followed to its origin, and any repo cross-claim is quoted verbatim with its path.
• Stake-holding sources are flagged, and self-favorable claims are confirmed against a neutral source.
• Every external number and attribution is cited inline or removed; no unrun verification is named as a caveat.

Provenance

Issue #3068 requested a stewardship and knowledge-transfer skill library for this repo. Fourteen domain knowledge-packs plus agent-harness-reference shipped in PR #2831 as the ai-agents-* skill set. This skill adds the one named domain that set did not cover: verifying external and third-party claims against authoritative primary sources before ingest. It is a documentation knowledge-pack (structure-only, no runner scripts), modeled on ai-agents-validation-and-qa.

Grounding sources, verified against the working tree:

Note on grounding: the "walk the gate, or file conservative" wording adapts a discipline the maintainer applies at the account level; the repo's own on-tree anchors for it are avoiding-manufactured-work and the voice.md Ownership rule cited above, not a named section in .claude/rules/voice.md.