# Fact Check Loop

> Iterative fact-checking loop for any document (PPT, Word, PDF, Markdown, etc.) against its original source. Spawns independent subagents in fresh contexts to eliminate confirmation bias. Each round checks all claims, the main agent fixes issues found, then a new subagent re-checks until zero issues remain. Use when: (1) user asks to fact-check a document, (2) user wants to verify a summary against its source, (3) user says "check for hallucinations", (4) after generating a report or presentation and wanting to validate accuracy. Requires an agent runtime that can spawn subagents in fresh contexts. Document text extraction and source retrieval use whatever tools the runtime provides (see Tooling below).

- Skill: `aws-samples/fact-check-loop` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add aws-samples/fact-check-loop`
- Raw SKILL.md: https://api.skillmd.com/api/skills/aws-samples/fact-check-loop/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: aws-samples (https://skillmd.com/u/aws-samples)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/aws-samples/fact-check-loop

---


# Fact-Check Loop

Iteratively fact-check a document against its original source using independent subagents.

The failure mode this skill exists to prevent: an agent that wrote a summary is a poor
judge of whether the summary is accurate. It already believes its own claims, so asking
it to re-read its work surfaces far fewer errors than it should. The fix is structural —
every check runs in a **fresh context** that has never seen the document being written,
only the document as it stands and the source it must match.

## Workflow

### 1. Gather inputs

Confirm two things (ask if not provided):

- **Document**: path to the file to check (`.pptx`, `.docx`, `.pdf`, `.md`, etc.)
- **Source**: the authoritative reference — URL(s), local file path(s), or a mix

### 2. Run the loop

Maintain a `fixes_log` list: `{round, page/section, issue, fix_applied}`.

For each round `N` (starting at 1):

**(a) Spawn a fact-check subagent.** Use the most capable model available to the runtime.
Each round MUST be a fresh subagent — never continue a conversation with a prior one.
Fresh context = independent judgment = no confirmation bias.

**(b) Build the subagent prompt** using the template in [references/subagent-prompt.md](references/subagent-prompt.md).
Include: document path, source location(s), cumulative `fixes_log`, round number, output format.

**(c) Parse the result.** The subagent returns either:
- **`ALL_PASS`** → exit the loop
- A list of issues → continue to (d)

**(d) Fix each issue** in the document. Append each fix to `fixes_log`.

**(e) Rebuild** if needed (e.g., re-run an HTML→PPTX build script so the fix lands in the artifact, not just the source).

**(f) Increment `N`**, repeat from (a). **Max 6 rounds** — if issues persist, report and stop.

### 3. Report

After the loop exits, output:

```
## Fact-Check Complete

| Round | Issues Found | Issues Fixed |
|-------|-------------|-------------|
| 1     | 7           | 7           |
| 2     | 3           | 3           |
| ...   | ...         | ...         |
| N     | 0 (PASS)    | —           |

Total rounds: N | Total issues fixed: X
Final file: <path>
```

## Key Rules

1. **Fresh agent every round.** A new subagent with a clean context is the entire point;
   reusing one collapses the check back into self-review.
2. **Cumulative fixes log.** Always pass the full history so each agent verifies prior
   fixes landed correctly and didn't introduce new errors.
3. **Error classification.** Instruct the subagent to classify each finding:
   - **Factual error** (wrong number, inverted labels, fabricated claim) — must fix
   - **Misleading omission** (missing context that changes meaning) — must fix
   - **Imprecise wording** (oversimplification, not wrong per se) — fix if easy
   - **Acceptable simplification** (inherent to summarization) — skip, don't count
4. **Source coverage.** The subagent must read the full source (chunk by chunk if needed)
   before reporting. A check against a partially-read source produces false passes on the
   sections it never opened.
5. **The subagent identifies, the main agent fixes.** Keep the roles split. A subagent that
   also edits starts justifying its own edits.

## Tooling

This skill is runtime-agnostic. It needs two capabilities; use whatever the runtime provides.

**Document text extraction:**

| Format | Approach |
|--------|----------|
| `.md` / `.txt` | Read the file directly |
| `.pdf` | A PDF reader that supports page ranges |
| `.pptx` / `.docx` | A converter such as [`markitdown`](https://github.com/microsoft/markitdown) (`pip install markitdown`), then `python3 -m markitdown <file>` |

**Source retrieval:**

| Source | Approach |
|--------|----------|
| Local file | Read directly; chunk large files with offset/limit |
| URL | Any web-fetch tool available (a built-in fetch tool, a crawler MCP, or `curl` piped through an HTML→Markdown converter). Chunk long pages and read sequentially until fully covered |

Record which extraction path was used in the final report — a check run against a bad text
extraction is worse than no check, because it looks like a pass.