n24q02m Fact Check
Verify a claim by actively searching for BOTH supporting and contradicting evidence. Counteracts LLM confirmation bias by enforcing adversarial search.
Steps
Decompose the claim into verifiable sub-claims:
- Separate factual assertions from opinions
- Identify the core falsifiable statement
- Note the domain: scientific, technical, historical, statistical, or current events
Search for SUPPORTING evidence using
search:search(action="search", query="[claim as stated]")- For scientific claims:
search(action="search", query="[claim] research evidence", search_type="academic") - Extract 2-3 strongest supporting sources with
extract
Search for CONTRADICTING evidence (mandatory — do NOT skip):
search(action="search", query="[claim] debunked OR wrong OR myth OR criticism")search(action="search", query="[opposite of claim] evidence")- For scientific claims:
search(action="search", query="[claim] replication failure OR meta-analysis", search_type="academic") - Extract 2-3 strongest contradicting sources with
extract
Assess source quality for each piece of evidence:
- Tier 1: Peer-reviewed journals, official statistics, primary sources
- Tier 2: Established news outlets, institutional reports, expert analysis
- Tier 3: Blogs, forums, social media, opinion pieces
- Tier 4: Anonymous sources, undated content, known biased outlets
- Discard Tier 4 unless no better sources exist
Produce verdict with structured output:
- Claim: [exact claim being checked]
- Verdict: Supported / Partially Supported / Disputed / Unsupported / Insufficient Evidence
- Confidence: High / Medium / Low (based on source tier and agreement)
- Supporting evidence: [bullet list with source tier tags]
- Contradicting evidence: [bullet list with source tier tags]
- Nuance: [important caveats, context-dependence, or scope limitations]
Decision Tree: Choosing Verdict
- All Tier 1-2 sources agree -> Supported (High confidence)
- Tier 1-2 support, some Tier 2-3 contradict -> Partially Supported (Medium)
- Tier 1-2 sources disagree with each other -> Disputed (note the split)
- Only Tier 3-4 sources support, Tier 1-2 contradict -> Unsupported (High)
- Fewer than 2 sources found either way -> Insufficient Evidence (Low)
Common Pitfalls
- Confirmation bias: Searching only for supporting evidence. The contradicting search in step 3 is MANDATORY.
- Recency bias: Old but well-cited research may outweigh recent blog posts.
- Authority bias: A claim from a famous person still needs evidence.
- Survivorship bias: "X worked for company Y" does not mean X works in general.
- Outdated facts: Check publication dates. A 2020 statistic may be wrong in 2026.
When to Use
- Verifying claims before including them in code comments or documentation
- Checking technical claims ("X is faster than Y", "Z is deprecated")
- Validating statistics or numerical claims from any source
- Fact-checking user assumptions before building on them