Sagan Standard

Sagan Standard

One-Liner

Extraordinary claims require extraordinary evidence—the more unusual the assertion, the stronger the proof needed.

Core Insight

The Sagan Standard, popularized by Carl Sagan ("extraordinary claims require extraordinary evidence"), establishes that the strength of evidence must be proportional to how unusual or improbable a claim is. It's not merely about requiring "more" evidence for surprising claims, but recognizing that claims contradicting massive existing evidence need proportionally stronger proof to rationally shift beliefs. The principle formalizes Bayesian reasoning: prior probability affects required evidence threshold.

Mental Model

Evidence Threshold Calibration:

Ordinary Claim (aligns with priors):
"I had coffee today"
Required Evidence: Minimal (your word suffices)

Moderately Unusual:
"I ran a 4-minute mile"
Required Evidence: Stopwatch, witnesses, video

Extraordinary (contradicts vast evidence):
"I can fly by flapping my arms"
Required Evidence: Rigorous controlled tests, replication, peer review

Prior Probability × Evidence Strength = Posterior Belief
(Very low prior) × (Modest evidence) = Still don't believe
(Very low prior) × (Overwhelming evidence) = Update belief

Bayesian Foundation: If P(claim) = 0.001%, evidence must shift likelihood ratio 100,000:1 to reach 50% confidence.

When to Use

• Evaluating novel scientific claims: Proportion skepticism to claim's implausibility
• Due diligence: Require stronger verification for too-good-to-be-true opportunities
• Debugging: Mundane explanations need less proof than exotic ones
• Hiring: Extraordinary resume claims demand extraordinary verification
• Product claims: Revolutionary benefits require revolutionary evidence
• Strategy validation: Paradigm-shifting plans need stronger evidence than incremental ones

Execution Steps

1. Calibrate Prior Probability

• Ask: "How much existing evidence contradicts this claim?"
• Quantify: Is this 1-in-10 unusual? 1-in-1000? 1-in-million?
• Consider base rates: How often do claims like this turn out true?
• Assess track record: Do similar extraordinary claims usually pan out?

2. Define "Extraordinary Evidence"

Not just "more evidence" but:

• Independent replication: Others reach same conclusion independently
• Mechanism explanation: Plausible causal story, not just correlation
• Controls for bias: Pre-registration, blinding, adversarial review
• Convergent evidence: Multiple independent lines pointing same direction
• Ruled out alternatives: Mundane explanations actively eliminated

3. Apply Proportional Skepticism

Evidence Quality Ladder (lowest to highest):
1. Anecdote/testimonial
2. Uncontrolled observation
3. Correlation study
4. Controlled experiment
5. Replicated controlled experiments
6. Meta-analysis of quality studies
7. Converging evidence across multiple methodologies

Match required level to claim extraordinariness.

4. Avoid Common Misapplications

Not about dismissing new ideas: Novel ≠ extraordinary

• Novel: "This new catalyst improves yield 10%"
• Extraordinary: "This catalyst violates thermodynamics"

Not about impossible burdens: Must be achievable evidence

• Bad: "Show me video of every atom behaving as quantum mechanics predicts"
• Good: "Show me repeatable experimental results inconsistent with alternatives"

Not about doubling standards: Apply consistently

• Apply same rigor to claims you like vs. dislike
• Don't accept weak evidence for preferred conclusions

5. Implement Evidence Thresholds in Decision-Making

Hiring Example:

• "5 years experience" → Check LinkedIn (ordinary)
• "Increased revenue 300%" → Verify with references, see documentation (extraordinary)
• "Invented algorithm 10x better than state-of-art" → Require proof-of-concept, technical deep-dive, independent review (very extraordinary)

Investment Example:

• "Growing 20% YoY" → Review financials (ordinary)
• "Discovered new market 10x current TAM" → Deep diligence, market evidence, customer validation (extraordinary)
• "Technology defies physical limits" → Probably fraud, require independent scientific validation (extremely extraordinary)

6. Update Beliefs Proportionally

• When extraordinary evidence appears: Update strongly
• When ordinary evidence appears for extraordinary claim: Remain skeptical
• Track claim's prior probability evolution as evidence accumulates
• Don't freeze beliefs; allow extraordinary evidence to shift extraordinary claims

Real-World Examples

Cold Fusion (1989)

• Claim: Room-temperature fusion in simple apparatus (contradicts massive physics evidence)
• Evidence Provided: Single experiment, press conference announcement
• Sagan Standard: Requires independent replication, mechanism explanation, peer review
• Outcome: Couldn't be replicated; mundane explanation (experimental error) prevailed

Helicobacter Pylori Causes Ulcers (1982)

• Claim: Bacteria cause ulcers (contradicted prevailing "stress and acid" theory)
• Evidence Provided: Initially weak, but Marshall self-experimented, then rigorous studies
• Sagan Standard: Unusual but not physics-violating; required good evidence, not impossible bar
• Outcome: Replicated studies convinced community; Nobel Prize 2005

Theranos (2003-2018)

• Claim: Comprehensive blood tests from finger prick (contradicted known technical limits)
• Evidence Provided: Charismatic founder, impressive board, media hype, ZERO peer review
• Sagan Standard Failure: Investors/partners didn't demand extraordinary evidence
• Outcome: Fraud; technology didn't work

Quantum Mechanics (1900-1930)

• Claim: Particles behave as waves, observer affects outcome (deeply counterintuitive)
• Evidence Provided: Repeatable experiments, mathematical framework, convergent evidence
• Sagan Standard: Extraordinary claim met with extraordinary evidence (replicated experiments worldwide)
• Outcome: Accepted as foundation of modern physics

Common Traps

Trap 1: Impossible Standards

• Demanding literally impossible evidence ("prove a negative")
• Sagan Standard requires proportional evidence, not infinite evidence

Trap 2: Dismissing All Novelty

• Conflating "new" with "extraordinary"
• New ideas contradicting little existing evidence need good evidence, not extraordinary

Trap 3: Motivated Reasoning

• Applying high bar to claims you dislike, low bar to claims you like
• Must calibrate to actual prior probability, not preference

Trap 4: Static Priors

• Refusing to update even when extraordinary evidence appears
• Prior probability should shift as evidence accumulates

Trap 5: Confusing Extraordinary with Unlikely

• "I won the lottery" is unlikely but not extraordinary (happens to someone)
• "I always win the lottery" is extraordinary (contradicts probability)

Relationship to Other Principles

Occam's Razor: Prefer simpler explanations

• Sagan Standard: Require stronger proof for complex explanations
• Complementary: Simple explanation needs less proof

Hitchens's Razor: "What asserted without evidence can be dismissed without evidence"

• More extreme: Demands some evidence for any claim
• Sagan Standard: Demands evidence proportional to extraordinariness

Hume's Principle: "No testimony is sufficient to establish a miracle unless..."

• Direct inspiration for Sagan Standard
• Focused on miracles; Sagan generalized to all extraordinary claims

Cross-Domain Applications

Product Management: "This feature will 10x retention" → Show A/B tests, not gut feel

Sales: "Our ROI is 50x" → Require customer case studies, not vendor white paper

Debugging: "Cosmic ray bit flip" → Eliminate prosaic bugs first (memory leak, race condition)

Medicine: "This supplement cures cancer" → Phase III RCTs or it's noise

Academia: "This overturns established theory" → Replication, peer review, alternative explanations ruled out

Further Reading

• Sagan, Carl (1979). Broca's Brain (original popularization)
• Hume, David (1748). "Of Miracles" in An Enquiry Concerning Human Understanding (philosophical foundation)
• Truzzi, Marcello (1987). "On Pseudo-Skepticism" (discusses misapplication)
• Pearl, Judea (2018). The Book of Why (Bayesian reasoning and evidence)
• Yudkowsky, Eliezer (2007). "How Much Evidence Does It Take?" (LessWrong formalization)

Source Domain: Military Strategy, Ancient Wisdom & Hidden Gems (07) Pattern Type: Epistemological Principle / Evidence Evaluation Practitioner Value: 9/10 | Clarity: 10/10 | ROI: 9/10 | Novelty: 7/10 | Cross-Domain: 10/10 Total Score: 45/50