lev-os Bayesian Updating
Overview
Probabilistic belief revision framework: Systematically update your confidence in hypotheses as new evidence arrives, weighting both prior beliefs and new information by their respective reliability.
Core insight: Neither trust new data blindly nor cling to old beliefs stubbornly—optimal belief revision lies between these extremes, proportional to precision of evidence.
Source: Thomas Bayes (18th century), formalized by LessWrong rationality community for practical cognition
The Framework
Bayes' Theorem (Simplified)
Updated Belief = (Prior Belief × Likelihood of Evidence) / Total Probability of Evidence
In plain language:
- Prior: What you believed before seeing new evidence
- Likelihood: How expected this evidence is if your hypothesis is true
- Posterior: Your updated belief after incorporating evidence
- Precision: How certain/reliable the information source is
The Cognitive Process
Step 1 - Establish Prior: What's your current confidence level in hypothesis H?
- Example: "30% chance it will rain today" (based on historical weather patterns)
Step 2 - Observe Evidence: New information arrives
- Example: Weather radar shows large storm system approaching
Step 3 - Evaluate Likelihood: How expected is this evidence under different hypotheses?
- If raining: Storm on radar is very expected (high likelihood)
- If not raining: Storm on radar is less expected (lower likelihood)
Step 4 - Calculate Posterior: Update belief proportionally
- Example: Updated to "70% chance of rain" after radar evidence
Step 5 - Iterate: Your posterior becomes the new prior for the next evidence cycle
When to Use
Explicit Bayesian updating when:
- Receiving new information that challenges existing beliefs
- Multiple information sources with different reliability levels
- Need to quantify uncertainty numerically
- Making predictions that can be calibrated over time
- Evaluating competing hypotheses
Implicit Bayesian thinking for:
- Any belief revision situation
- Assessing credibility of sources
- Medical diagnosis (symptoms as evidence)
- Debugging code (test results as evidence)
- Investment decisions (market signals as evidence)
Implementation Steps
1. Quantify Your Prior
Make your existing belief explicit:
- "I'm 60% confident that feature X will increase engagement"
- "There's a 20% chance this bug is in the API layer"
- "I believe with 80% confidence that candidate A is better fit"
Avoid vague language like "probably" or "might"—use numbers.
2. Identify the Evidence
What new information are you receiving?
- User testing results
- Stack trace from error logs
- Candidate's take-home project quality
- Competitor's product launch
Be specific about what you're observing, not your interpretation yet.
3. Assess Likelihood Ratio
Ask: "How much more expected is this evidence if my hypothesis is true versus false?"
Strong evidence: Very expected under hypothesis, very unexpected otherwise
- Ratio might be 10:1 or 100:1
- Example: If bug is in API, seeing API error logs is 50x more likely than if bug is frontend
Weak evidence: Only somewhat more expected under hypothesis
- Ratio might be 2:1 or 3:1
- Example: User engagement up 5% could happen with or without feature
Misleading evidence: More expected if hypothesis is false
- Ratio less than 1:1
- Should update belief downward
4. Update Proportionally
Rough heuristic (for intuitive updating without calculation):
- Strong evidence (10:1 ratio): Update belief significantly (±20-30%)
- Moderate evidence (3:1 ratio): Update moderately (±10-15%)
- Weak evidence (1.5:1 ratio): Update slightly (±5%)
Direction: Move toward the hypothesis the evidence supports.
Magnitude: Stronger evidence = larger update, but never jump to 100% certainty from single data point.
5. Track Your Calibration
Periodically check: Are your 70% predictions actually coming true 70% of the time?
- If yes: Well-calibrated
- If predictions come true >70%: You're underconfident (update priors upward faster)
- If predictions come true <70%: You're overconfident (update more conservatively)
This feedback loop improves your Bayesian instincts over time.
6. Weight Source Precision
Not all evidence is equally reliable:
High precision (trust more, update more):
- Randomized controlled experiments
- Large sample sizes
- Direct observation
- Domain expert analysis
Low precision (trust less, update less):
- Anecdotal reports
- Small samples
- Indirect indicators
- Biased sources
Adjust your update magnitude by source reliability.
Common Pitfalls
Ignoring base rates: Jumping to conclusions from evidence without considering prior probability. (Example: Rare disease with 99% accurate test can still be unlikely even with positive result if disease is 0.1% prevalent.)
Confirmation bias: Selectively updating on evidence that supports existing beliefs, dismissing contradictory evidence. True Bayesian updating is symmetric—update in both directions.
Overconfidence: Updating too much from single data points, reaching near-certainty prematurely. Keep some probability mass on alternative hypotheses.
Binary thinking: Treating beliefs as true/false rather than probabilistic confidence levels. Everything is a percentage.
Neglecting alternative hypotheses: Updating P(H) without considering P(not-H) and other competing explanations for the evidence.
Anchoring on priors: Refusing to update sufficiently when strong evidence arrives. Your prior shouldn't be sacred—it's just your starting point.
Conservation of expected evidence: If you think evidence might arrive, you should already have an opinion on what different results would mean. Don't wait for the data to decide how to interpret it.
Real-World Applications
Medical diagnosis: Doctor starts with base rate of disease prevalence (prior), updates based on symptoms (evidence), orders tests (more evidence), revises diagnosis (posterior).
Software debugging: Initial hypothesis about bug location (prior), run test revealing error location (evidence), update belief about root cause (posterior), test fix (more evidence).
Hiring decisions: Initial assessment from resume (prior), performance on technical interview (evidence), reference checks (more evidence), final confidence in candidate fit (posterior).
Investment analysis: Market belief about company value (prior), earnings report (evidence), updated stock price reflecting collective Bayesian updating (posterior).
Product development: Hypothesis about user need (prior), user research findings (evidence), A/B test results (more evidence), conviction to ship feature (posterior).
Power Moves
Pre-commit to belief changes: Before seeing evidence, state explicitly: "If I see X, I'll update my belief from Y% to Z%." This prevents post-hoc rationalization.
Calibration training: Make many probabilistic predictions, track accuracy, adjust to hit calibration targets. This builds Bayesian intuition.
Likelihood ratio shortcut: Instead of full Bayes calculation, ask "How many times more likely is this evidence under hypothesis A vs. B?" Adjust beliefs proportionally.
Update incrementally: Don't wait for "decisive" evidence. Small updates from weak evidence compound over time into strong beliefs when consistent.
Separate observation from interpretation: Clearly distinguish what you observed (evidence) from what it means (likelihood). Mix these up and you double-count the same information.
Quantify uncertainty explicitly: Force yourself to use numbers. "Probably" is too vague—is it 60% or 90%? Numbers enable proper updating.
Related Frameworks
- Epistemic Rationality: Bayesian updating is the mathematical foundation for systematic belief accuracy
- Expected Value Calculation: Uses Bayesian probabilities to weight outcomes
- Scientific Method: Hypothesis testing is formalized Bayesian updating
- Prediction Markets: Aggregated Bayesian updating across many individuals
- Kalman Filtering: Continuous Bayesian updating for dynamic systems (robotics, control systems)
Technical Note
Full Bayesian calculation:
P(H|E) = P(E|H) × P(H) / P(E)
Where:
P(H|E) = Posterior (updated belief in hypothesis given evidence)
P(E|H) = Likelihood (probability of evidence if hypothesis true)
P(H) = Prior (initial belief in hypothesis)
P(E) = Total probability of evidence
For practical use, focus on likelihood ratios:
P(H|E) / P(~H|E) = [P(E|H) / P(E|~H)] × [P(H) / P(~H)]
Posterior Odds = Likelihood Ratio × Prior Odds
This "odds form" is often more intuitive for incremental updating.