Correct for Survivorship Bias
Datasets built from what still exists exclude what failed, and that exclusion is not random. Every conclusion about performance, quality, or retention has to survive adding back the dead and the departed.
Procedure
- Ask what had to happen for a row to be present: did the entity survive, keep paying, stay listed, or answer the survey?
- Find the survival filter in the pipeline — a JOIN to a current table, a
status='active' predicate, or a partner API that returns only live accounts.
- Reconstruct the full universe. Keep churned users with their churn date; for delisted funds or dead tokens, source historical listings externally (exchange archives, dated snapshots).
- Add the dropouts and recompute. If a headline reverses, the original was a survivorship artefact:
-- include accounts closed before today, with their closing date
SELECT account_id, opened_at, closed_at, realized_pnl
FROM accounts -- no WHERE status='active'
WHERE opened_at <= '2026-01-01';
- For survey or review data, compare the respondent profile to the eligible population; differences in the tail are the bias.
- For backtests, include the assets that went to zero and the funds that shut — a universe of today's largest names is a look-ahead filter.
- State coverage explicitly: "n=1,204 of an estimated 1,880 eligible (64% observed)".
Pitfalls
- A vendor API returning only currently-listed symbols silently drops every delisting.
- A "current customers" cohort for churn analysis cannot see people who already left — retention is overstated.
- Fund databases that list only survivors overstate average returns by the failed-fund drag.
- Snapshot tables overwritten nightly lose the rows that later disappeared; keep dated snapshots.
- Adding back failures narrows the gap but rarely closes it, because failure records are thinner.
Verification
psql "$DSN" -c "SELECT (SELECT count(*) FROM accounts) observed, (SELECT count(*) FROM account_openings) eligible;"
# coverage number must be reported alongside the corrected estimate
Report: "Rebuilt the sample with 212 closed accounts added (observed 1,204 of 1,880 eligible, 64%); median return fell from 7.1% to 4.3% — the earlier figure was survivorship."
1---2name: correct-for-survivorship-bias3description: Use when analysis only includes entities that still exist. Rebuilds the sample with failures and dropouts before drawing conclusions.4metadata:5 source: https://skillmd.com/skills/jiayaoqijia/correct-for-survivorship-bias6---78# Correct for Survivorship Bias910Datasets built from what still exists exclude what failed, and that exclusion is not random. Every conclusion about performance, quality, or retention has to survive adding back the dead and the departed.1112## Procedure13141. Ask what had to happen for a row to be present: did the entity survive, keep paying, stay listed, or answer the survey?152. Find the survival filter in the pipeline — a JOIN to a current table, a `status='active'` predicate, or a partner API that returns only live accounts.163. Reconstruct the full universe. Keep churned users with their churn date; for delisted funds or dead tokens, source historical listings externally (exchange archives, dated snapshots).174. Add the dropouts and recompute. If a headline reverses, the original was a survivorship artefact:18 ```sql19 -- include accounts closed before today, with their closing date20 SELECT account_id, opened_at, closed_at, realized_pnl21 FROM accounts -- no WHERE status='active'22 WHERE opened_at <= '2026-01-01';23 ```245. For survey or review data, compare the respondent profile to the eligible population; differences in the tail are the bias.256. For backtests, include the assets that went to zero and the funds that shut — a universe of today's largest names is a look-ahead filter.267. State coverage explicitly: "n=1,204 of an estimated 1,880 eligible (64% observed)".2728## Pitfalls2930- A vendor API returning only currently-listed symbols silently drops every delisting.31- A "current customers" cohort for churn analysis cannot see people who already left — retention is overstated.32- Fund databases that list only survivors overstate average returns by the failed-fund drag.33- Snapshot tables overwritten nightly lose the rows that later disappeared; keep dated snapshots.34- Adding back failures narrows the gap but rarely closes it, because failure records are thinner.3536## Verification3738 psql "$DSN" -c "SELECT (SELECT count(*) FROM accounts) observed, (SELECT count(*) FROM account_openings) eligible;"39 # coverage number must be reported alongside the corrected estimate4041Report: "Rebuilt the sample with 212 closed accounts added (observed 1,204 of 1,880 eligible, 64%); median return fell from 7.1% to 4.3% — the earlier figure was survivorship."