Survivorship Bias

Category: Probability & Belief

The logical error of concentrating on the people or things that made it past some selection process and overlooking those that did not, typically because of their lack of visibility.

How it works

We analyze the things that made it through a selection process and never see the things that didn't, so our sample is silently, catastrophically skewed. The winners are visible, loud, and available for study; the losers are gone, quiet, and invisible. Drawing lessons from the survivors alone is like studying lottery winners to learn how to win the lottery.

The trap is that the missing data doesn't announce itself. Nothing in the survivors says 'remember the hundreds who tried this and vanished.' So we confidently extract the 'secrets of success' from the few who succeeded, their habits, their bold bets, their college-dropout origin stories, without checking whether the failures had all the same traits. Often they did, which means those traits explain nothing.

This is what makes survivorship bias so seductive and so wrong: every example you can find supports the conclusion, because the counterexamples have been deleted from view. The pattern looks airtight precisely because the disconfirming cases were filtered out before you ever started looking.

Watch: the story behind it

In 1943, bombers came home covered in bullet holes, so the generals wanted armor where the holes were. Abraham Wald said to armor the places with no holes: every plane they measured had already made it home.

Where you'll see it

• 'Drop out and follow your passion!' draws on Gates, Jobs, and Zuckerberg, the handful who made it, while the millions of dropouts whose ventures quietly failed never get a TED talk, biasing the whole lesson.
• Mutual-fund advertising touts 'top performers over ten years,' but funds that did badly get quietly closed or merged away, so the surviving list *looks* like investing is easy, the losers were deleted from the chart.
• A new manager copies the 'work 90-hour weeks and never compromise' habits of one celebrated founder, not realizing countless founders with the identical habits burned out and went bankrupt, leaving only the lucky survivor to be studied.

Where it comes from

The most famous illustration comes from World War II and statistician Abraham Wald at the Statistical Research Group. The military examined bombers returning from missions and wanted to add armor where the planes showed the most bullet holes, the wings and fuselage. Wald's insight inverted the logic: those were the planes that survived being hit there. The places with no holes on returning aircraft, engines and cockpit, were precisely where a hit was fatal, so those planes never came back to be counted. Armor belonged where the survivors showed no damage. The story is the canonical demonstration that the missing data, not the visible data, holds the answer.

How to counter it

Always ask the haunting question: 'Where are the ones that didn't make it?' Before you copy a success, deliberately hunt for the failures, the dead startups, the closed funds, the dropouts who didn't become billionaires, and check whether they shared the very traits you're about to credit for success.

Look for the selection filter. Whenever you're handed a sample, ask what process determined who got into it. If only winners are visible (top performers, returning planes, published studies), you're looking at survivors, and the conclusion needs the invisible group to be valid.

Weigh base rates over highlight reels. 'How many people tried this, and what fraction succeeded?' is the question that converts an inspiring anecdote back into honest odds. The visible success story is one draw from a distribution whose failures you'll have to go dig up on purpose.

The tell

You're doing it when you're studying only the winners to learn what works and never asking what the losers had in common.

Related biases

• Confirmation Bias
• Availability Heuristic
• Gambler's Fallacy
• Base Rate Fallacy
• Optimism Bias
• Ostrich Effect

Featured in

• 7 cognitive biases that quietly wreck investors
• How to Actually Beat Your Cognitive Biases (Awareness Isn't Enough)
• Cognitive Bias vs. Logical Fallacy: What's the Difference?
• The Complete List of 58 Cognitive Biases (with real examples)

Common questions

What is Survivorship Bias?

Survivorship Bias is the logical error of concentrating on the people or things that made it past some selection process while overlooking those that did not, typically because the failures lack visibility. Because only the winners are visible and studied, conclusions drawn from them are silently and catastrophically skewed. It leads people to learn the wrong lessons from an incomplete, filtered sample.

Why does Survivorship Bias happen?

Survivorship Bias happens because we can only analyze the things that made it through a selection process and never see the things that didn't, so our sample is quietly distorted. The winners are visible, loud, and available for study, while the losers are gone, quiet, and invisible. Drawing lessons from the survivors alone is like studying lottery winners to learn how to win the lottery.

What is an example of Survivorship Bias?

A classic example of Survivorship Bias is the advice to 'drop out and follow your passion,' which points to Gates, Jobs, and Zuckerberg as proof. That lesson counts only the handful who made it while ignoring the millions of dropouts whose ventures quietly failed and never got a TED talk. The invisible failures skew the entire conclusion toward a false pattern.

How do you avoid Survivorship Bias?

You avoid Survivorship Bias by always asking the haunting question, 'Where are the ones that didn't make it?' Before copying any success, deliberately hunt for the failures, such as the dead startups, closed funds, and dropouts who never became billionaires. Then check whether those failures shared the very traits you were about to credit for the winners' success.

What is the difference between Survivorship Bias and Confirmation Bias?

Survivorship Bias is a sampling error where the failures are invisible, so you unknowingly study only the winners and never see the full picture. Confirmation Bias is a reasoning error where you selectively seek or favor evidence that supports what you already believe. In short, Survivorship Bias hides the losing data from you, while Confirmation Bias makes you ignore contrary data you could have seen.

References

1. Abraham Wald (1943). A Method of Estimating Plane Vulnerability Based on Damage of Survivors (reprinted 1980 as CRC 432, Center for Naval Analyses). Statistical Research Group, Columbia University / Center for Naval Analyses (CRC 432); DTIC ADA091073
2. Stephen J. Brown, William N. Goetzmann, Roger G. Ibbotson, Stephen A. Ross (1992). Survivorship Bias in Performance Studies. The Review of Financial Studies, 5(4), 553-580
3. Mark M. Carhart (1997). On Persistence in Mutual Fund Performance. The Journal of Finance, 52(1), 57-82