Regression Fallacy

Category: Probability & Belief

The regression fallacy is when you credit (or blame) some intervention for a change that was going to happen anyway because of regression toward the mean. Extreme results tend to be followed by more average ones for purely statistical reasons, but you invent a cause: the treatment, the punishment, the pep talk, the new coach.

How it works

Most real-world outcomes are a mix of a stable component (skill, true value) and a random component (luck, noise, measurement error). When you observe an extreme value, it is extreme partly because the random part happened to line up in one direction, and that alignment almost never repeats. So the next measurement drifts back toward the true average. This is regression toward the mean, and it is a mathematical near-certainty, not a force. The fallacy is watching that drift and attaching a cause to it: you selected a group precisely because it was extreme, you did something, the group regressed, and you conclude the something worked.

Where you'll see it

• Kahneman and Tversky's flight instructors (1973): Israeli Air Force instructors were convinced that harsh criticism improved the next landing and praise ruined it. Kahneman realized both were pure regression. He later called it one of the most important insights of his career, because it means the world punishes you for rewarding people and rewards you for punishing them, entirely by accident.
• Horace Secrist's 'The Triumph of Mediocrity in Business' (1933): after ten years and 200-plus charts, Secrist 'proved' that top firms decline toward mediocrity and weak firms rise toward it, blaming competition. Harold Hotelling's review pointed out he had proved nothing but regression to the mean. Secrist had, in Hotelling's framing, proved the multiplication table by arranging elephants in rows.
• The Sports Illustrated cover 'jinx': athletes on the cover after a career-best streak tend to slump afterward. Fans blame the curse of the cover. You get the cover for an extreme peak, and extreme peaks regress. Same logic behind the rookie-of-the-year sophomore slump.
• Medical and policy evaluations: a clinic recruits patients with the highest blood pressure, gives a treatment, and sees pressure drop at follow-up. Some of that drop is regression, because you selected the most extreme readings. Without a control group you cannot tell how much of the improvement was the drug and how much was the mean pulling back.

Where it comes from

Francis Galton discovered the underlying phenomenon in 1886, measuring the heights of 928 adult children against their parents and finding that tall parents had children closer to average, and short parents likewise. He called it 'regression towards mediocrity in hereditary stature,' which is where the word 'regression' in statistics comes from. The leap from statistical phenomenon to named reasoning error came later. Harold Hotelling's 1933 demolition of Horace Secrist's business book is the canonical worked example, and Daniel Kahneman and Amos Tversky's 1973 paper 'On the Psychology of Prediction' established it as a systematic cognitive bias, showing people ignore regression when making intuitive predictions.

How to counter it

Imagine the no-treatment world. Before you credit the intervention, ask what would have happened if you had done nothing. If you only intervened because a value hit an extreme (worst quarter, sickest patient, hottest streak), the honest baseline is movement toward average, not staying put.

Demand a control group. The only way to separate your effect from regression is comparing against a group that was equally extreme but got no treatment. Any before-and-after study that selected on an extreme and skipped a control is contaminated by regression, full stop.

Watch for selection on the extreme. The fallacy needs one specific setup: you pick cases because they are unusually high or low, then measure again. Whenever you notice you started from a peak or a trough (top salespeople, failing schools, a record month), flag that any subsequent 'improvement' or 'decline' is suspect.

Separate skill from noise. Estimate how much of the outcome is stable versus luck. The noisier the metric, the harder it regresses, so a single quarterly number or one test score will regress far more than a ten-year average. Trust aggregates over dramatic single data points.

The tell

You catch yourself saying 'ever since we did X, things went back to normal' about a metric you only started tracking because it was at an extreme. The intervention always looks like it works, and it always kicks in right after the worst (or best) moment.

Related biases

• Confirmation Bias
• Availability Heuristic
• Survivorship Bias
• Gambler's Fallacy
• Base Rate Fallacy
• Optimism Bias

Common questions

What is Regression Fallacy?

The Regression Fallacy is the mistake of crediting or blaming an intervention for a change that was going to happen anyway because of regression toward the mean. Extreme results tend to be followed by more average ones for purely statistical reasons, so the intervention gets credit it did not earn. It fools people into seeing cause and effect where only chance and averaging are at work.

Why does Regression Fallacy happen?

The Regression Fallacy happens because most real-world outcomes mix a stable component, such as skill or true value, with a random component, such as luck, noise, or measurement error. An extreme observation is extreme partly because the random part happened to line up in one direction, and that alignment almost never repeats. So the next measurement naturally drifts back toward the true average, and people wrongly attribute that drift to whatever they did in between.

What is an example of Regression Fallacy?

A classic example of the Regression Fallacy comes from Kahneman and Tversky's 1973 study of Israeli Air Force flight instructors. The instructors were convinced that harsh criticism improved a pilot's next landing while praise ruined it, but Kahneman realized both effects were pure regression toward the mean. A very bad landing was likely to be followed by a better one, and a very good landing by a worse one, regardless of any feedback.

How do you avoid Regression Fallacy?

To avoid the Regression Fallacy, imagine the no-treatment world and ask what would have happened if you had done nothing before crediting any intervention. If you only intervened because a value hit an extreme, such as the worst quarter, the sickest patient, or the hottest streak, then the honest baseline is movement toward average rather than stability. Comparing against that expected drift, ideally with a control group, keeps you from mistaking regression for results.

What is the difference between Regression Fallacy and Gambler's Fallacy?

The Regression Fallacy is misattributing a natural drift back toward average to an intervention, treating a statistical rebound as proof that something worked. The Gambler's Fallacy is the opposite error of expecting past random outcomes to influence future ones, such as believing a coin is due for tails after several heads. Regression Fallacy over-credits a cause for a real return to average, while Gambler's Fallacy invents a false pattern in independent events.

References

1. Francis Galton (1886). Regression towards Mediocrity in Hereditary Stature. Journal of the Anthropological Institute of Great Britain and Ireland, 15, 246-263
2. Harold Hotelling (1933). Review of The Triumph of Mediocrity in Business by Horace Secrist. Journal of the American Statistical Association, 28(184), 463-465
3. Horace Secrist (1933). The Triumph of Mediocrity in Business. Bureau of Business Research, Northwestern University
4. Daniel Kahneman & Amos Tversky (1973). On the Psychology of Prediction. Psychological Review, 80(4), 237-251
5. Daniel Kahneman (2011). Thinking, Fast and Slow (Ch. 17, Regression to the Mean). Farrar, Straus and Giroux