Expert Overconfidence Pattern (Tetlock)

Philip Tetlock's landmark research on expert prediction reveals that domain experts are no better than chance at forecasting outcomes in their own field—and worse, they're more confident in their wrong predictions than non-experts. The mechanism: (1) deep domain knowledge creates a sense of understanding that generalizes to prediction, (2) the expert generates a compelling causal narrative for why X will happen, (3) the narrative's internal coherence is mistaken for predictive accuracy, (4) the expert's reputation and identity become tied to the prediction, (5) disconfirming evidence is dismissed or reinterpreted to preserve the prediction. Tetlock's 20-year study of 284 experts making 82,361 predictions found that the average expert barely outperformed a 'dart-throwing chimpanzee.' The critical finding: the experts who were most famous and most confident were the least accurate. Foxes (who know many things and maintain uncertainty) outperformed hedgehogs (who know one big thing and make confident predictions) on virtually every measure.

When to use it

When relying on expert predictions for important decisions. When an expert is very confident about a future outcome. When evaluating competing expert opinions. When making your own predictions in your domain of expertise.

How it can help

Calibrates appropriate confidence in expert predictions, including your own. Interventions: (1) weight expert reasoning but discount expert confidence—the strength of their argument is informative, the strength of their conviction is not. (2) Seek fox-type experts who express uncertainty and consider multiple factors over hedgehog-type experts who tell compelling single-cause stories. (3) Track expert predictions over time to build empirical accuracy records rather than relying on reputation. (4) Apply the 'superforecaster' practices from Tetlock's follow-up research: update frequently, use probabilities rather than binary predictions, keep score, and decompose complex predictions into estimatable components.