Variability in Judgment

Kahneman's finding from 'Noise' that professionals making the same judgment about the same case produce alarmingly different conclusions—and the variability is far greater than anyone expects. Different judges give different sentences for identical cases. Different underwriters set different premiums for identical risks. Different doctors give different diagnoses for identical symptoms. This variability (noise) is as large a source of error as systematic bias, but receives far less attention because it's invisible: you only see one judgment, never the distribution of possible judgments that COULD have been made.

When to use it

When important decisions are made by different people with inconsistent standards; when you suspect that outcomes depend more on who makes the decision than on the case itself; when designing evaluation, pricing, hiring, or diagnostic processes; when the goal is decision quality improvement through procedural discipline.

How it can help

Measure and reduce judgment variability for any important, repeatable decision. The noise audit: have multiple decision-makers evaluate the same cases independently and measure the spread. Most organizations are shocked by the results—the variability dwarfs what they expected. Reduction strategies: structured decision protocols (evaluation criteria applied in fixed order), independent assessment before group discussion, reference cases (benchmarks that anchor judgment), and decision rules that replace human judgment for routine cases. The key insight: reducing noise is as valuable as reducing bias but far easier to implement.