Encyclopedia · Free preview
Data Dredging
Searching through large datasets for statistically significant patterns without pre-specified hypotheses, then presenting the 'discoveries' as if they were predicted in advance. Also called p-hacking or data mining (in the pejorative sense). With enough variables, you'll always find spurious correlations—ice cream sales correlate with drowning deaths, but only because both correlate with summer. Data dredging produces Type I errors at industrial scale: the more you look, the more false patterns you find.
When to use it
When evaluating any data-driven claim, especially surprising ones; when reviewing analytics that tested many hypotheses; when 'insights' emerge from large datasets without pre-registered predictions; when someone presents only their successful findings without mentioning all the tests they ran.
How it can help
When someone presents a surprising data finding, ask: was this hypothesis specified before looking at the data, or after? Post-hoc findings in large datasets are almost certainly spurious unless replicated. In your own analysis, separate exploration (generating hypotheses from data) from confirmation (testing hypotheses with new data). The most common form in business: running dozens of A/B tests or segment analyses until something is 'significant,' then reporting only the positive result.
Keep exploring
Read the full page.
Create your free access to continue reading and explore the complete library.
Register free with ChatGPT →Already registered? Use the same button to sign in.
Sign-in shares your email with Michael Simmons to create your site access. No payment required. Newsletter signup is separate. How your data is used