Publication Bias

The systematic distortion of the scientific literature caused by the preferential publication of positive, significant, or novel results—and the suppression of null, negative, or replicating results. Journals publish exciting findings ('this treatment works!') and reject boring ones ('this treatment doesn't work')—creating a literature that systematically overstates effect sizes and understates null results. A drug tested in 20 studies might show positive results in 3 (published) and null results in 17 (unpublished)—making the published literature suggest the drug works when it doesn't. The file drawer problem: the unpublished null results sit in researchers' file drawers, invisible to anyone reading the literature.

When to use it

When evaluating scientific evidence for decision-making; when research findings seem too consistently positive (publication bias may be hiding null results); when designing internal experimentation programs; when building evidence-based practices that need to account for the full evidence base.

How it can help

When evaluating evidence: consider not just what was published but what WASN'T. Meta-analyses try to account for publication bias statistically, but the gold standard is pre-registration (declaring your study before running it, so null results can't be hidden). For business decisions based on research: weight replication studies as much as original findings. For organizational learning: create systems where null results (this didn't work) are as valued as positive results (this worked)—because knowing what DOESN'T work is as valuable as knowing what does.