Replication Crisis

The replication crisis refers to concerns that many influential research claims have not held up consistently in new studies. Findings vary across fields and projects, and replication success can be defined in several ways. Recomputing a result from the same data is distinct from testing the claim with new data.

Replication concerns whether a new study addressing the same claim obtains results consistent with it under relevant conditions. Reproducibility sometimes refers more narrowly to obtaining the same computational results from the same data and methods; terminology varies. Neither a published paper nor a successful reanalysis alone establishes that an effect will recur in new observations.

Large replication projects have raised concerns about effect sizes, selective reporting, study design, and incentives in particular research literatures. Their results depend on which studies were selected and how success was assessed. Avoid turning a project-specific percentage into a failure rate for an entire discipline. Evaluate the body of evidence, uncertainty, and possible boundary conditions while preserving unsuccessful and null results.

When to use it

When a decision rests on a striking study, when interpreting a replication report, or when designing a follow-up intended to test an earlier claim.

How it can help

Evaluate methods, effect sizes, uncertainty, reporting practices, and independent evidence before treating a claim as established or transferable.