Sampling Bias

The error of drawing conclusions from a sample that doesn't represent the population you're trying to understand—because the sampling process systematically over-includes some types and under-includes others. Surveying only your existing customers tells you about your customers, not about the market. Evaluating only successful companies tells you about success, not about what causes success (survivorship bias). Studying only people who visit hospitals tells you about hospital-visitors, not about disease prevalence. Every dataset is a sample, and every sample has a selection process—understanding that process is the first step in understanding what the data actually represents.

When to use it

When drawing conclusions from data that may not represent the full population; when customer feedback seems inconsistent with actual customer behavior; when research findings don't replicate in different populations; when any analysis depends on data that was collected through a non-random process.

How it can help

For any dataset or observation set: ask 'HOW was this sample generated, and who or what is systematically EXCLUDED?' Customer feedback over-samples extremes (very happy and very unhappy). Your professional network over-samples people similar to you. Media coverage over-samples the dramatic and novel. The correction: identify the selection mechanism, estimate what it excludes, and either adjust your conclusions or supplement with data from the excluded population. The most dangerous sampling bias is the invisible one—when you don't realize your data is a sample and treat it as the complete picture.