Sample and coverage

Specify exactly which cases, people, and time windows your evidence covers, including the failures and the missing data.

Reading through customer feedback, your product looks loved. Dozens of warm emails, a testimonial wall, four-star-plus reviews. Then a colleague asks a deflating question: who writes in? The customers who churned silently last quarter never sent an email. Neither did the ones who signed up, poked around once, and vanished. Your glowing sample is the survivors talking.

Sample and coverage is the move of specifying exactly which cases, people, and time windows your evidence covers, and naming what's missing, especially the failures and the silent. The cue is drawing a broad conclusion from whatever data happened to arrive on its own. The operation: write down who is in the sample (customers who chose to write), who is absent (churned users, one-visit signups), what window it spans, and whether the missing cases would plausibly tell a different story. Then, if the decision matters, go sample the absent group directly, such as emailing 20 churned customers. What changes is that "customers love us" becomes "vocal current customers love us, and we know nothing about the leavers," which is a different planning input entirely. Skip the audit when your data genuinely covers the whole population, like a report on every transaction in the system.