Brier Score

The binary Brier score is the mean squared difference between forecast probabilities and zero-or-one outcomes. Lower scores are better, and truthful probabilities minimize expected loss under the scoring model.

For a binary event, the Brier score averages (p − y)², where p is the forecast probability and y is zero or one. Lower is better. Its expected value is minimized by reporting the true event probability under the model, which makes it a proper scoring rule. A realized score on one event does not establish forecast quality.

The score reflects more than calibration. A constant base-rate forecast can be calibrated yet fail to distinguish easier from harder cases. Compare forecasts on the same resolved events and state the scoring convention: the binary form ranges from zero to one, while a sum over all categories uses a different range.

When to use it

When analyzing data; when building predictive models; when evaluating statistical claims; when quantifying uncertainty.

How it can help

Record forecasts before resolution, define events clearly, calculate a consistent score, and compare forecasts on the same cases with an appropriate baseline.