Data Science & ML
Intermediate

Brier Score (Binary Outcomes)

Mean squared error of probability forecasts for a binary event. With probabilities between 0 and 1 and outcomes coded 0 or 1, this convention ranges from 0 (perfect) to 1 (worst). Lower is better.

Formula

BS=1N∑i=1N(pi−yi)2BS=\frac{1}{N}\sum_{i=1}^{N}(p_i-y_i)^2

Variables

BSBinary Brier score
NNumber of forecasts (positive integer)
p_iForecast probability of the event, between 0 and 1
y_iObserved outcome: 1 if the event happens, 0 otherwise

Example

Probabilities 0.8, 0.3, 0.6 and outcomes 1, 0, 1 give BS = (0.04 + 0.09 + 0.16)/3 ≈ 0.0967.

How it works

A forecast of an 80% chance of rain should be judged as a probability, not converted into a yes/no guess before scoring. The binary Brier score compares each forecast probability with the eventual outcome and averages the squared gaps. Large confident errors are penalised more heavily. Across repeated forecasts, this strictly proper score is minimised in expectation by reporting the true event probabilities.

When to use it

Evaluate binary probability forecasts for weather, risk or classification on held-out outcomes. Compare models on the same events and against a suitable baseline; the score is not a calibration-only measure.

Worked example, step by step

Step 1: Forecast probabilities are 0.8, 0.3 and 0.6. Observed outcomes are 1, 0 and 1.

Step 2: First squared error: (0.8 − 1)² = 0.04.

Step 3: Second squared error: (0.3 − 0)² = 0.09.

Step 4: Third squared error: (0.6 − 1)² = 0.16.

Step 5: Average: BS = (0.04 + 0.09 + 0.16)/3 = 0.29/3 ≈ 0.0967.

Interpretation: lower is better. A score of zero means every forecast matched its outcome perfectly; this small example alone does not establish a generally reliable model.

Common mistakes to avoid

  • Entering percentages such as 80 instead of probabilities such as 0.8.
  • Thresholding probabilities into class labels before scoring.
  • Calling the score prediction accuracy or calibration alone.
  • Mixing the binary 0–1 convention with the two-class summed convention, which is twice as large.

Did You Know?

The Brier score rewards honest probability forecasts in expectation. It reflects both calibration and resolution, not calibration alone.

Share this formula

Educational examples use simplified assumptions. Check units and applicability before using a result; health, financial and safety decisions need appropriate professional guidance. About FormulaWon · Report a correction