Brier Score Loss

brier-score-loss

Metric brier_score_loss from scikit-learn (sklearn.metrics.brier_score_loss)

When to invoke this skill

The user has predictions + ground truth and asks to evaluate with brier_score_loss, or mentions sklearn.metrics.brier_score_loss directly, or wants the standard scikit-learn implementation.

Reference signature

from sklearn.metrics import brier_score_loss

# brier_score_loss(y_true, y_proba, *, sample_weight=None, pos_label=None, labels=None, scale_by_half='auto')

Library docstring

Compute the Brier score loss.

The smaller the Brier score loss, the better, hence the naming with "loss".
The Brier score measures the mean squared difference between the predicted
probability and the actual outcome. The Brier score is a strictly proper scoring
rule.

Read more in the :ref:`User Guide <brier_score_loss>`.

Parameters
----------
y_true : array-like of shape (n_samples,)
    True targets.

y_proba : array-like of shape (n_samples,) or (n_samples, n_classes)
    Predicted probabilities. If `y_proba.shape = (n_samples,)`
    the probabilities provided are assumed to be that of the
    positive class. If `y_proba.shape = (n_samples, n_classes)`
    the columns in `y_proba` are assumed to correspond to the
    labels in alphabetical order, as done by
    :class:`~sklearn.preprocessing.LabelBinarizer`.

sample_weight : array-like of shape (n_samples,), default=None
    Sample weights.

pos_label : int, float, bool or str, default=None
    Label of the positive class when `y_proba.shape = (n_samples,)`.
    If not provided, `pos_label` will be inferred in the
    following manner:

    * if `y_true` in {-1, 1} or {0, 1}, `pos_label` defaults to 1;
    * else if `y_true` contains string, an error will be raised and
      `pos_label` should be explicitly specified;
    * otherwise, `pos_label` defaults to the greater label,
      i.e. `np.unique(y_true)[-1]`.

labels : array-like of shape (n_classes,), default=None
    Class labels when `y_proba.shape = (n_samples, n_classes)`.
    If not provided, labels will be inferred from `y_true`.

    .. versionadded:: 1.7

scale_by_half : bool or "auto", default="auto"
    When True, scale the Brier score by 1/2 to lie in the [0, 1] range instead
    of the [0, 2] range. The default "auto" option implements the rescaling to
    [0, 1] only for binary classification (as customary) but keeps the
    original [0, 2] range for multiclass classification.

    .. versionadded:: 1.7

Returns
-------
score : float
    Brier score loss.

Notes
-----

For :math:`N` observations labeled from :math:`C` possible classes, the Brier
score is defined as:

.. math::
    \frac{1}{N}\sum_{i=1}^{N}\sum_{c=1}^{C}(y_{ic} - \hat{p}_{ic

Quick recipe

import sklearn.metrics as _m
score = _m.brier_score_loss(y_true, y_pred)

Don'ts

• Don't reimplement when the library version handles edge cases (NaN, ties, empty inputs) better than a hand-rolled formula.
• Always check the library version's argument order — sklearn is (y_true, y_pred) while torchmetrics is (preds, target).