Continuousrankedprobabilityscore

continuousrankedprobabilityscore

Metric ContinuousRankedProbabilityScore from torchmetrics (torchmetrics.regression.ContinuousRankedProbabilityScore)

When to invoke this skill

The user has predictions + ground truth and asks to evaluate with ContinuousRankedProbabilityScore, or mentions torchmetrics.regression.ContinuousRankedProbabilityScore directly, or wants the standard torchmetrics implementation.

Reference signature

from torchmetrics.regression import ContinuousRankedProbabilityScore

# ContinuousRankedProbabilityScore(**kwargs: Any) -> None

Library docstring

Computes continuous ranked probability score.

.. math::
    CRPS(F, y) = \int_{-\infty}^{\infty} (F(x) - 1_{x \geq y})^2 dx

where :math:`F` is the predicted cumulative distribution function and :math:`y` is the true target. The metric is
usually used to evaluate probabilistic regression models, such as forecasting models. A lower CRPS indicates a
better forecast, meaning that forecasted probabilities are closer to the true observed values. CRPS can also be
seen as a generalization of the brier score for non binary classification problems.

As input to ``forward`` and ``update`` the metric accepts the following input:

- ``preds`` (:class:`~torch.Tensor`): Predicted float tensor with shape ``(N,d)``
- ``target`` (:class:`~torch.Tensor`): Ground truth float tensor with shape ``(N,d)``

As output of ``forward`` and ``compute`` the metric returns the following output:

- ``cosine_similarity`` (:class:`~torch.Tensor`): A float tensor with the cosine similarity

Args:
    reduction: how to reduce over the batch dimension using 'sum', 'mean' or 'none' (taking the individual scores)
    kwargs: Additional keyword arguments, see :ref:`Metric kwargs` for more info.

Example:
    >>> from torch import randn
    >>> from torchmetrics.regression import ContinuousRankedProbabilityScore
    >>> preds = randn(10, 5)
    >>> target = randn(10)
    >>> crps = ContinuousRankedProbabilityScore()
    >>> crps(preds, target)
    tensor(0.7731)

Quick recipe

import torchmetrics.regression as _m
score = _m.ContinuousRankedProbabilityScore(y_true, y_pred)

Don'ts

• Don't reimplement when the library version handles edge cases (NaN, ties, empty inputs) better than a hand-rolled formula.
• Always check the library version's argument order — sklearn is (y_true, y_pred) while torchmetrics is (preds, target).