Binaryroc

binaryroc

Metric BinaryROC from torchmetrics (torchmetrics.classification.BinaryROC)

When to invoke this skill

The user has predictions + ground truth and asks to evaluate with BinaryROC, or mentions torchmetrics.classification.BinaryROC directly, or wants the standard torchmetrics implementation.

Reference signature

from torchmetrics.classification import BinaryROC

# BinaryROC(thresholds: Union[int, list[float], torch.Tensor, NoneType] = None, ignore_index: Optional[int] = None, validate_args: bool = True, normalization: Optional[Literal['sigmoid', 'softmax']] = 'sigmoid', **kwargs: Any) -> None

Library docstring

Compute the Receiver Operating Characteristic (ROC) for binary tasks.

The curve consist of multiple pairs of true positive rate (TPR) and false positive rate (FPR) values evaluated at
different thresholds, such that the tradeoff between the two values can be seen.

As input to ``forward`` and ``update`` the metric accepts the following input:

- ``preds`` (:class:`~torch.Tensor`): A float tensor of shape ``(N, ...)``. Preds should be a tensor containing
  probabilities or logits for each observation. If preds has values outside [0,1] range we consider the input
  to be logits and will auto apply sigmoid per element.
- ``target`` (:class:`~torch.Tensor`): An int tensor of shape ``(N, ...)``. Target should be a tensor containing
  ground truth labels, and therefore only contain {0,1} values (except if `ignore_index` is specified). The value
  1 always encodes the positive class.

.. tip::
   Additional dimension ``...`` will be flattened into the batch dimension.

As output to ``forward`` and ``compute`` the metric returns a tuple of 3 tensors containing:

- ``fpr`` (:class:`~torch.Tensor`): A 1d tensor of size ``(n_thresholds+1, )`` with false positive rate values
- ``tpr`` (:class:`~torch.Tensor`): A 1d tensor of size ``(n_thresholds+1, )`` with true positive rate values
- ``thresholds`` (:class:`~torch.Tensor`): A 1d tensor of size ``(n_thresholds, )`` with decreasing threshold
  values

.. note::
   The implementation both supports calculating the metric in a non-binned but accurate version and a
   binned version that is less accurate but more memory efficient. Setting the `thresholds` argument to `None` will
   activate the non-binned  version that uses memory of size :math:`\mathcal{O}(n_{samples})` whereas setting the
   `thresholds` argument to either an integer, list or a 1d tensor will use a binned version that uses memory of
   size :math:`\mathcal{O}(n_{thresholds})` (constant memory).

.. attention::
   The outputted thresholds will be in reversed order to ensure that they correspond to both fpr and
   tpr which are sorted in reversed order during their calculation, such that they are monotome increasing.

Args:
    thresholds:
        Can be one of

Quick recipe

import torchmetrics.classification as _m
score = _m.BinaryROC(y_true, y_pred)

Don'ts

• Don't reimplement when the library version handles edge cases (NaN, ties, empty inputs) better than a hand-rolled formula.
• Always check the library version's argument order — sklearn is (y_true, y_pred) while torchmetrics is (preds, target).