Retrievalauroc

retrievalauroc

Metric RetrievalAUROC from torchmetrics (torchmetrics.retrieval.RetrievalAUROC)

When to invoke this skill

The user has predictions + ground truth and asks to evaluate with RetrievalAUROC, or mentions torchmetrics.retrieval.RetrievalAUROC directly, or wants the standard torchmetrics implementation.

Reference signature

from torchmetrics.retrieval import RetrievalAUROC

# RetrievalAUROC(empty_target_action: Literal['error', 'skip', 'neg', 'pos'] = 'neg', ignore_index: Optional[int] = None, top_k: Optional[int] = None, max_fpr: Optional[float] = None, aggregation: Union[Literal['mean', 'median', 'min', 'max'], Callable] = 'mean', **kwargs: Any) -> None

Library docstring

Compute area under the receiver operating characteristic curve (AUROC) for information retrieval.

Works with binary target data. Accepts float predictions from a model output.

As input to ``forward`` and ``update`` the metric accepts the following input:

- ``preds`` (:class:`~torch.Tensor`): A float tensor of shape ``(N, ...)``
- ``target`` (:class:`~torch.Tensor`): A long or bool tensor of shape ``(N, ...)``
- ``indexes`` (:class:`~torch.Tensor`): A long tensor of shape ``(N, ...)`` which indicate to which query a
  prediction belongs

As output to ``forward`` and ``compute`` the metric returns the following output:

- ``auroc@k`` (:class:`~torch.Tensor`): A single-value tensor with the auroc value
  of the predictions ``preds`` w.r.t. the labels ``target``.

All ``indexes``, ``preds`` and ``target`` must have the same dimension and will be flatten at the beginning,
so that for example, a tensor of shape ``(N, M)`` is treated as ``(N * M, )``. Predictions will be first grouped by
``indexes`` and then will be computed as the mean of the metric over each query.

Args:
    empty_target_action:
        Specify what to do with queries that do not have at least a positive ``target``. Choose from:

        - ``'neg'``: those queries count as ``0.0`` (default)
        - ``'pos'``: those queries count as ``1.0``
        - ``'skip'``: skip those queries; if all queries are skipped, ``0.0`` is returned
        - ``'error'``: raise a ``ValueError``

    ignore_index: Ignore predictions where the target is equal to this number.
    top_k: Consider only the top k elements for each query (default: ``None``, which considers them all)
    max_fpr: If not ``None``, calculates standardized partial AUC over the range ``[0, max_fpr]``.
    aggregation:
        Specify how to aggregate over indexes. Can either a custom callable function that takes in a single tensor
        and returns a scalar value or one of the following strings:

        - ``'mean'``: average value is returned
        - ``'median'``: median value is returned
        - ``'max'``: max value is returned
        - ``'min'``: min value is returned

    kwargs: Additional keyword arguments, see :ref:`Metric kwargs`

Quick recipe

import torchmetrics.retrieval as _m
score = _m.RetrievalAUROC(y_true, y_pred)

Don'ts

• Don't reimplement when the library version handles edge cases (NaN, ties, empty inputs) better than a hand-rolled formula.
• Always check the library version's argument order — sklearn is (y_true, y_pred) while torchmetrics is (preds, target).