Multitaskwrapper

multitaskwrapper

Metric MultitaskWrapper from torchmetrics (torchmetrics.MultitaskWrapper)

When to invoke this skill

The user has predictions + ground truth and asks to evaluate with MultitaskWrapper, or mentions torchmetrics.MultitaskWrapper directly, or wants the standard torchmetrics implementation.

Reference signature

from torchmetrics import MultitaskWrapper

# MultitaskWrapper(task_metrics: dict[str, typing.Union[torchmetrics.metric.Metric, torchmetrics.collections.MetricCollection]], prefix: Optional[str] = None, postfix: Optional[str] = None) -> None

Library docstring

Wrapper class for computing different metrics on different tasks in the context of multitask learning.

In multitask learning the different tasks requires different metrics to be evaluated. This wrapper allows
for easy evaluation in such cases by supporting multiple predictions and targets through a dictionary.
Note that only metrics where the signature of `update` follows the standard `preds, target` is supported.

Args:
    task_metrics:
        Dictionary associating each task to a Metric or a MetricCollection. The keys of the dictionary represent the
        names of the tasks, and the values represent the metrics to use for each task.
    prefix:
        A string to append in front of the metric keys. If not provided, will default to an empty string.
    postfix:
        A string to append after the keys of the output dict. If not provided, will default to an empty string.

.. tip::
    The use prefix and postfix allows for easily creating task wrappers for training, validation and test.
    The arguments are only changing the output keys of the computed metrics and not the input keys. This means
    that a ``MultitaskWrapper`` initialized as ``MultitaskWrapper({"task": Metric()}, prefix="train_")`` will
    still expect the input to be a dictionary with the key "task", but the output will be a dictionary with the key
    "train_task".

Raises:
    TypeError:
        If argument `task_metrics` is not an dictionary
    TypeError:
        If not all values in the `task_metrics` dictionary is instances of `Metric` or `MetricCollection`
    ValueError:
        If `prefix` is not a string
    ValueError:
        If `postfix` is not a string

Example (with a single metric per class):
     >>> import torch
     >>> from torchmetrics.wrappers import MultitaskWrapper
     >>> from torchmetrics.regression import MeanSquaredError
     >>> from torchmetrics.classification import BinaryAccuracy
     >>>
     >>> classification_target = torch.tensor([0, 1, 0])
     >>> regression_target = torch.tensor([2.5, 5.0, 4.0])
     >>> targets = {"Classification": classification_target, "Regression": regression_target}
     >>>
     >>> classification_preds = torch.tensor([0, 0, 

Quick recipe

import torchmetrics as _m
score = _m.MultitaskWrapper(y_true, y_pred)

Don'ts

• Don't reimplement when the library version handles edge cases (NaN, ties, empty inputs) better than a hand-rolled formula.
• Always check the library version's argument order — sklearn is (y_true, y_pred) while torchmetrics is (preds, target).