Binaryfairness

binaryfairness

Metric BinaryFairness from torchmetrics (torchmetrics.classification.BinaryFairness)

When to invoke this skill

The user has predictions + ground truth and asks to evaluate with BinaryFairness, or mentions torchmetrics.classification.BinaryFairness directly, or wants the standard torchmetrics implementation.

Reference signature

from torchmetrics.classification import BinaryFairness

# BinaryFairness(num_groups: int, task: Literal['demographic_parity', 'equal_opportunity', 'all'] = 'all', threshold: float = 0.5, ignore_index: Optional[int] = None, validate_args: bool = True, **kwargs: Any) -> None

Library docstring

Computes `Demographic parity`_ and `Equal opportunity`_ ratio for binary classification problems.

Accepts the following input tensors:

- ``preds`` (int or float tensor): ``(N, ...)``. If preds is a floating point tensor with values outside
  [0,1] range we consider the input to be logits and will auto apply sigmoid per element. Additionally,
  we convert to int tensor with thresholding using the value in ``threshold``.
- ``groups`` (int tensor): ``(N, ...)``. The group identifiers should be ``0, 1, ..., (num_groups - 1)``.
- ``target`` (int tensor): ``(N, ...)``.

The additional dimensions are flatted along the batch dimension.

This class computes the ratio between positivity rates and true positives rates for different groups.
If more than two groups are present, the disparity between the lowest and highest group is reported.
A disparity between positivity rates indicates a potential violation of demographic parity, and between
true positive rates indicates a potential violation of equal opportunity.

The lowest rate is divided by the highest, so a lower value means more discrimination against the numerator.
In the results this is also indicated as the key of dict is {metric}_{identifier_low_group}_{identifier_high_group}.

Args:
    num_groups: The number of groups.
    task: The task to compute. Can be either ``demographic_parity`` or ``equal_opportunity`` or ``all``.
    threshold: Threshold for transforming probability to binary {0,1} predictions.
    ignore_index: Specifies a target value that is ignored and does not contribute to the metric calculation
    validate_args: bool indicating if input arguments and tensors should be validated for correctness.
        Set to ``False`` for faster computations.
    kwargs: Additional keyword arguments, see :ref:`Metric kwargs` for more info.

Returns:
    The metric returns a dict where the key identifies the metric and groups with the lowest and highest true
    positives rates as follows: {metric}__{identifier_low_group}_{identifier_high_group}.
    The value is a tensor with the disparity rate.

Example (preds is int tensor):
    >>> from torchmetrics.classification import BinaryFairness
    >>> target = tor

Quick recipe

import torchmetrics.classification as _m
score = _m.BinaryFairness(y_true, y_pred)

Don'ts

• Don't reimplement when the library version handles edge cases (NaN, ties, empty inputs) better than a hand-rolled formula.
• Always check the library version's argument order — sklearn is (y_true, y_pred) while torchmetrics is (preds, target).