Cohenkappa

cohenkappa

Metric CohenKappa from torchmetrics (torchmetrics.CohenKappa)

When to invoke this skill

The user has predictions + ground truth and asks to evaluate with CohenKappa, or mentions torchmetrics.CohenKappa directly, or wants the standard torchmetrics implementation.

Reference signature

from torchmetrics import CohenKappa

# CohenKappa(task: Literal['binary', 'multiclass'], threshold: float = 0.5, num_classes: Optional[int] = None, weights: Optional[Literal['linear', 'quadratic', 'none']] = None, ignore_index: Optional[int] = None, validate_args: bool = True, **kwargs: Any) -> torchmetrics.metric.Metric

Library docstring

Calculate `Cohen's kappa score`_ that measures inter-annotator agreement.

.. math::
    \kappa = (p_o - p_e) / (1 - p_e)

where :math:`p_o` is the empirical probability of agreement and :math:`p_e` is
the expected agreement when both annotators assign labels randomly. Note that
:math:`p_e` is estimated using a per-annotator empirical prior over the
class labels.

This function is a simple wrapper to get the task specific versions of this metric, which is done by setting the
``task`` argument to either ``'binary'`` or ``'multiclass'``. See the documentation of
:class:`~torchmetrics.classification.BinaryCohenKappa` and
:class:`~torchmetrics.classification.MulticlassCohenKappa` for the specific details of each argument influence and
examples.

Legacy Example:
    >>> from torch import tensor
    >>> target = tensor([1, 1, 0, 0])
    >>> preds = tensor([0, 1, 0, 0])
    >>> cohenkappa = CohenKappa(task="multiclass", num_classes=2)
    >>> cohenkappa(preds, target)
    tensor(0.5000)

Quick recipe

import torchmetrics as _m
score = _m.CohenKappa(y_true, y_pred)

Don'ts

• Don't reimplement when the library version handles edge cases (NaN, ties, empty inputs) better than a hand-rolled formula.
• Always check the library version's argument order — sklearn is (y_true, y_pred) while torchmetrics is (preds, target).