# Cohen Kappa Score

> Compute the cohen_kappa_score metric — provided by scikit-learn. Use when the user has predictions and ground-truth and needs to compute cohen_kappa_score, or asks how to score with cohen_kappa_score.

- Skill: `qhjqhj00/cohen-kappa-score` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/cohen-kappa-score`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/cohen-kappa-score/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/cohen-kappa-score

---


# cohen-kappa-score

> Metric `cohen_kappa_score` from `scikit-learn` (sklearn.metrics.cohen_kappa_score)

## When to invoke this skill

The user has predictions + ground truth and asks to evaluate with cohen_kappa_score, or
mentions `sklearn.metrics.cohen_kappa_score` directly, or wants the standard scikit-learn implementation.

## Reference signature

```python
from sklearn.metrics import cohen_kappa_score

# cohen_kappa_score(y1, y2, *, labels=None, weights=None, sample_weight=None)
```

## Library docstring

```
Compute Cohen's kappa: a statistic that measures inter-annotator agreement.

This function computes Cohen's kappa [1]_, a score that expresses the level
of agreement between two annotators on a classification problem. It is
defined as

.. math::
    \kappa = (p_o - p_e) / (1 - p_e)

where :math:`p_o` is the empirical probability of agreement on the label
assigned to any sample (the observed agreement ratio), and :math:`p_e` is
the expected agreement when both annotators assign labels randomly.
:math:`p_e` is estimated using a per-annotator empirical prior over the
class labels [2]_.

Read more in the :ref:`User Guide <cohen_kappa>`.

Parameters
----------
y1 : array-like of shape (n_samples,)
    Labels assigned by the first annotator.

y2 : array-like of shape (n_samples,)
    Labels assigned by the second annotator. The kappa statistic is
    symmetric, so swapping ``y1`` and ``y2`` doesn't change the value.

labels : array-like of shape (n_classes,), default=None
    List of labels to index the matrix. This may be used to select a
    subset of labels. If `None`, all labels that appear at least once in
    ``y1`` or ``y2`` are used. Note that at least one label in `labels` must be
    present in `y1`, even though this function is otherwise agnostic to the order
    of `y1` and `y2`.

weights : {'linear', 'quadratic'}, default=None
    Weighting type to calculate the score. `None` means not weighted;
    "linear" means linear weighting; "quadratic" means quadratic weighting.

sample_weight : array-like of shape (n_samples,), default=None
    Sample weights.

Returns
-------
kappa : float
    The kappa statistic, which is a number between -1 and 1. The maximum
    value means complete agreement; zero or lower means chance agreement.

References
----------
.. [1] :doi:`J. Cohen (1960). "A coefficient of agreement for nominal scales".
       Educational and Psychological Measurement 20(1):37-46.
       <10.1177/001316446002000104>`
.. [2] `R. Artstein and M. Poesio (2008). "Inter-coder agreement for
       computational linguistics". Computational Linguistics 34(4):555-596
       <https://www.mitpressjournals.org/doi/pdf/10.1162/coli.07-034-R2>`_.
.. [3] `Wikiped
```

## Quick recipe

```python
import sklearn.metrics as _m
score = _m.cohen_kappa_score(y_true, y_pred)
```

## Don'ts

- Don't reimplement when the library version handles edge cases (NaN, ties, empty inputs) better than a hand-rolled formula.
- Always check the library version's argument order — sklearn is `(y_true, y_pred)` while torchmetrics is `(preds, target)`.