Editdistance

editdistance

Metric EditDistance from torchmetrics (torchmetrics.text.EditDistance)

When to invoke this skill

The user has predictions + ground truth and asks to evaluate with EditDistance, or mentions torchmetrics.text.EditDistance directly, or wants the standard torchmetrics implementation.

Reference signature

from torchmetrics.text import EditDistance

# EditDistance(substitution_cost: int = 1, reduction: Optional[Literal['mean', 'sum', 'none']] = 'mean', **kwargs: Any) -> None

Library docstring

Calculates the Levenshtein edit distance between two sequences.

The edit distance is the number of characters that need to be substituted, inserted, or deleted, to transform the
predicted text into the reference text. The lower the distance, the more accurate the model is considered to be.

Implementation is similar to `nltk.edit_distance <https://www.nltk.org/_modules/nltk/metrics/distance.html>`_.

As input to ``forward`` and ``update`` the metric accepts the following input:

- ``preds`` (:class:`~Sequence`): An iterable of hypothesis corpus
- ``target`` (:class:`~Sequence`): An iterable of iterables of reference corpus

As output of ``forward`` and ``compute`` the metric returns the following output:

- ``eed`` (:class:`~torch.Tensor`): A tensor with the extended edit distance score. If `reduction` is set to
  ``'none'`` or ``None``, this has shape ``(N, )``, where ``N`` is the batch size. Otherwise, this is a scalar.

Args:
    substitution_cost: The cost of substituting one character for another.
    reduction: a method to reduce metric score over samples.

        - ``'mean'``: takes the mean over samples
        - ``'sum'``: takes the sum over samples
        - ``None`` or ``'none'``: return the score per sample

    kwargs: Additional keyword arguments, see :ref:`Metric kwargs` for more info.

Example::
    Basic example with two strings. Going from “rain” -> “sain” -> “shin” -> “shine” takes 3 edits:

    >>> from torchmetrics.text import EditDistance
    >>> metric = EditDistance()
    >>> metric(["rain"], ["shine"])
    tensor(3.)

Example::
    Basic example with two strings and substitution cost of 2. Going from “rain” -> “sain” -> “shin” -> “shine”
    takes 3 edits, where two of them are substitutions:

    >>> from torchmetrics.text import EditDistance
    >>> metric = EditDistance(substitution_cost=2)
    >>> metric(["rain"], ["shine"])
    tensor(5.)

Example::
    Multiple strings example:

    >>> from torchmetrics.text import EditDistance
    >>> metric = EditDistance(reduction=None)
    >>> metric(["rain", "lnaguaeg"], ["shine", "language"])
    tensor([3, 4], dtype=torch.int32)
    >>> metric = EditDistance(reduction="mean")
    >>> metr

Quick recipe

import torchmetrics.text as _m
score = _m.EditDistance(y_true, y_pred)

Don'ts

• Don't reimplement when the library version handles edge cases (NaN, ties, empty inputs) better than a hand-rolled formula.
• Always check the library version's argument order — sklearn is (y_true, y_pred) while torchmetrics is (preds, target).