Lipvertexerror

lipvertexerror

Metric LipVertexError from torchmetrics (torchmetrics.multimodal.LipVertexError)

When to invoke this skill

The user has predictions + ground truth and asks to evaluate with LipVertexError, or mentions torchmetrics.multimodal.LipVertexError directly, or wants the standard torchmetrics implementation.

Reference signature

from torchmetrics.multimodal import LipVertexError

# LipVertexError(mouth_map: List[int], validate_args: bool = True, **kwargs: Any) -> None

Library docstring

Implements Lip Vertex Error (LVE) metric for 3D talking head evaluation.

The Lip Vertex Error (LVE) metric evaluates the quality of lip synchronization in 3D facial animations by measuring
the maximum Euclidean distance (L2 error) between corresponding lip vertices of the generated and ground truth
meshes for each frame. The metric is defined as:

.. math::
    \text{LVE} = \frac{1}{N} \sum_{i=1}^{N} \max_{v \in \text{lip}} \|x_{i,v} - \hat{x}_{i,v}\|_2^2

where :math:`N` is the number of frames, :math:`x_{i,v}` represents the 3D coordinates of vertex :math:`v` in the
lip region of the ground truth frame :math:`i`, and :math:`\hat{x}_{i,v}` represents the corresponding vertex in the
predicted frame. The metric computes the maximum squared L2 distance between corresponding lip vertices for each
frame and averages across all frames. A lower LVE value indicates better lip synchronization quality.

As input to ``forward`` and ``update``, the metric accepts the following input:

- ``preds`` (:class:`~torch.Tensor`): Predicted vertices tensor of shape (T, V, 3) where T is number of frames,
            V is number of vertices, and 3 represents XYZ coordinates
- ``target`` (:class:`~torch.Tensor`): Ground truth vertices tensor of shape (T', V, 3) where T' can be different
            from T

As output of ``forward`` and ``compute``, the metric returns the following output:

- ``lve_score`` (:class:`~torch.Tensor`): A scalar tensor containing the mean Lip Vertex Error value across
            all frames.

Args:
    mouth_map: List of vertex indices corresponding to the mouth region
    validate_args: bool indicating if input arguments and tensors should be validated for correctness.
        Set to ``False`` for faster computations.
    kwargs: Additional keyword arguments, see :ref:`Metric kwargs` for more info.

Raises:
    ValueError:
        If the number of dimensions of `vertices_pred` or `vertices_gt` is not 3.
        If vertex dimensions (V) or coordinate dimensions (3) don't match
        If ``mouth_map`` is empty or contains invalid indices

Example:
    >>> import torch
    >>> from torchmetrics.functional.multimodal import lip_vertex_error
    >>> vertices_p

Quick recipe

import torchmetrics.multimodal as _m
score = _m.LipVertexError(y_true, y_pred)

Don'ts

• Don't reimplement when the library version handles edge cases (NaN, ties, empty inputs) better than a hand-rolled formula.
• Always check the library version's argument order — sklearn is (y_true, y_pred) while torchmetrics is (preds, target).