Memorizationinformedfrechetinceptiondistance

memorizationinformedfrechetinceptiondistance

Metric MemorizationInformedFrechetInceptionDistance from torchmetrics (torchmetrics.image.MemorizationInformedFrechetInceptionDistance)

When to invoke this skill

The user has predictions + ground truth and asks to evaluate with MemorizationInformedFrechetInceptionDistance, or mentions torchmetrics.image.MemorizationInformedFrechetInceptionDistance directly, or wants the standard torchmetrics implementation.

Reference signature

from torchmetrics.image import MemorizationInformedFrechetInceptionDistance

# MemorizationInformedFrechetInceptionDistance(feature: Union[int, torch.nn.modules.module.Module] = 2048, reset_real_features: bool = True, normalize: bool = False, cosine_distance_eps: float = 0.1, **kwargs: Any) -> None

Library docstring

Calculate Memorization-Informed Frechet Inception Distance (MIFID_).

MIFID is a improved variation of the Frechet Inception Distance (FID_) that penalizes memorization of the training
set by the generator. It is calculated as

.. math::
    MIFID = \frac{FID(F_{real}, F_{fake})}{M(F_{real}, F_{fake})}

where :math:`FID` is the normal FID score and :math:`M` is the memorization penalty. The memorization penalty
essentially corresponds to the average minimum cosine distance between the features of the real and fake
distribution.

Using the default feature extraction (Inception v3 using the original weights from `fid ref2`_), the input is
expected to be mini-batches of 3-channel RGB images of shape ``(3 x H x W)``. If argument ``normalize``
is ``True`` images are expected to be dtype ``float`` and have values in the ``[0, 1]`` range, else if
``normalize`` is set to ``False`` images are expected to have dtype ``uint8`` and take values in the ``[0, 255]``
range. All images will be resized to 299 x 299 which is the size of the original training data. The boolian
flag ``real`` determines if the images should update the statistics of the real distribution or the
fake distribution.

.. hint::
    Using this metrics requires you to have ``scipy`` install. Either install as ``pip install
    torchmetrics[image]`` or ``pip install scipy``

.. hint::
    Using this metric with the default feature extractor requires that ``torch-fidelity``
    is installed. Either install as ``pip install torchmetrics[image]`` or
    ``pip install torch-fidelity``

As input to ``forward`` and ``update`` the metric accepts the following input

- ``imgs`` (:class:`~torch.Tensor`): tensor with images feed to the feature extractor with
- ``real`` (:class:`~bool`): bool indicating if ``imgs`` belong to the real or the fake distribution

As output of `forward` and `compute` the metric returns the following output

- ``mifid`` (:class:`~torch.Tensor`): float scalar tensor with mean MIFID value over samples

Args:
    feature:
        Either an integer or ``nn.Module``:

        - an integer will indicate the inceptionv3 feature layer to choose. Can be one of the following:
          64, 192, 768, 2

Quick recipe

import torchmetrics.image as _m
score = _m.MemorizationInformedFrechetInceptionDistance(y_true, y_pred)

Don'ts

• Don't reimplement when the library version handles edge cases (NaN, ties, empty inputs) better than a hand-rolled formula.
• Always check the library version's argument order — sklearn is (y_true, y_pred) while torchmetrics is (preds, target).