Fisher Exact

fisher-exact

Metric fisher_exact from scipy.stats (scipy.stats.fisher_exact)

When to invoke this skill

The user has predictions + ground truth and asks to evaluate with fisher_exact, or mentions scipy.stats.fisher_exact directly, or wants the standard scipy.stats implementation.

Reference signature

from scipy.stats import fisher_exact

# fisher_exact(table, alternative=None, *, method=None)

Library docstring

Perform a Fisher exact test on a contingency table.

For a 2x2 table,
the null hypothesis is that the true odds ratio of the populations
underlying the observations is one, and the observations were sampled
from these populations under a condition: the marginals of the
resulting table must equal those of the observed table.
The statistic is the unconditional maximum likelihood estimate of the odds
ratio, and the p-value is the probability under the null hypothesis of
obtaining a table at least as extreme as the one that was actually
observed.

For other table sizes, or if `method` is provided, the null hypothesis
is that the rows and columns of the tables have fixed sums and are
independent; i.e., the table was sampled from a `scipy.stats.random_table`
distribution with the observed marginals. The statistic is the
probability mass of this distribution evaluated at `table`, and the
p-value is the percentage of the population of tables with statistic at
least as extreme (small) as that of `table`. There is only one alternative
hypothesis available: the rows and columns are not independent.

There are other possible choices of statistic and two-sided
p-value definition associated with Fisher's exact test; please see the
Notes for more information.

Parameters
----------
table : array_like of ints
    A contingency table.  Elements must be non-negative integers.
alternative : {'two-sided', 'less', 'greater'}, optional
    Defines the alternative hypothesis for 2x2 tables; unused for other
    table sizes.
    The following options are available (default is 'two-sided'):

    * 'two-sided': the odds ratio of the underlying population is not one
    * 'less': the odds ratio of the underlying population is less than one
    * 'greater': the odds ratio of the underlying population is greater
      than one

    See the Notes for more details.
method : ResamplingMethod, optional
    Defines the method used to compute the p-value.
    If `method` is an instance of `PermutationMethod`/`MonteCarloMethod`,
    the p-value is computed using
    `scipy.stats.permutation_test`/`scipy.stats.monte_carlo_test` with the
    provided configuration options and other appropriate setti

Quick recipe

import scipy.stats as _m
score = _m.fisher_exact(y_true, y_pred)

Don'ts

• Don't reimplement when the library version handles edge cases (NaN, ties, empty inputs) better than a hand-rolled formula.
• Always check the library version's argument order — sklearn is (y_true, y_pred) while torchmetrics is (preds, target).