fisher-exact-test-computation
Summary
Compute Fisher exact test p-values to statistically compare the proportion of validated links across categorized link sets, determining whether dual-score enrichment (links exceeding multiple scoring thresholds) is significantly different from single-score enrichment. This enables rigorous hypothesis testing of complementary scoring approach performance.
When to use
You have partitioned genomic-metabolomic links into discrete categories (e.g., above 90th percentile for score A only, score B only, both scores, or neither) and need to test whether the proportion of validated links differs significantly between categories—particularly when comparing a dual-threshold (combined) category against single-threshold or baseline categories.
When NOT to use
- Input is continuous score data rather than categorical counts—use rank-based tests (Mann–Whitney U) or regression instead.
- Sample sizes are very large (>1000 per cell) and expected frequencies are all >5; a χ² test may be more efficient.
- You seek to rank individual links rather than compare category proportions—use threshold-filtering or scoring directly.
Inputs
- Partitioned link categories with counts: {validated_in_category, total_in_category} per category
- Link categorization scheme: e.g., [both_scores_≥90th_percentile, score_A_only_≥90th_percentile, score_B_only_≥90th_percentile, both_scores_<90th_percentile]
- Pooled or per-dataset contingency tables
Outputs
- Fisher exact test p-values comparing dual-score category to single-score categories
- Proportions of validated links per category
- Statistical comparison table (e.g., Table 2 format: proportions and p-values across datasets and pooled analysis)
How to apply
For each link category, compute the proportion of validated links (number of validated links in category / total links in category). Pool data across datasets (e.g., Crüsemann, Gross, Leão) to increase statistical power. Use Fisher exact test to compare the proportion of validated links in the dual-score category (links scoring ≥90th percentile on both standardised strain correlation AND IOKR) against each single-score category. Report two-tailed p-values; significance is typically assessed at α = 0.05. The rationale is that Fisher exact test is appropriate for small sample sizes and categorical (2×2 contingency table) data, making it ideal for validating whether combining independent scoring functions genuinely enriches for true links relative to using either score alone.
Related tools
- NPLinker (Framework that generates partitioned GCF-MF link categories and scores (strain correlation, IOKR) on which Fisher test is applied) — https://github.com/sdrogers/nplinker
- Python scipy.stats.fisher_exact or R stats::fisher.test (Computes Fisher exact test p-values and contingency statistics)
Examples
from scipy.stats import fisher_exact; contingency = [[validated_dual, non_validated_dual], [validated_single, non_validated_single]]; oddsratio, pvalue = fisher_exact(contingency, alternative='two-sided'); print(f'Fisher p-value: {pvalue}')
Evaluation signals
- P-values for dual-score category vs. single-score categories are computed correctly from 2×2 contingency tables (validated/non-validated × in_category/not_in_category).
- Proportions are calculated correctly: validated_in_category / total_in_category for each category, matching Table 2 reported values.
- P-values from pooled analysis match or corroborate per-dataset p-values; pooling should yield smaller p-values if effect is consistent.
- Dual-score p-value is significantly smaller than single-score p-values, demonstrating enrichment; reported p-values are 2.633 × 10⁻⁴ (IOKR) and 0.0208 (standardised strain correlation) in the article.
- Results are reproducible across all three datasets (Crüsemann, Gross, Leão) and tabulated in comparable structure to reported findings.
Limitations
- Fisher exact test requires categorical data (counts); if scores are continuous, thresholding introduces information loss and the choice of threshold (e.g., 90th percentile) affects statistical power and conclusions.
- Test assumes independence of observations; if links share underlying biological structure (e.g., same BGC or metabolite cluster), multiple testing correction may be needed.
- Small sample sizes in individual link categories can reduce power; pooling across datasets helps but may mask dataset-specific effects.
- Fisher exact test is one-tailed by default in some implementations; two-tailed p-values must be explicitly requested or computed to avoid underestimating significance.
Evidence
- [results] Links scoring above the 90th percentile on both standardised strain correlation and IOKR scores are significantly enriched for validated links (p-value 2.633 × 10−4 from IOKR and 0.0208 from standardised strain correlation): "the set of links scoring above the 90th percentile on both scores is significantly enriched compared to the set that exceed either of the individual scores (p-value of 2.633 × 10−4 and 0.0208"
- [other] For each category, compute the proportion of validated links (number of validated links in category / total links in category). Pool data across all three datasets and compute Fisher exact test p-values.: "For each category, compute the proportion of validated links (number of validated links in category / total links in category). 5. Pool data across all three datasets and compute Fisher exact test"
- [other] Tabulate results showing proportions and p-values for all three datasets and pooled analysis, matching the structure and values reported in Table 2.: "Tabulate results showing proportions and p-values for all three datasets and pooled analysis, matching the structure and values reported in Table 2"
- [other] Fisher exact test p-values comparing the proportion of validated links in the dual-score (both scores ≥90th percentile) category to each single-score category.: "compute Fisher exact test p-values comparing the proportion of validated links in the dual-score (both scores ≥90th percentile) category to each single-score category"
- [other] Partition links into four categories: above 90th percentile for standardised correlation only, above 90th percentile for IOKR only, above 90th percentile for both scores, and below 90th percentile for both.: "Partition links into four categories: above 90th percentile for standardised correlation only, above 90th percentile for IOKR only, above 90th percentile for both scores, and below 90th percentile"
1---2name: fisher-exact-test-computation3description: Use when you have partitioned genomic-metabolomic links into discrete categories (e.4license: CC-BY-4.05---67# fisher-exact-test-computation89## Summary1011Compute Fisher exact test p-values to statistically compare the proportion of validated links across categorized link sets, determining whether dual-score enrichment (links exceeding multiple scoring thresholds) is significantly different from single-score enrichment. This enables rigorous hypothesis testing of complementary scoring approach performance.1213## When to use1415You have partitioned genomic-metabolomic links into discrete categories (e.g., above 90th percentile for score A only, score B only, both scores, or neither) and need to test whether the proportion of validated links differs significantly between categories—particularly when comparing a dual-threshold (combined) category against single-threshold or baseline categories.1617## When NOT to use1819- Input is continuous score data rather than categorical counts—use rank-based tests (Mann–Whitney U) or regression instead.20- Sample sizes are very large (>1000 per cell) and expected frequencies are all >5; a χ² test may be more efficient.21- You seek to rank individual links rather than compare category proportions—use threshold-filtering or scoring directly.2223## Inputs2425- Partitioned link categories with counts: {validated_in_category, total_in_category} per category26- Link categorization scheme: e.g., [both_scores_≥90th_percentile, score_A_only_≥90th_percentile, score_B_only_≥90th_percentile, both_scores_<90th_percentile]27- Pooled or per-dataset contingency tables2829## Outputs3031- Fisher exact test p-values comparing dual-score category to single-score categories32- Proportions of validated links per category33- Statistical comparison table (e.g., Table 2 format: proportions and p-values across datasets and pooled analysis)3435## How to apply3637For each link category, compute the proportion of validated links (number of validated links in category / total links in category). Pool data across datasets (e.g., Crüsemann, Gross, Leão) to increase statistical power. Use Fisher exact test to compare the proportion of validated links in the dual-score category (links scoring ≥90th percentile on both standardised strain correlation AND IOKR) against each single-score category. Report two-tailed p-values; significance is typically assessed at α = 0.05. The rationale is that Fisher exact test is appropriate for small sample sizes and categorical (2×2 contingency table) data, making it ideal for validating whether combining independent scoring functions genuinely enriches for true links relative to using either score alone.3839## Related tools4041- **NPLinker** (Framework that generates partitioned GCF-MF link categories and scores (strain correlation, IOKR) on which Fisher test is applied) — https://github.com/sdrogers/nplinker42- **Python scipy.stats.fisher_exact or R stats::fisher.test** (Computes Fisher exact test p-values and contingency statistics)4344## Examples4546```47from scipy.stats import fisher_exact; contingency = [[validated_dual, non_validated_dual], [validated_single, non_validated_single]]; oddsratio, pvalue = fisher_exact(contingency, alternative='two-sided'); print(f'Fisher p-value: {pvalue}')48```4950## Evaluation signals5152- P-values for dual-score category vs. single-score categories are computed correctly from 2×2 contingency tables (validated/non-validated × in_category/not_in_category).53- Proportions are calculated correctly: validated_in_category / total_in_category for each category, matching Table 2 reported values.54- P-values from pooled analysis match or corroborate per-dataset p-values; pooling should yield smaller p-values if effect is consistent.55- Dual-score p-value is significantly smaller than single-score p-values, demonstrating enrichment; reported p-values are 2.633 × 10⁻⁴ (IOKR) and 0.0208 (standardised strain correlation) in the article.56- Results are reproducible across all three datasets (Crüsemann, Gross, Leão) and tabulated in comparable structure to reported findings.5758## Limitations5960- Fisher exact test requires categorical data (counts); if scores are continuous, thresholding introduces information loss and the choice of threshold (e.g., 90th percentile) affects statistical power and conclusions.61- Test assumes independence of observations; if links share underlying biological structure (e.g., same BGC or metabolite cluster), multiple testing correction may be needed.62- Small sample sizes in individual link categories can reduce power; pooling across datasets helps but may mask dataset-specific effects.63- Fisher exact test is one-tailed by default in some implementations; two-tailed p-values must be explicitly requested or computed to avoid underestimating significance.6465## Evidence6667- [results] Links scoring above the 90th percentile on both standardised strain correlation and IOKR scores are significantly enriched for validated links (p-value 2.633 × 10−4 from IOKR and 0.0208 from standardised strain correlation): "the set of links scoring above the 90th percentile on both scores is significantly enriched compared to the set that exceed either of the individual scores (p-value of 2.633 × 10−4 and 0.0208"68- [other] For each category, compute the proportion of validated links (number of validated links in category / total links in category). Pool data across all three datasets and compute Fisher exact test p-values.: "For each category, compute the proportion of validated links (number of validated links in category / total links in category). 5. Pool data across all three datasets and compute Fisher exact test"69- [other] Tabulate results showing proportions and p-values for all three datasets and pooled analysis, matching the structure and values reported in Table 2.: "Tabulate results showing proportions and p-values for all three datasets and pooled analysis, matching the structure and values reported in Table 2"70- [other] Fisher exact test p-values comparing the proportion of validated links in the dual-score (both scores ≥90th percentile) category to each single-score category.: "compute Fisher exact test p-values comparing the proportion of validated links in the dual-score (both scores ≥90th percentile) category to each single-score category"71- [other] Partition links into four categories: above 90th percentile for standardised correlation only, above 90th percentile for IOKR only, above 90th percentile for both scores, and below 90th percentile for both.: "Partition links into four categories: above 90th percentile for standardised correlation only, above 90th percentile for IOKR only, above 90th percentile for both scores, and below 90th percentile"
HolobiomicsLab