# Multiple Testing Correction

> Use when whenever you have performed Fisher's exact test or another statistical enrichment test on multiple pathways, lipid categories, or metabolite sets simultaneously (typically ≥2 tests, often 50–100+ tests in practice).

- Skill: `holobiomicslab/multiple-testing-correction` (Agent Skill)
- Install (CLI): `npx skillmds@latest add holobiomicslab/multiple-testing-correction`
- Raw SKILL.md: https://api.skillmd.com/api/skills/holobiomicslab/multiple-testing-correction/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- License: CC-BY-4.0
- Author: HolobiomicsLab (https://skillmd.com/u/holobiomicslab)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/holobiomicslab/multiple-testing-correction

---


# multiple-testing-correction

> **License: restricted** — no clear open-source license detected for the underlying tool; verify licensing before commercial use or redistribution. <!-- asb-license-banner -->
## Summary

Apply multiple-testing correction (Benjamini–Hochberg or other family-wise error rate control) to p-values derived from pathway enrichment tests to control false discovery rate across many simultaneous statistical tests. This skill prevents inflated Type I error when testing hundreds of pathways or metabolite categories against a single input metabolite set.

## When to use

Apply this skill whenever you have performed Fisher's exact test or another statistical enrichment test on multiple pathways, lipid categories, or metabolite sets simultaneously (typically ≥2 tests, often 50–100+ tests in practice). The raw p-values from each individual test do not account for the multiple comparisons problem; correction is required before reporting or filtering significant results.

## When NOT to use

- Input is a single hypothesis test (n=1 pathway or category) — no correction needed.
- P-values have already been corrected by the upstream software — applying correction twice will introduce bias.
- Analysis is exploratory and raw p-values are acceptable — though not recommended for publication without adjustment disclosure.

## Inputs

- vector or data.frame column of raw p-values from Fisher's exact test (one p-value per pathway or metabolite category)
- number of tests performed (inferred from p-value vector length)

## Outputs

- data.frame or vector of adjusted p-values (same length as input)
- enrichment results table with columns: pathway/category name, raw p-value, adjusted p-value, odds ratio or effect size, metabolite count

## How to apply

After computing raw p-values from Fisher's exact test for each pathway or category, apply Benjamini–Hochberg (BH) correction to the full set of p-values using a standard method (e.g., R's p.adjust() function with method='BH'). The BH procedure controls false discovery rate (FDR) while preserving statistical power better than strict Bonferroni correction. Store both raw and adjusted p-values in the enrichment results table; use adjusted p-values (typically with threshold p_adj ≤ 0.05) for downstream filtering, visualization, and reporting. Document the correction method and cutoff in all output tables and figures.

## Related tools

- **R p.adjust() function** (Performs Benjamini–Hochberg and other multiple-testing corrections on p-value vectors)
- **enrichmet** (Integrates Benjamini–Hochberg correction as a built-in step in pathway enrichment workflow; automatically computes and returns adjusted p-values in enrichment results table) — https://github.com/biodatalab/enrichmet
- **stats::p.adjust (R base)** (General-purpose multiple-testing correction for any set of p-values)

## Examples

```
results <- enrichmet(inputMetabolites = inputMetabolites, PathwayVsMetabolites = PathwayVsMetabolites, p_value_cutoff = 0.05, min_pathway_occurrence = 2); # Benjamini–Hochberg correction is applied internally; view corrected p-values in results$pathway_enrichment_all
```

## Evaluation signals

- Adjusted p-values are monotonically non-decreasing when sorted by raw p-value (verification of correct BH ranking).
- Adjusted p-value ≥ corresponding raw p-value for all tests (BH correction always inflates p-values to be conservative).
- Number of significant pathways after adjustment (adj_p ≤ 0.05) is ≤ number before adjustment.
- Enrichment results table contains both 'P_value' (raw) and adjusted p-value columns with clear labeling (e.g., 'Adjusted_P_value' or 'FDR').
- Reported significant findings cite the corrected p-value threshold and correction method (e.g., 'Benjamini–Hochberg corrected p ≤ 0.05').

## Limitations

- BH correction assumes tests are independent or positively dependent; if pathways share metabolites (common in real data), the assumption may be violated, though BH remains valid.
- With very large numbers of tests (e.g., >10,000), BH correction may be overly conservative and reduce power; more sophisticated methods (e.g., Storey's q-value) may be preferred in those cases.
- Correction cannot recover signal from inherently noisy or underpowered individual tests; if raw p-values are all close to 1, correction will not rescue significance.
- Choice of FDR threshold (0.05 vs. 0.1 vs. 0.01) is arbitrary and should be stated a priori; post-hoc threshold selection risks p-hacking.

## Evidence

- [intro] Compute adjusted p-values using Benjamini–Hochberg correction: "Compute adjusted p-values using Benjamini–Hochberg correction."
- [intro] enrichment results table with pathways, metabolite counts, p-values, adjusted p-values, and effect sizes: "data.frame with pathways, metabolite counts, p-values, adjusted p-values, and effect sizes"
- [intro] p_value_cutoff parameter controls significance threshold for individual tests: "p_value_cutoff = 0.05"
- [intro] Fisher's exact test enrichment workflow executed on each category: "Execute Fisher's exact test on each lipid ontology category using the enrichmet workflow to test for significant association"