Adaptive Multi-Level Split Monte-Carlo P-Value Estimation
Summary
A method for computing arbitrarily low gene set enrichment analysis (GSEA) P-values using an adaptive multi-level split Monte-Carlo scheme, eliminating fixed lower-bound constraints by enabling precision refinement through iterative sampling. This skill is essential when gene set rankings must be compared at extreme statistical significance levels or when default P-value resolution (e.g., eps=1e-10) is insufficient for downstream filtering or interpretation.
When to use
When you have ranked gene statistics and gene set collections, and your analysis requires P-value discrimination below a fixed lower bound (e.g., distinguishing between pathways at p < 1e-10). Particularly relevant when comparing many gene sets where the default eps=1e-10 threshold collapses multiple pathways to identical P-values, or when publication-grade precision is needed for pathway ranking.
When NOT to use
- Input gene sets are very small (< 15 genes) or very large (> 500 genes) — these are filtered by minSize/maxSize and will not yield valid estimates regardless of eps setting.
- Computational resources are severely constrained — eps=0 requires iterative refinement and will be substantially slower than eps=1e-10; use default eps if runtime is critical.
- Gene ranking is unweighted or non-continuous — fgsea requires preranked, continuous statistics; if you have binary phenotype calls, compute Wilcoxon or t-test statistics first.
Inputs
- ranked gene statistics (numeric vector with gene identifiers as names)
- gene set collection (list of character vectors containing gene identifiers)
- minSize and maxSize thresholds (integer, typically 15 and 500)
Outputs
- fgsea results table with columns: pathway, pval, padj, log2err, ES, NES, size
- P-value precision estimates with associated log2err convergence metrics
- comparison table of default vs. adaptive estimates (differences, fold-changes)
How to apply
Load ranked gene statistics (exampleRanks) and pathway definitions (examplePathways). Run fgsea() with eps=0.0 to disable the lower-bound estimate and activate the adaptive multi-level split Monte-Carlo refinement scheme. This parameter switch allows the algorithm to iteratively refine P-value estimates without a fixed minimum threshold. Compare results against a default run (eps=1e-10) by joining on pathway identifier and computing P-value fold-change ratios and difference distributions (log-scale). Validate by confirming pathway rankings remain consistent between runs, no numerical errors occur, and P-values decrease monotonically as precision increases. Record log2err values to assess convergence of the multi-level scheme.
Related tools
- fgsea (Core implementation of adaptive multi-level split Monte-Carlo P-value estimation; provides fgsea() function with eps parameter control and output table generation.) — https://github.com/ctlab/fgsea
- data.table (Efficient joining and comparison of results from eps=1e-10 and eps=0 runs; grouping and aggregation of P-value differences.)
- ggplot2 (Visualization of P-value distribution changes (log-scale) and identification of pathways with greatest precision improvement.)
- R (Runtime environment for fgsea, data.table, and ggplot2 integration.)
Examples
fgseaRes <- fgsea(pathways = examplePathways, stats = exampleRanks, eps = 0.0, minSize = 15, maxSize = 500)
Evaluation signals
- P-values computed with eps=0 are ≤ those with eps=1e-10 for the same pathways (monotonic precision gain).
- log2err values are numeric and finite (not NA or Inf) for eps=0 run, indicating convergence of the multi-level scheme.
- Pathway rankings (order by p-value) remain consistent or improve (no rank inversions) between eps=1e-10 and eps=0 runs.
- No numerical errors or warnings during fgsea execution with eps=0; function completes without NaN or undefined statistics in output table.
- Comparison table shows meaningful P-value differences for pathways originally tied at eps=1e-10 boundary; fold-change ratio distribution is unimodal and right-skewed (most pathways show modest refinement, a few show extreme improvement).
Limitations
- Adaptive refinement is computationally expensive; eps=0 will significantly increase runtime compared to eps=1e-10, especially for large pathway collections or highly significant pathways.
- Extremely low P-values (< 1e-20) become numerically unstable and may be sensitive to random seed and number of Monte-Carlo iterations; reproducibility requires explicit seed control.
- The adaptive scheme assumes sufficiently many gene-set members to enable reliable split-level refinement; very small gene sets (near minSize) may show erratic log2err or unrepresentative P-value estimates.
- No changelog tracking for algorithm evolution; users upgrading fgsea versions must re-run eps=0 analyses as the underlying multi-level scheme may be refined.
Evidence
- [methods] fgsea implements an adaptive multi-level split Monte-Carlo scheme that enables calculation of arbitrarily low GSEA P-values for gene set collections without a fixed lower-bound constraint.: "fgsea implements an adaptive multi-level split Monte-Carlo scheme that enables calculation of arbitrarily low GSEA P-values"
- [readme] P-value estimation is based on an adaptive multi-level split Monte-Carlo scheme allowing arbitrarily low P-values.: "P-value estimation is based on an adaptive multi-level split Monte-Carlo scheme"
- [readme] As you can see
fgsea has a default lower bound eps=1e-10 for estimating P-values. If you need to estimate P-value more accurately, you can set the eps argument to zero in the fgsea function.: "If you need to estimate P-value more accurately, you can set the eps argument to zero"
- [methods] Run fgsea with eps=0 to disable the lower-bound estimate and enable adaptive multi-level split Monte Carlo refinement.: "Run fgsea again with eps=0 to disable the lower-bound estimate and enable adaptive multi-level split Monte Carlo refinement"
- [methods] Validate that eps=0 run completes without numerical errors and produces valid P-value rankings consistent with default run.: "Validate that eps=0 run completes without numerical errors and produces valid P-value rankings consistent with default run"
1---2name: monte-carlo-p-value-estimation-adaptive3description: Use when when you have ranked gene statistics and gene set collections, and your analysis requires P-value discrimination below a fixed lower bound (e.g., distinguishing between pathways at p < 1e-10).4license: CC-BY-4.05---67# Adaptive Multi-Level Split Monte-Carlo P-Value Estimation89## Summary1011A method for computing arbitrarily low gene set enrichment analysis (GSEA) P-values using an adaptive multi-level split Monte-Carlo scheme, eliminating fixed lower-bound constraints by enabling precision refinement through iterative sampling. This skill is essential when gene set rankings must be compared at extreme statistical significance levels or when default P-value resolution (e.g., eps=1e-10) is insufficient for downstream filtering or interpretation.1213## When to use1415When you have ranked gene statistics and gene set collections, and your analysis requires P-value discrimination below a fixed lower bound (e.g., distinguishing between pathways at p < 1e-10). Particularly relevant when comparing many gene sets where the default eps=1e-10 threshold collapses multiple pathways to identical P-values, or when publication-grade precision is needed for pathway ranking.1617## When NOT to use1819- Input gene sets are very small (< 15 genes) or very large (> 500 genes) — these are filtered by minSize/maxSize and will not yield valid estimates regardless of eps setting.20- Computational resources are severely constrained — eps=0 requires iterative refinement and will be substantially slower than eps=1e-10; use default eps if runtime is critical.21- Gene ranking is unweighted or non-continuous — fgsea requires preranked, continuous statistics; if you have binary phenotype calls, compute Wilcoxon or t-test statistics first.2223## Inputs2425- ranked gene statistics (numeric vector with gene identifiers as names)26- gene set collection (list of character vectors containing gene identifiers)27- minSize and maxSize thresholds (integer, typically 15 and 500)2829## Outputs3031- fgsea results table with columns: pathway, pval, padj, log2err, ES, NES, size32- P-value precision estimates with associated log2err convergence metrics33- comparison table of default vs. adaptive estimates (differences, fold-changes)3435## How to apply3637Load ranked gene statistics (exampleRanks) and pathway definitions (examplePathways). Run fgsea() with eps=0.0 to disable the lower-bound estimate and activate the adaptive multi-level split Monte-Carlo refinement scheme. This parameter switch allows the algorithm to iteratively refine P-value estimates without a fixed minimum threshold. Compare results against a default run (eps=1e-10) by joining on pathway identifier and computing P-value fold-change ratios and difference distributions (log-scale). Validate by confirming pathway rankings remain consistent between runs, no numerical errors occur, and P-values decrease monotonically as precision increases. Record log2err values to assess convergence of the multi-level scheme.3839## Related tools4041- **fgsea** (Core implementation of adaptive multi-level split Monte-Carlo P-value estimation; provides fgsea() function with eps parameter control and output table generation.) — https://github.com/ctlab/fgsea42- **data.table** (Efficient joining and comparison of results from eps=1e-10 and eps=0 runs; grouping and aggregation of P-value differences.)43- **ggplot2** (Visualization of P-value distribution changes (log-scale) and identification of pathways with greatest precision improvement.)44- **R** (Runtime environment for fgsea, data.table, and ggplot2 integration.)4546## Examples4748```49fgseaRes <- fgsea(pathways = examplePathways, stats = exampleRanks, eps = 0.0, minSize = 15, maxSize = 500)50```5152## Evaluation signals5354- P-values computed with eps=0 are ≤ those with eps=1e-10 for the same pathways (monotonic precision gain).55- log2err values are numeric and finite (not NA or Inf) for eps=0 run, indicating convergence of the multi-level scheme.56- Pathway rankings (order by p-value) remain consistent or improve (no rank inversions) between eps=1e-10 and eps=0 runs.57- No numerical errors or warnings during fgsea execution with eps=0; function completes without NaN or undefined statistics in output table.58- Comparison table shows meaningful P-value differences for pathways originally tied at eps=1e-10 boundary; fold-change ratio distribution is unimodal and right-skewed (most pathways show modest refinement, a few show extreme improvement).5960## Limitations6162- Adaptive refinement is computationally expensive; eps=0 will significantly increase runtime compared to eps=1e-10, especially for large pathway collections or highly significant pathways.63- Extremely low P-values (< 1e-20) become numerically unstable and may be sensitive to random seed and number of Monte-Carlo iterations; reproducibility requires explicit seed control.64- The adaptive scheme assumes sufficiently many gene-set members to enable reliable split-level refinement; very small gene sets (near minSize) may show erratic log2err or unrepresentative P-value estimates.65- No changelog tracking for algorithm evolution; users upgrading fgsea versions must re-run eps=0 analyses as the underlying multi-level scheme may be refined.6667## Evidence6869- [methods] fgsea implements an adaptive multi-level split Monte-Carlo scheme that enables calculation of arbitrarily low GSEA P-values for gene set collections without a fixed lower-bound constraint.: "fgsea implements an adaptive multi-level split Monte-Carlo scheme that enables calculation of arbitrarily low GSEA P-values"70- [readme] P-value estimation is based on an adaptive multi-level split Monte-Carlo scheme allowing arbitrarily low P-values.: "P-value estimation is based on an adaptive multi-level split Monte-Carlo scheme"71- [readme] As you can see `fgsea` has a default lower bound `eps=1e-10` for estimating P-values. If you need to estimate P-value more accurately, you can set the `eps` argument to zero in the `fgsea` function.: "If you need to estimate P-value more accurately, you can set the `eps` argument to zero"72- [methods] Run fgsea with eps=0 to disable the lower-bound estimate and enable adaptive multi-level split Monte Carlo refinement.: "Run fgsea again with eps=0 to disable the lower-bound estimate and enable adaptive multi-level split Monte Carlo refinement"73- [methods] Validate that eps=0 run completes without numerical errors and produces valid P-value rankings consistent with default run.: "Validate that eps=0 run completes without numerical errors and produces valid P-value rankings consistent with default run"
