Spearman Correlation Computation Microbiome

Spearman Correlation Computation for Microbiome-Metabolome Prediction

Summary

Compute Spearman correlation coefficients (SCCs) between predicted and observed metabolite abundances to quantify prediction accuracy in microbiome-metabolome models. This metric is used to identify well-predicted metabolites and compare the predictive performance of competing methods across paired microbiome and metabolomic datasets.

When to use

Apply this skill when you have cross-validated predictions of metabolite abundances from a microbiome-metabolome model and need to: (1) measure predictive accuracy at the individual metabolite level; (2) aggregate performance across all metabolites to report mean SCC; (3) generate a distribution of SCCs under a null model (shuffled data) to establish a significance threshold; or (4) compare the performance of multiple competing prediction methods (e.g., MiMeNet vs. MelonnPan) on the same dataset.

When NOT to use

• When the metabolite or microbiome data has not been normalized (e.g., still in raw counts); apply centered log-ratio or relative abundance transformation first.
• When comparing predictions across datasets with very different distributions (e.g., IBD cohort vs. soil samples) without first checking that the background distribution threshold is computed independently for each dataset.
• When metabolite abundances are binary or categorical rather than continuous; Spearman correlation assumes ranked or continuous data.
• When sample sizes are very small (<20 samples per fold), as SCC estimates become unstable and the background distribution may not be robust.

Inputs

• Predicted metabolite abundance matrix (samples × metabolites, from cross-validated model predictions)
• Observed metabolite abundance matrix (samples × metabolites, ground truth from test set)
• Microbiome abundance matrix (samples × microbial features, for null model shuffling)
• Metabolomic abundance matrix (samples × metabolites, for null model shuffling)

Outputs

• Per-metabolite Spearman correlation coefficients (vector or table)
• Mean SCC across all metabolites (scalar summary statistic)
• Background distribution of SCCs from shuffled null model (vector of length = num_background × num_cv)
• 95th percentile SCC threshold for well-predicted metabolites (scalar)
• Count of well-predicted metabolites (scalar)
• Metabolite prediction quality table (metabolite ID, SCC, significance flag)

How to apply

For each metabolite in the test set, compute the Spearman rank correlation coefficient between its predicted abundance (from cross-validated model output) and its observed abundance (ground truth). Aggregate these per-metabolite SCCs by taking the mean across all metabolites to obtain a dataset-level performance summary. To establish a significance cutoff, generate a background distribution by shuffling the microbiome and metabolomic samples independently, performing the same cross-validated prediction and SCC computation on the shuffled data across multiple iterations (e.g., 100 times), then defining the 95th percentile of this null distribution as the threshold for 'well-predicted' metabolites. Metabolites with SCC ≥ this threshold are retained for downstream analysis (network analysis, module detection, biomarker discovery); those below are considered poorly predicted and may reflect metabolites not mechanistically linked to the microbiome or features with low signal-to-noise. Report both the count of well-predicted metabolites and the mean SCC across all metabolites to fully characterize model performance.

Related tools

• MiMeNet (Neural network model that generates predicted metabolite abundances; SCC is used to evaluate its cross-validated predictions) — https://github.com/YDaiLab/MiMeNet
• MelonnPan (Elastic Net-based baseline method whose predictions are compared to MiMeNet using SCC as the performance metric) — https://github.com/biobakery/melonnpan
• scikit-learn (Python library providing scipy.stats.spearmanr for computing Spearman correlation coefficients)
• scipy (Python library with scipy.stats.spearmanr function for rank correlation computation)

Examples

from scipy.stats import spearmanr; import numpy as np; scc_per_metabolite = [spearmanr(predicted[:, i], observed[:, i])[0] for i in range(predicted.shape[1])]; mean_scc = np.mean(scc_per_metabolite); well_predicted = np.sum(np.array(scc_per_metabolite) >= 0.136)

Evaluation signals

• Mean SCC should increase relative to the null model (shuffled background); if mean SCC ≈ mean of background distribution, the model has no predictive power.
• The count of well-predicted metabolites (SCC ≥ 95th percentile threshold) should be substantially greater than the count expected by random chance (e.g., ~5% of total metabolites if threshold is set at 95th percentile of a uniform null distribution).
• The 95th percentile threshold computed from the background distribution should be notably lower than the observed SCCs for well-predicted metabolites; thresholds for IBD (PRISM) typically range 0.13–0.14, while well-predicted metabolites often have SCC > 0.3.
• When comparing two methods (e.g., MiMeNet vs. MelonnPan), the method with higher mean SCC and larger count of well-predicted metabolites should be reproducible across multiple cross-validation runs; report 95% confidence intervals or standard deviations.
• SCC distributions should be approximately normal or at least unimodal when plotted; bimodal or heavily skewed distributions may indicate data quality issues or the presence of two populations of metabolites with very different predictability.

Limitations

• Not all metabolites may be mechanistically associated with the microbiome, so some metabolites will have lower prediction correlations by design; this results in an overall lower mean SCC across all metabolites and should not be interpreted as model failure.
• The 95th percentile threshold is dataset-specific and must be recomputed for each new dataset using its own shuffled background distribution; thresholds computed from one dataset (e.g., IBD PRISM) cannot be directly applied to another (e.g., soil biocrust).
• SCC is sensitive to outliers in the predicted or observed abundance values; data with extreme values or measurement errors can inflate or deflate correlations; centered log-ratio or other compositional transformations should be applied before computing SCC.
• Small sample sizes (N < 20) lead to unstable SCC estimates; confidence intervals widen and rank-based correlation becomes less reliable.
• The computational cost of generating the background distribution (multiple iterations of shuffling + cross-validation) scales linearly with the number of samples and metabolites; for very large datasets, subsampling or parallel computation may be needed.

Evidence

• [results] the predictive performance of a model is measured by the average Spearman correlation coefficients (SCCs) between the predicted and the observed abundance of the metabolites: "the predictive performance of a model is measured by the average Spearman correlation coefficients (SCCs) between the predicted and the observed abundance of the metabolites"
• [results] MiMeNet then generates a background distribution of SCCs through multiple iterations of shuffling the dataset and performing a cross-validated evaluation on the shuffled set: "MiMeNet then generates a background distribution of SCCs through multiple iterations of shuffling the dataset and performing a cross-validated evaluation on the shuffled set"
• [methods] We then defined a metabolite as well-predicted if its SCC is above the 95th percentile of the background correlations: "We then defined a metabolite as well-predicted if its SCC is above the 95th percentile of the background correlations"
• [results] the cutoffs for SCCs between the predicted and observed abundances of metabolites were found to be 0.136, 0.129, and 0.410 for the IBD (PRISM), cystic fibrosis, and soil datasets, respectively: "the cutoffs for SCCs between the predicted and observed abundances of metabolites were found to be 0.136, 0.129, and 0.410 for the IBD (PRISM), cystic fibrosis, and soil datasets, respectively"
• [other] MiMeNet identified 351 well-predicted metabolites from 8848 total metabolites, whereas MelonnPan identified 198 well-predicted metabolites using the same correlation cutoff of 0.3: "MiMeNet identified 351 well-predicted metabolites from 8848 total metabolites, whereas MelonnPan identified 198 well-predicted metabolites using the same correlation cutoff of 0.3"
• [discussion] since not all metabolites may be associated with microbes, some metabolites will have lower prediction correlations, which resulted in an overall lower mean correlation across all metabolites: "since not all metabolites may be associated with microbes, some metabolites will have lower prediction correlations, which resulted in an overall lower mean correlation across all metabolites"