metabolite-prediction-correlation-analysis
Summary
Quantifies prediction accuracy of machine learning models for metabolite abundance by computing Spearman correlation coefficients (SCC) between predicted and observed metabolite abundances, then stratifies results against an empirical background distribution generated from shuffled data to identify well-predicted metabolites.
When to use
Apply this skill after training a neural network or regression model to predict metabolomic profiles from microbiome data. Use it to: (1) measure per-metabolite prediction quality across cross-validation folds, (2) distinguish signal from noise by comparing observed correlations to a permutation baseline, and (3) count metabolites above a statistical threshold to enable downstream functional module discovery or biomarker validation.
When NOT to use
- Input metabolomic or microbiome data is not pre-filtered to ≥10% sample presence; apply feature filtering before correlation analysis.
- Model predictions have not been generated on held-out test data; use cross-validation or external validation sets, not training set predictions.
- Background distribution has not been generated; correlation thresholds cannot be interpreted without the empirical null from shuffled data.
Inputs
- Microbiome abundance matrix (samples × microbial features, CLR- or relative abundance–normalized, pre-filtered to ≥10% sample presence)
- Metabolomic abundance matrix (samples × metabolite features, CLR- or relative abundance–normalized, pre-filtered to ≥10% sample presence)
- Trained neural network or regression model with learned weights
- Predicted metabolite abundance matrix (samples × metabolites) from model inference on test set
- Observed metabolite abundance matrix (samples × metabolites) for test set
Outputs
- Per-metabolite Spearman correlation coefficient (SCC) array
- Mean SCC across all metabolites
- Count of well-predicted metabolites (SCC > 95th percentile of background)
- Empirical background SCC distribution (from 10–100 shuffled cross-validation iterations)
- Well-predicted metabolite list with SCC values
- Microbe-metabolite feature attribution score matrix (optional, for module discovery)
How to apply
First, train your predictive model (e.g., MLPNN, Elastic Net, Random Forest) on microbiome features and compute predictions on held-out test folds. For each metabolite, calculate the Spearman correlation coefficient (SCC) between its predicted and observed abundances across all samples or folds. In parallel, generate an empirical background distribution by shuffling the microbiome and metabolomic data independently, then re-running the full cross-validation pipeline 10–100 times on shuffled pairs and computing SCCs. Define the significance threshold as the 95th percentile of the background SCC distribution. Classify metabolites with observed SCC > 95th percentile threshold as well-predicted. Report: mean SCC across all metabolites, count of well-predicted metabolites, and optionally construct feature attribution matrices from model weights to enable biclustering into functional modules.
Related tools
- MiMeNet (Integrative multilayer perceptron neural network that predicts metabolomic profiles from microbiome data and computes correlation-based evaluation metrics) — https://github.com/YDaiLab/MiMeNet
- MelonnPan (Baseline linear regression model using Elastic Net that predicts metabolite abundances for benchmarking against neural network approaches) — https://github.com/biobakery/melonnpan
- Scikit-learn (Provides cross-validation utilities, neural network implementations, and general regression models (Random Forest, Elastic Net) used for prediction)
- TensorFlow (Deep learning framework used to train multilayer perceptron neural networks with dropout and L2 regularization)
- SciPy (Computes Spearman correlation coefficients between predicted and observed metabolite abundances)
Examples
python MiMeNet_train.py -micro data/IBD/microbiome_PRISM.csv -metab data/IBD/metabolome_PRISM.csv -external_micro data/IBD/microbiome_external.csv -external_metab data/IBD/metabolome_external.csv -micro_norm None -metab_norm CLR -num_run_cv 10 -output IBD
Evaluation signals
- Mean SCC increases relative to baseline shuffled correlations (e.g., from 0.108 to 0.309 for well-performing models on IBD data)
- Count of well-predicted metabolites (SCC > 95th percentile) is substantially higher than expected by chance; compare to number detected in shuffled background
- Background SCC distribution is centered near zero and shows expected tail behavior; 95th percentile threshold should be noticeably above mean background
- External validation: when trained model is evaluated on held-out external cohort, mean SCC and well-predicted metabolite count remain consistent with internal cross-validation
- Feature attribution scores derived from model weights exhibit interpretable clustering (via biclustering) into functional modules consistent with prior biological knowledge
Limitations
- Not all metabolites may be associated with microbes; some metabolites will have low prediction correlations regardless of model quality, reducing overall mean SCC.
- Analysis is data-driven without incorporating mechanistic knowledge; correlations alone do not prove causal microbe-metabolite interactions.
- Hyperparameter tuning strategy (single vs. per-partition) can affect metabolite counts; shared hyperparameters may miss dataset-specific optimization but improve generalization.
- Background distribution threshold (95th percentile) is empirical; choice may vary across datasets with different noise structures or biological associations.
- Longitudinal or hierarchical study designs (e.g., time-series soil data) may exhibit higher correlation thresholds due to repeated measurements within samples, complicating cross-study comparisons.
Evidence
- [methods] the predictive performance of a model is measured by the average Spearman correlation coefficients (SCCs) between the predicted and the observed abundances: "the predictive performance of a model is measured by the average Spearman correlation coefficients (SCCs) between the predicted and the observed abundances"
- [methods] MiMeNet then generates a background distribution of SCCs through multiple iterations of shuffling the dataset and performing a cross-validation on the shuffled set: "MiMeNet then generates a background distribution of SCCs through multiple iterations of shuffling the dataset and performing a cross-validation on the shuffled set"
- [methods] We then defined a metabolite as well-predicted if its SCC is above the 95th percentile of the background correlations: "We then defined a metabolite as well-predicted if its SCC is above the 95th percentile of the background correlations"
- [abstract] MiMeNet more accurately predicts metabolite abundances (mean Spearman correlation coefficients increase from 0.108 to 0.309, 0.276 to 0.457, and -0.272 to 0.264): "MiMeNet more accurately predicts metabolite abundances (mean Spearman correlation coefficients increase from 0.108 to 0.309, 0.276 to 0.457, and -0.272 to 0.264)"
- [abstract] identifies more well-predicted metabolites (increase in the number of well-predicted metabolites from 198 to 366, 104 to 143, and 4 to 29) compared to state-of-art linear models: "identifies more well-predicted metabolites (increase in the number of well-predicted metabolites from 198 to 366, 104 to 143, and 4 to 29) compared to state-of-art linear models"
- [discussion] since not all metabolites may be associated with microbes, some metabolites will have lower prediction correlations, which resulted in an overall lower mean correlation across all metabolites: "since not all metabolites may be associated with microbes, some metabolites will have lower prediction correlations, which resulted in an overall lower mean correlation across all metabolites"
- [methods] Any input or output feature that is present in less than 10% of samples was removed: "Any input or output feature that is present in less than 10% of samples was removed"
- [results] MiMeNet constructs a score matrix of microbe-metabolite feature attributions between the microbes and well-predicted metabolites: "MiMeNet constructs a score matrix of microbe-metabolite feature attributions between the microbes and well-predicted metabolites"
1---2name: metabolite-prediction-correlation-analysis3description: Use when after training a neural network or regression model to predict metabolomic profiles from microbiome data.4license: CC-BY-4.05---67# metabolite-prediction-correlation-analysis89## Summary1011Quantifies prediction accuracy of machine learning models for metabolite abundance by computing Spearman correlation coefficients (SCC) between predicted and observed metabolite abundances, then stratifies results against an empirical background distribution generated from shuffled data to identify well-predicted metabolites.1213## When to use1415Apply this skill after training a neural network or regression model to predict metabolomic profiles from microbiome data. Use it to: (1) measure per-metabolite prediction quality across cross-validation folds, (2) distinguish signal from noise by comparing observed correlations to a permutation baseline, and (3) count metabolites above a statistical threshold to enable downstream functional module discovery or biomarker validation.1617## When NOT to use1819- Input metabolomic or microbiome data is not pre-filtered to ≥10% sample presence; apply feature filtering before correlation analysis.20- Model predictions have not been generated on held-out test data; use cross-validation or external validation sets, not training set predictions.21- Background distribution has not been generated; correlation thresholds cannot be interpreted without the empirical null from shuffled data.2223## Inputs2425- Microbiome abundance matrix (samples × microbial features, CLR- or relative abundance–normalized, pre-filtered to ≥10% sample presence)26- Metabolomic abundance matrix (samples × metabolite features, CLR- or relative abundance–normalized, pre-filtered to ≥10% sample presence)27- Trained neural network or regression model with learned weights28- Predicted metabolite abundance matrix (samples × metabolites) from model inference on test set29- Observed metabolite abundance matrix (samples × metabolites) for test set3031## Outputs3233- Per-metabolite Spearman correlation coefficient (SCC) array34- Mean SCC across all metabolites35- Count of well-predicted metabolites (SCC > 95th percentile of background)36- Empirical background SCC distribution (from 10–100 shuffled cross-validation iterations)37- Well-predicted metabolite list with SCC values38- Microbe-metabolite feature attribution score matrix (optional, for module discovery)3940## How to apply4142First, train your predictive model (e.g., MLPNN, Elastic Net, Random Forest) on microbiome features and compute predictions on held-out test folds. For each metabolite, calculate the Spearman correlation coefficient (SCC) between its predicted and observed abundances across all samples or folds. In parallel, generate an empirical background distribution by shuffling the microbiome and metabolomic data independently, then re-running the full cross-validation pipeline 10–100 times on shuffled pairs and computing SCCs. Define the significance threshold as the 95th percentile of the background SCC distribution. Classify metabolites with observed SCC > 95th percentile threshold as well-predicted. Report: mean SCC across all metabolites, count of well-predicted metabolites, and optionally construct feature attribution matrices from model weights to enable biclustering into functional modules.4344## Related tools4546- **MiMeNet** (Integrative multilayer perceptron neural network that predicts metabolomic profiles from microbiome data and computes correlation-based evaluation metrics) — https://github.com/YDaiLab/MiMeNet47- **MelonnPan** (Baseline linear regression model using Elastic Net that predicts metabolite abundances for benchmarking against neural network approaches) — https://github.com/biobakery/melonnpan48- **Scikit-learn** (Provides cross-validation utilities, neural network implementations, and general regression models (Random Forest, Elastic Net) used for prediction)49- **TensorFlow** (Deep learning framework used to train multilayer perceptron neural networks with dropout and L2 regularization)50- **SciPy** (Computes Spearman correlation coefficients between predicted and observed metabolite abundances)5152## Examples5354```55python MiMeNet_train.py -micro data/IBD/microbiome_PRISM.csv -metab data/IBD/metabolome_PRISM.csv -external_micro data/IBD/microbiome_external.csv -external_metab data/IBD/metabolome_external.csv -micro_norm None -metab_norm CLR -num_run_cv 10 -output IBD56```5758## Evaluation signals5960- Mean SCC increases relative to baseline shuffled correlations (e.g., from 0.108 to 0.309 for well-performing models on IBD data)61- Count of well-predicted metabolites (SCC > 95th percentile) is substantially higher than expected by chance; compare to number detected in shuffled background62- Background SCC distribution is centered near zero and shows expected tail behavior; 95th percentile threshold should be noticeably above mean background63- External validation: when trained model is evaluated on held-out external cohort, mean SCC and well-predicted metabolite count remain consistent with internal cross-validation64- Feature attribution scores derived from model weights exhibit interpretable clustering (via biclustering) into functional modules consistent with prior biological knowledge6566## Limitations6768- Not all metabolites may be associated with microbes; some metabolites will have low prediction correlations regardless of model quality, reducing overall mean SCC.69- Analysis is data-driven without incorporating mechanistic knowledge; correlations alone do not prove causal microbe-metabolite interactions.70- Hyperparameter tuning strategy (single vs. per-partition) can affect metabolite counts; shared hyperparameters may miss dataset-specific optimization but improve generalization.71- Background distribution threshold (95th percentile) is empirical; choice may vary across datasets with different noise structures or biological associations.72- Longitudinal or hierarchical study designs (e.g., time-series soil data) may exhibit higher correlation thresholds due to repeated measurements within samples, complicating cross-study comparisons.7374## Evidence7576- [methods] the predictive performance of a model is measured by the average Spearman correlation coefficients (SCCs) between the predicted and the observed abundances: "the predictive performance of a model is measured by the average Spearman correlation coefficients (SCCs) between the predicted and the observed abundances"77- [methods] MiMeNet then generates a background distribution of SCCs through multiple iterations of shuffling the dataset and performing a cross-validation on the shuffled set: "MiMeNet then generates a background distribution of SCCs through multiple iterations of shuffling the dataset and performing a cross-validation on the shuffled set"78- [methods] We then defined a metabolite as well-predicted if its SCC is above the 95th percentile of the background correlations: "We then defined a metabolite as well-predicted if its SCC is above the 95th percentile of the background correlations"79- [abstract] MiMeNet more accurately predicts metabolite abundances (mean Spearman correlation coefficients increase from 0.108 to 0.309, 0.276 to 0.457, and -0.272 to 0.264): "MiMeNet more accurately predicts metabolite abundances (mean Spearman correlation coefficients increase from 0.108 to 0.309, 0.276 to 0.457, and -0.272 to 0.264)"80- [abstract] identifies more well-predicted metabolites (increase in the number of well-predicted metabolites from 198 to 366, 104 to 143, and 4 to 29) compared to state-of-art linear models: "identifies more well-predicted metabolites (increase in the number of well-predicted metabolites from 198 to 366, 104 to 143, and 4 to 29) compared to state-of-art linear models"81- [discussion] since not all metabolites may be associated with microbes, some metabolites will have lower prediction correlations, which resulted in an overall lower mean correlation across all metabolites: "since not all metabolites may be associated with microbes, some metabolites will have lower prediction correlations, which resulted in an overall lower mean correlation across all metabolites"82- [methods] Any input or output feature that is present in less than 10% of samples was removed: "Any input or output feature that is present in less than 10% of samples was removed"83- [results] MiMeNet constructs a score matrix of microbe-metabolite feature attributions between the microbes and well-predicted metabolites: "MiMeNet constructs a score matrix of microbe-metabolite feature attributions between the microbes and well-predicted metabolites"
