feature-attribution-score-calculation
Summary
Calculate neural network feature attribution scores (weights) between microbes and metabolites using Olden's method to quantify the strength and direction of learned interactions. These scores reveal which microbes most strongly influence metabolite predictions and form the basis for clustering microbes and metabolites into functional modules.
When to use
After training a multi-layer perceptron neural network to predict metabolomic features from microbiome features using cross-validated evaluation, extract attribution scores to interpret which microbe-metabolite pairs have learned predictive relationships. Apply this when you have trained network weights from models that achieved non-random predictive performance (SCC above background threshold) and need to move from black-box predictions to interpretable interaction networks.
When NOT to use
- Models with mean SCC at or below background noise levels (below 95th percentile threshold) — attribution scores will not distinguish true interactions from artifacts.
- Linear regression models (e.g., Elastic Net, MelonnPan) — Olden's method is specific to multi-layer neural networks and requires layered weight matrices.
- When mechanistic or catalytic knowledge is essential and data-driven attribution alone is insufficient — the article notes MiMeNet analysis is 'data-driven without incorporating mechanistic knowledge'.
Inputs
- Trained neural network weight matrices from cross-validated MiMeNet models (input-to-hidden and hidden-to-output layer weights)
- Microbe feature identifiers (from the training microbiome table)
- Metabolite feature identifiers (from the training metabolome table, filtered to well-predicted metabolites only)
- Background distribution of attribution scores from shuffled data (97.5th percentile threshold)
Outputs
- Feature attribution score matrix (microbes × well-predicted metabolites) with normalized values in [−1, 1]
- Filtered attribution matrix containing only significant interactions (absolute value ≥ normalized threshold)
- List of microbe-metabolite interaction pairs suitable for downstream consensus clustering and module construction
How to apply
Using the learned network weights from cross-validated training iterations, construct a microbe-metabolite feature attribution score matrix by applying Olden's method, which multiplies input-to-hidden layer weights by hidden-to-output layer weights. Filter the attribution matrix to retain only significant interactions at or above the 97.5th percentile threshold of the background distribution (generated by the same method applied to shuffled negative control data). Normalize each score matrix by dividing all values by the background threshold and clip the resulting normalized scores to the range [−1, 1] to enable comparison across datasets and conditions. This produces a bounded interaction matrix where values closer to ±1 indicate strong microbe-metabolite associations and values near 0 indicate weak or noise-driven interactions.
Related tools
- MiMeNet (Neural network training and cross-validation framework that produces the learned weights used as input to Olden's method for attribution score extraction) — https://github.com/YDaiLab/MiMeNet
- scikit-learn (Provides matrix operations and utilities for constructing and manipulating the attribution score matrices)
- NumPy (Enables efficient matrix multiplication and element-wise operations for calculating Olden's attribution scores)
Evaluation signals
- Attribution matrix has the correct dimensions (number of microbes × number of well-predicted metabolites) and contains numeric values.
- After normalization and clipping, all values fall within [−1, 1] with no NaN or infinite values.
- Significant interactions (filtered at 97.5th percentile) number fewer than total possible pairs, confirming that the threshold is not degenerate; for IBD (PRISM) dataset, 163 microbes were retained with significant scores.
- Distribution of significant attribution scores shows separation from the background distribution used to compute the threshold (97.5th percentile of shuffled data).
- Clustering the filtered attribution matrix produces stable, interpretable microbial and metabolite modules with consistent membership across consensus iterations.
Limitations
- Olden's method is specific to multi-layer perceptrons with clear input-to-hidden and hidden-to-output weight layers; does not apply to other neural network architectures (e.g., convolutional networks, recurrent networks) or non-neural methods.
- Attribution scores quantify learned weights but do not necessarily reflect causal or mechanistic relationships; high scores may reflect correlations in the training data rather than true microbe-metabolite interactions.
- Background distribution generation requires computationally expensive shuffling and cross-validation iterations (article used 100 shuffled models per condition), making the method sensitive to the number of background iterations performed.
- Not all metabolites are associated with microbes, so unassociated metabolites will have lower or undetectable attribution scores, limiting the scope of the interaction network.
Evidence
- [methods] Extract feature attribution score matrices using Olden's method for significant microbe-metabolite interactions (97.5th percentile threshold).: "Extract feature attribution score matrices using Olden's method for significant microbe-metabolite interactions (97.5th percentile threshold)"
- [methods] A threshold was set at the 97.5 percentile. Any feature attribution score in the observed dataset with an absolute value above the threshold was considered significant.: "a threshold was set at the 97.5 percentile. Any feature attribution score in the observed dataset with an absolute value above the threshold was considered significant"
- [methods] For each condition, normalize attribution scores by dividing by background threshold and clipping to [−1, 1]; construct microbial and metabolite modules via consensus clustering.: "normalize attribution scores by dividing by background threshold and clipping to [−1, 1]; construct microbial and metabolite modules via consensus clustering"
- [results] We identified 163 microbes that had at least one significant attribution score with a well-predicted metabolite.: "We identified 163 microbes that had at least one significant attribution score with a well-predicted metabolite"
- [results] using the learned network weights obtained from cross-validation training, MiMeNet constructs a score matrix of microbe-metabolite feature attributions between the microbes and well-predicted: "using the learned network weights obtained from cross-validation training, MiMeNet constructs a score matrix of microbe-metabolite feature attributions"
1---2name: feature-attribution-score-calculation3description: Use when after training a multi-layer perceptron neural network to predict metabolomic features from microbiome features using cross-validated evaluation, extract attribution scores to interpret which microbe-metabolite pairs have learned predictive relationships.4license: CC-BY-4.05---67# feature-attribution-score-calculation89## Summary1011Calculate neural network feature attribution scores (weights) between microbes and metabolites using Olden's method to quantify the strength and direction of learned interactions. These scores reveal which microbes most strongly influence metabolite predictions and form the basis for clustering microbes and metabolites into functional modules.1213## When to use1415After training a multi-layer perceptron neural network to predict metabolomic features from microbiome features using cross-validated evaluation, extract attribution scores to interpret which microbe-metabolite pairs have learned predictive relationships. Apply this when you have trained network weights from models that achieved non-random predictive performance (SCC above background threshold) and need to move from black-box predictions to interpretable interaction networks.1617## When NOT to use1819- Models with mean SCC at or below background noise levels (below 95th percentile threshold) — attribution scores will not distinguish true interactions from artifacts.20- Linear regression models (e.g., Elastic Net, MelonnPan) — Olden's method is specific to multi-layer neural networks and requires layered weight matrices.21- When mechanistic or catalytic knowledge is essential and data-driven attribution alone is insufficient — the article notes MiMeNet analysis is 'data-driven without incorporating mechanistic knowledge'.2223## Inputs2425- Trained neural network weight matrices from cross-validated MiMeNet models (input-to-hidden and hidden-to-output layer weights)26- Microbe feature identifiers (from the training microbiome table)27- Metabolite feature identifiers (from the training metabolome table, filtered to well-predicted metabolites only)28- Background distribution of attribution scores from shuffled data (97.5th percentile threshold)2930## Outputs3132- Feature attribution score matrix (microbes × well-predicted metabolites) with normalized values in [−1, 1]33- Filtered attribution matrix containing only significant interactions (absolute value ≥ normalized threshold)34- List of microbe-metabolite interaction pairs suitable for downstream consensus clustering and module construction3536## How to apply3738Using the learned network weights from cross-validated training iterations, construct a microbe-metabolite feature attribution score matrix by applying Olden's method, which multiplies input-to-hidden layer weights by hidden-to-output layer weights. Filter the attribution matrix to retain only significant interactions at or above the 97.5th percentile threshold of the background distribution (generated by the same method applied to shuffled negative control data). Normalize each score matrix by dividing all values by the background threshold and clip the resulting normalized scores to the range [−1, 1] to enable comparison across datasets and conditions. This produces a bounded interaction matrix where values closer to ±1 indicate strong microbe-metabolite associations and values near 0 indicate weak or noise-driven interactions.3940## Related tools4142- **MiMeNet** (Neural network training and cross-validation framework that produces the learned weights used as input to Olden's method for attribution score extraction) — https://github.com/YDaiLab/MiMeNet43- **scikit-learn** (Provides matrix operations and utilities for constructing and manipulating the attribution score matrices)44- **NumPy** (Enables efficient matrix multiplication and element-wise operations for calculating Olden's attribution scores)4546## Evaluation signals4748- Attribution matrix has the correct dimensions (number of microbes × number of well-predicted metabolites) and contains numeric values.49- After normalization and clipping, all values fall within [−1, 1] with no NaN or infinite values.50- Significant interactions (filtered at 97.5th percentile) number fewer than total possible pairs, confirming that the threshold is not degenerate; for IBD (PRISM) dataset, 163 microbes were retained with significant scores.51- Distribution of significant attribution scores shows separation from the background distribution used to compute the threshold (97.5th percentile of shuffled data).52- Clustering the filtered attribution matrix produces stable, interpretable microbial and metabolite modules with consistent membership across consensus iterations.5354## Limitations5556- Olden's method is specific to multi-layer perceptrons with clear input-to-hidden and hidden-to-output weight layers; does not apply to other neural network architectures (e.g., convolutional networks, recurrent networks) or non-neural methods.57- Attribution scores quantify learned weights but do not necessarily reflect causal or mechanistic relationships; high scores may reflect correlations in the training data rather than true microbe-metabolite interactions.58- Background distribution generation requires computationally expensive shuffling and cross-validation iterations (article used 100 shuffled models per condition), making the method sensitive to the number of background iterations performed.59- Not all metabolites are associated with microbes, so unassociated metabolites will have lower or undetectable attribution scores, limiting the scope of the interaction network.6061## Evidence6263- [methods] Extract feature attribution score matrices using Olden's method for significant microbe-metabolite interactions (97.5th percentile threshold).: "Extract feature attribution score matrices using Olden's method for significant microbe-metabolite interactions (97.5th percentile threshold)"64- [methods] A threshold was set at the 97.5 percentile. Any feature attribution score in the observed dataset with an absolute value above the threshold was considered significant.: "a threshold was set at the 97.5 percentile. Any feature attribution score in the observed dataset with an absolute value above the threshold was considered significant"65- [methods] For each condition, normalize attribution scores by dividing by background threshold and clipping to [−1, 1]; construct microbial and metabolite modules via consensus clustering.: "normalize attribution scores by dividing by background threshold and clipping to [−1, 1]; construct microbial and metabolite modules via consensus clustering"66- [results] We identified 163 microbes that had at least one significant attribution score with a well-predicted metabolite.: "We identified 163 microbes that had at least one significant attribution score with a well-predicted metabolite"67- [results] using the learned network weights obtained from cross-validation training, MiMeNet constructs a score matrix of microbe-metabolite feature attributions between the microbes and well-predicted: "using the learned network weights obtained from cross-validation training, MiMeNet constructs a score matrix of microbe-metabolite feature attributions"
HolobiomicsLab