metabolomics-peak-data-normalization-and-handling
Summary
Standardize metabolomics peak intensity matrices through log-2 transformation and z-score normalization to enable robust pathway activity scoring. This preprocessing step is essential for metabolomics data because it addresses missing values, scales intensities across samples, and prepares data for decomposition-based pathway analysis methods like PLAGE.
When to use
When you have raw peak intensity matrices from metabolomics LC-MS/MS experiments with zero values (missing peaks or undetected compounds) and need to prepare data for pathway-level analysis using PLAGE, ORA, or GSEA. Apply this skill before computing pathway activity scores, especially if your data exhibits high noise or missing peak rates (5–20%), which are prevalent in metabolomics and would otherwise bias downstream comparisons.
When NOT to use
- Data is already quantile-normalized or batch-corrected by external tools (e.g., limma, ComBat); re-normalizing may introduce bias.
- Peak intensities are already log-transformed and z-scored by the mass spectrometry processing pipeline.
- The experimental design has nested or paired factors where group-level imputation is inappropriate; use custom imputation strategies instead.
Inputs
- Peak intensity CSV matrix (rows = peaks, columns = samples, first column = peak ID)
- Annotation CSV matrix (peak ID, entity_id [KEGG/ChEBI compound ID])
- Experimental design specification (sample-to-group assignments)
Outputs
- Normalized log-2 z-score intensity matrix (zero mean, unit variance across samples)
- Imputed missing value indicator (optional, for tracking replaced zeros)
- Preprocessed data structure ready for pathway decomposition (PLAGE/ORA/GSEA input)
How to apply
Load the peak intensity matrix (rows = peaks with column 1 = peak ID, remaining columns = sample intensities) and annotation matrix (peak ID → KEGG/ChEBI compound ID). Apply data imputation: if all samples in a single experimental factor have zero intensity, replace with a user-defined minimum (default 5000); if only some samples in a factor are zero, replace with the mean of non-zero samples in that factor. This preserves group structure while avoiding log(0). Then apply log-2 transformation to the entire imputed matrix and z-score standardize across all samples (zero mean, unit variance) using scipy preprocessing. This normalization stabilizes variance across intensity ranges and makes the data amenable to SVD-based decomposition methods. Validate that the output matrix has no NaN or Inf values and that column means are approximately zero.
Related tools
- PALS (Pathway Activity Level Scoring) (Executes normalization as a preprocessing step before PLAGE decomposition or ORA/GSEA ranking) — https://github.com/glasgowcompbio/PALS
- Scipy preprocessing module (Provides StandardScaler for z-score normalization and standardization functions)
Examples
python pals/run.py PLAGE notebooks/test_data/HAT/int_df.csv notebooks/test_data/HAT/annotation_df.csv test_output.csv --db PiMP_KEGG --comparisons Stage_1/Control Stage_2/Control --min_replace 5000
Evaluation signals
- Output matrix has zero column mean (±1e-10 tolerance) and unit column variance for all samples.
- No NaN, Inf, or -Inf values present in the normalized matrix after transformation.
- Imputed zeros are documented; verify that group-wise means of non-imputed values are preserved before and after log transformation.
- Comparison of pathway activity scores (Spearman correlation) between baseline and 5% Gaussian noise-perturbed data shows ≥0.85 correlation for PLAGE (metric of normalization robustness).
- Intensity ranges span multiple orders of magnitude in the log-2 domain (typically 0–20 range for LC-MS intensities) without extreme outliers (>3 σ from mean after z-scoring).
Limitations
- Zero replacement strategy assumes missing peaks are missing at random (MAR) within experimental factors; non-random missingness may bias results.
- Group-level imputation requires correct experimental design specification; misassigned samples will propagate incorrect mean values.
- Log-2 transformation assumes all intensities are positive; negative or zero imputation values will produce undefined logarithms.
- Z-score normalization assumes roughly symmetric or light-tailed intensity distributions; extreme skewness or multimodal distributions may require alternative transformations (e.g., rank-based).
- The method is sensitive to outlier samples; anomalous samples with extreme intensities will inflate standard deviations and compress other samples.
Evidence
- [readme] Data imputation is performed to the intensity matrix when it is loaded: if all of the samples in a single experimental factor have intensities of zero these are replaced by the minimum intensity value (which can be set by the user); and if only some of the sample values in a factor are zero then these are replaced by the mean value of the non-zero samples in that factor.: "Data imputation is performed to the intensity matrix when it is loaded: if all of the samples in a single experimental factor have intensities of zero these are replaced by the minimum intensity"
- [readme] The data is subsequently transformed to log-2 base and standardised using the preprocessing module in Scipy such that the intensity matrix has a zero mean and unit variance across the samples.: "The data is subsequently transformed to log-2 base and standardised using the preprocessing module in Scipy such that the intensity matrix has a zero mean and unit variance across the samples."
- [intro] PALS results demonstrate greater robustness to noise and missing peaks compared to ORA and GSEA, which is particularly important for metabolomics peak data where noise and missing peaks are prevalent.: "PALS results demonstrate greater robustness to noise and missing peaks compared to ORA and GSEA, which is particularly important for metabolomics peak data where noise and missing peaks are prevalent."
- [readme] --min_replace: The minimum intensity value for data imputation, e.g.
--min_replace 5000. Defaults to 5000.: "--min_replace: The minimum intensity value for data imputation, e.g. --min_replace 5000. Defaults to 5000."
1---2name: metabolomics-peak-data-normalization-and-handling3description: Use when when you have raw peak intensity matrices from metabolomics LC-MS/MS experiments with zero values (missing peaks or undetected compounds) and need to prepare data for pathway-level analysis using PLAGE, ORA, or GSEA.4license: CC-BY-4.05---67# metabolomics-peak-data-normalization-and-handling89## Summary1011Standardize metabolomics peak intensity matrices through log-2 transformation and z-score normalization to enable robust pathway activity scoring. This preprocessing step is essential for metabolomics data because it addresses missing values, scales intensities across samples, and prepares data for decomposition-based pathway analysis methods like PLAGE.1213## When to use1415When you have raw peak intensity matrices from metabolomics LC-MS/MS experiments with zero values (missing peaks or undetected compounds) and need to prepare data for pathway-level analysis using PLAGE, ORA, or GSEA. Apply this skill before computing pathway activity scores, especially if your data exhibits high noise or missing peak rates (5–20%), which are prevalent in metabolomics and would otherwise bias downstream comparisons.1617## When NOT to use1819- Data is already quantile-normalized or batch-corrected by external tools (e.g., limma, ComBat); re-normalizing may introduce bias.20- Peak intensities are already log-transformed and z-scored by the mass spectrometry processing pipeline.21- The experimental design has nested or paired factors where group-level imputation is inappropriate; use custom imputation strategies instead.2223## Inputs2425- Peak intensity CSV matrix (rows = peaks, columns = samples, first column = peak ID)26- Annotation CSV matrix (peak ID, entity_id [KEGG/ChEBI compound ID])27- Experimental design specification (sample-to-group assignments)2829## Outputs3031- Normalized log-2 z-score intensity matrix (zero mean, unit variance across samples)32- Imputed missing value indicator (optional, for tracking replaced zeros)33- Preprocessed data structure ready for pathway decomposition (PLAGE/ORA/GSEA input)3435## How to apply3637Load the peak intensity matrix (rows = peaks with column 1 = peak ID, remaining columns = sample intensities) and annotation matrix (peak ID → KEGG/ChEBI compound ID). Apply data imputation: if all samples in a single experimental factor have zero intensity, replace with a user-defined minimum (default 5000); if only some samples in a factor are zero, replace with the mean of non-zero samples in that factor. This preserves group structure while avoiding log(0). Then apply log-2 transformation to the entire imputed matrix and z-score standardize across all samples (zero mean, unit variance) using scipy preprocessing. This normalization stabilizes variance across intensity ranges and makes the data amenable to SVD-based decomposition methods. Validate that the output matrix has no NaN or Inf values and that column means are approximately zero.3839## Related tools4041- **PALS (Pathway Activity Level Scoring)** (Executes normalization as a preprocessing step before PLAGE decomposition or ORA/GSEA ranking) — https://github.com/glasgowcompbio/PALS42- **Scipy preprocessing module** (Provides StandardScaler for z-score normalization and standardization functions)4344## Examples4546```47python pals/run.py PLAGE notebooks/test_data/HAT/int_df.csv notebooks/test_data/HAT/annotation_df.csv test_output.csv --db PiMP_KEGG --comparisons Stage_1/Control Stage_2/Control --min_replace 500048```4950## Evaluation signals5152- Output matrix has zero column mean (±1e-10 tolerance) and unit column variance for all samples.53- No NaN, Inf, or -Inf values present in the normalized matrix after transformation.54- Imputed zeros are documented; verify that group-wise means of non-imputed values are preserved before and after log transformation.55- Comparison of pathway activity scores (Spearman correlation) between baseline and 5% Gaussian noise-perturbed data shows ≥0.85 correlation for PLAGE (metric of normalization robustness).56- Intensity ranges span multiple orders of magnitude in the log-2 domain (typically 0–20 range for LC-MS intensities) without extreme outliers (>3 σ from mean after z-scoring).5758## Limitations5960- Zero replacement strategy assumes missing peaks are missing at random (MAR) within experimental factors; non-random missingness may bias results.61- Group-level imputation requires correct experimental design specification; misassigned samples will propagate incorrect mean values.62- Log-2 transformation assumes all intensities are positive; negative or zero imputation values will produce undefined logarithms.63- Z-score normalization assumes roughly symmetric or light-tailed intensity distributions; extreme skewness or multimodal distributions may require alternative transformations (e.g., rank-based).64- The method is sensitive to outlier samples; anomalous samples with extreme intensities will inflate standard deviations and compress other samples.6566## Evidence6768- [readme] Data imputation is performed to the intensity matrix when it is loaded: if all of the samples in a single experimental factor have intensities of zero these are replaced by the minimum intensity value (which can be set by the user); and if only some of the sample values in a factor are zero then these are replaced by the mean value of the non-zero samples in that factor.: "Data imputation is performed to the intensity matrix when it is loaded: if all of the samples in a single experimental factor have intensities of zero these are replaced by the minimum intensity"69- [readme] The data is subsequently transformed to log-2 base and standardised using the preprocessing module in Scipy such that the intensity matrix has a zero mean and unit variance across the samples.: "The data is subsequently transformed to log-2 base and standardised using the preprocessing module in Scipy such that the intensity matrix has a zero mean and unit variance across the samples."70- [intro] PALS results demonstrate greater robustness to noise and missing peaks compared to ORA and GSEA, which is particularly important for metabolomics peak data where noise and missing peaks are prevalent.: "PALS results demonstrate greater robustness to noise and missing peaks compared to ORA and GSEA, which is particularly important for metabolomics peak data where noise and missing peaks are prevalent."71- [readme] --min_replace: The minimum intensity value for data imputation, e.g. `--min_replace 5000`. Defaults to 5000.: "--min_replace: The minimum intensity value for data imputation, e.g. `--min_replace 5000`. Defaults to 5000."
