spectral-feature-vector-aggregation
Summary
Aggregate per-sample MS2 fingerprint vectors (counts of MS2 peaks and neutral losses) into a unified comparable matrix across multiple samples. This enables cross-sample metabolomics comparisons in a retention-time agnostic manner, particularly for chemodiverse samples or those acquired on different LC/MS platforms.
When to use
You have generated per-sample MS2 fingerprints (as spec2vec document representations counting MS2 peaks and neutral losses to precursor in each sample) and need to align them into a single matrix for downstream cross-sample comparison, filtering, or visualization (e.g., MDS/PCoA, TMAP, heatmap). Use this when samples are chemodiverse, show poor feature overlap, or have strong retention-time shifts across different LC methods or mass spectrometers.
When NOT to use
- Input is already a pre-aggregated feature table or sample-by-feature matrix; use this skill only when starting from individual per-sample fingerprints.
- Samples have fundamentally different MS2 peak vocabularies or you intend to perform sample-specific peak selection before aggregation; alignment requires consistent feature space.
- You require retention-time aligned data as a primary criterion; MEMO is retention-time agnostic and suited for RT-shifted or multi-platform comparisons.
Inputs
- Per-sample MS2 fingerprint vectors (spec2vec document representations)
- List or collection of individual sample fingerprint files or objects from matchms-processed spectra
Outputs
- Unified MemoMatrix: a 2D numpy array with shape (n_samples, n_features) where n_features is the count of unique MS2 peaks and neutral losses
- Validated feature-aligned matrix ready for filtering and visualization
How to apply
Load each per-sample MS2 fingerprint vector (generated by matchms and spec2vec from processed spectra) into memory. Stack or concatenate these vectors using numpy array operations (e.g., numpy.vstack or numpy.column_stack) to create a unified feature matrix with consistent dimensions: rows representing samples and columns representing unique MS2 peaks or neutral losses. Validate matrix dimensions and confirm that all samples have the same feature set (same number of columns) and that no missing values or misaligned indices exist. This aggregated matrix then serves as input to filtering steps (e.g., removing peaks from blanks) and visualization or statistical downstream analysis.
Related tools
- matchms (Import, process, and clean per-sample MS2 spectra into standardized spectrum objects prior to fingerprinting) — https://github.com/matchms/matchms
- spec2vec (Generate MS2 fingerprint document representations (embeddings) from spectrum data using learned fragmental relationships) — https://github.com/iomega/spec2vec
- numpy (Perform array aggregation and concatenation operations (vstack, column_stack) and dimension validation)
- MEMO (Orchestrate the complete MS2-based sample vectorization workflow including fingerprint generation and matrix aggregation) — https://github.com/mandelbrot-project/memo
Examples
import numpy as np; from memo import load_fingerprints; fingerprints = [load_fingerprints(f) for f in sample_files]; memo_matrix = np.vstack(fingerprints); print(memo_matrix.shape)
Evaluation signals
- Matrix shape is (n_samples, n_features) with n_samples matching the count of input fingerprints and n_features consistent across all samples.
- All rows (samples) have no missing or NaN values; feature indices align across all samples (identical column order and label correspondence).
- Matrix can be successfully passed to downstream filtering (e.g., blank-peak removal) or visualization functions (MDS, TMAP, heatmap) without shape or alignment errors.
- Spot-check: verify that fingerprint counts are preserved (sum of features for a given sample before and after aggregation should be consistent if no filtering was applied).
- No duplicate or corrupted feature columns; each MS2 peak or neutral loss is represented exactly once.
Limitations
- Aggregation requires all input fingerprints to share a common feature vocabulary (MS2 peaks and neutral losses); samples with unique fragments may create sparsity or require feature imputation decisions.
- Large sample counts or high-dimensional feature spaces (many unique peaks/losses) can result in sparse matrices; consider sparse matrix formats for memory efficiency beyond ~10,000 samples.
- Matrix alignment is agnostic to retention time; cross-platform or cross-LC-method samples may still have batch effects not removed at this stage—filtering and normalization in downstream steps are needed.
- Assumes spec2vec or equivalent MS2 fingerprinting has already been applied; raw spectrum counts without document embeddings or without accounting for fragmental relationships may not be suitable for this aggregation.
Evidence
- [other] per-sample MS2 fingerprints (each generated by counting occurrences of MS2 peaks and neutral losses to the precursor): "Load per-sample MS2 fingerprints (each generated by counting occurrences of MS2 peaks and neutral losses to the precursor) from matchms-processed spectra using spec2vec document representations."
- [other] aggregate fingerprint vectors across all samples into a unified matrix using numpy array operations, ensuring consistent feature alignment: "Aggregate fingerprint vectors across all samples into a unified matrix using numpy array operations, ensuring consistent feature alignment."
- [other] validate matrix dimensions and feature consistency across samples to confirm preparedness for downstream alignment and filtering steps: "Validate matrix dimensions and feature consistency across samples to confirm preparedness for downstream alignment and filtering steps."
- [intro] MS2 fingerprints are generated by counting occurrences of MS2 peaks and neutral losses in each sample, and these fingerprints are then aligned in a second stage to compare different samples: "The occurence of MS2 peaks and neutral losses (to the precursor) in each sample is counted and used to generate an MS2 fingerprint. These fingerprints can in a second stage be aligned to compare"
- [readme] MEMO suits particularly well to compare chemodiverse samples with poor features overlap, or to compare samples with a strong RT shift, acquired using different LC methods or even different mass spectrometers: "MEMO suits particularly well to compare chemodiverse samples, ie with a poor features overlap, or to compare samples with a strong RT shift, acquired using different LC methods or even different mass"
1---2name: spectral-feature-vector-aggregation3description: Use when you have generated per-sample MS2 fingerprints (as spec2vec document representations counting MS2 peaks and neutral losses to precursor in each sample) and need to align them into a single matrix for downstream cross-sample comparison, filtering, or visualization (e.4license: CC-BY-4.05---67# spectral-feature-vector-aggregation89## Summary1011Aggregate per-sample MS2 fingerprint vectors (counts of MS2 peaks and neutral losses) into a unified comparable matrix across multiple samples. This enables cross-sample metabolomics comparisons in a retention-time agnostic manner, particularly for chemodiverse samples or those acquired on different LC/MS platforms.1213## When to use1415You have generated per-sample MS2 fingerprints (as spec2vec document representations counting MS2 peaks and neutral losses to precursor in each sample) and need to align them into a single matrix for downstream cross-sample comparison, filtering, or visualization (e.g., MDS/PCoA, TMAP, heatmap). Use this when samples are chemodiverse, show poor feature overlap, or have strong retention-time shifts across different LC methods or mass spectrometers.1617## When NOT to use1819- Input is already a pre-aggregated feature table or sample-by-feature matrix; use this skill only when starting from individual per-sample fingerprints.20- Samples have fundamentally different MS2 peak vocabularies or you intend to perform sample-specific peak selection before aggregation; alignment requires consistent feature space.21- You require retention-time aligned data as a primary criterion; MEMO is retention-time agnostic and suited for RT-shifted or multi-platform comparisons.2223## Inputs2425- Per-sample MS2 fingerprint vectors (spec2vec document representations)26- List or collection of individual sample fingerprint files or objects from matchms-processed spectra2728## Outputs2930- Unified MemoMatrix: a 2D numpy array with shape (n_samples, n_features) where n_features is the count of unique MS2 peaks and neutral losses31- Validated feature-aligned matrix ready for filtering and visualization3233## How to apply3435Load each per-sample MS2 fingerprint vector (generated by matchms and spec2vec from processed spectra) into memory. Stack or concatenate these vectors using numpy array operations (e.g., numpy.vstack or numpy.column_stack) to create a unified feature matrix with consistent dimensions: rows representing samples and columns representing unique MS2 peaks or neutral losses. Validate matrix dimensions and confirm that all samples have the same feature set (same number of columns) and that no missing values or misaligned indices exist. This aggregated matrix then serves as input to filtering steps (e.g., removing peaks from blanks) and visualization or statistical downstream analysis.3637## Related tools3839- **matchms** (Import, process, and clean per-sample MS2 spectra into standardized spectrum objects prior to fingerprinting) — https://github.com/matchms/matchms40- **spec2vec** (Generate MS2 fingerprint document representations (embeddings) from spectrum data using learned fragmental relationships) — https://github.com/iomega/spec2vec41- **numpy** (Perform array aggregation and concatenation operations (vstack, column_stack) and dimension validation)42- **MEMO** (Orchestrate the complete MS2-based sample vectorization workflow including fingerprint generation and matrix aggregation) — https://github.com/mandelbrot-project/memo4344## Examples4546```47import numpy as np; from memo import load_fingerprints; fingerprints = [load_fingerprints(f) for f in sample_files]; memo_matrix = np.vstack(fingerprints); print(memo_matrix.shape)48```4950## Evaluation signals5152- Matrix shape is (n_samples, n_features) with n_samples matching the count of input fingerprints and n_features consistent across all samples.53- All rows (samples) have no missing or NaN values; feature indices align across all samples (identical column order and label correspondence).54- Matrix can be successfully passed to downstream filtering (e.g., blank-peak removal) or visualization functions (MDS, TMAP, heatmap) without shape or alignment errors.55- Spot-check: verify that fingerprint counts are preserved (sum of features for a given sample before and after aggregation should be consistent if no filtering was applied).56- No duplicate or corrupted feature columns; each MS2 peak or neutral loss is represented exactly once.5758## Limitations5960- Aggregation requires all input fingerprints to share a common feature vocabulary (MS2 peaks and neutral losses); samples with unique fragments may create sparsity or require feature imputation decisions.61- Large sample counts or high-dimensional feature spaces (many unique peaks/losses) can result in sparse matrices; consider sparse matrix formats for memory efficiency beyond ~10,000 samples.62- Matrix alignment is agnostic to retention time; cross-platform or cross-LC-method samples may still have batch effects not removed at this stage—filtering and normalization in downstream steps are needed.63- Assumes spec2vec or equivalent MS2 fingerprinting has already been applied; raw spectrum counts without document embeddings or without accounting for fragmental relationships may not be suitable for this aggregation.6465## Evidence6667- [other] per-sample MS2 fingerprints (each generated by counting occurrences of MS2 peaks and neutral losses to the precursor): "Load per-sample MS2 fingerprints (each generated by counting occurrences of MS2 peaks and neutral losses to the precursor) from matchms-processed spectra using spec2vec document representations."68- [other] aggregate fingerprint vectors across all samples into a unified matrix using numpy array operations, ensuring consistent feature alignment: "Aggregate fingerprint vectors across all samples into a unified matrix using numpy array operations, ensuring consistent feature alignment."69- [other] validate matrix dimensions and feature consistency across samples to confirm preparedness for downstream alignment and filtering steps: "Validate matrix dimensions and feature consistency across samples to confirm preparedness for downstream alignment and filtering steps."70- [intro] MS2 fingerprints are generated by counting occurrences of MS2 peaks and neutral losses in each sample, and these fingerprints are then aligned in a second stage to compare different samples: "The occurence of MS2 peaks and neutral losses (to the precursor) in each sample is counted and used to generate an *MS2 fingerprint*. These fingerprints can in a second stage be aligned to compare"71- [readme] MEMO suits particularly well to compare chemodiverse samples with poor features overlap, or to compare samples with a strong RT shift, acquired using different LC methods or even different mass spectrometers: "MEMO suits particularly well to compare chemodiverse samples, ie with a poor features overlap, or to compare samples with a strong RT shift, acquired using different LC methods or even different mass"
HolobiomicsLab 