Chemical Annotation Matrix Construction

chemical-annotation-matrix-construction

Summary

Construct a feature-by-sample matrix from GNPS chemical annotations (m/z, retention time, spectral library matches) retrieved from a subset of ReDU/MassIVE files, handling missing values by imputation or removal to prepare data for unsupervised dimensionality reduction and visualization.

When to use

You have selected a subset of public tandem MS files from ReDU/MassIVE that have been processed through GNPS spectral library matching, and you need to organize their chemical annotations into a matrix (rows = files, columns = annotated features) before performing PCA or other multivariate analysis to explore sample relationships and chemical annotation patterns across the cohort.

When NOT to use

• Input files have not been processed through GNPS spectral library matching (raw MS2 spectra only, no annotations).
• Only unannotated m/z features are available (use raw feature tables from mass spectrometry preprocessing instead).
• You need to include multiple levels of annotation confidence (e.g., level 2 and level 3 GNPS annotations separately); this skill merges all available annotations without stratifying by confidence.

Inputs

• Selected MassIVE file identifiers (MSV IDs)
• GNPS spectral library search results with compound identifications
• ReDU chemical annotation database query (per-file feature lists with m/z, RT, annotation ID)

Outputs

• Feature-by-sample matrix (tab-separated text file or CSV: rows = files, columns = annotated chemical features)
• Metadata file mapping file identifiers to MassIVE IDs and sample attributes
• Matrix statistics (feature count, file count, sparsity, missing value report)

How to apply

Query the ReDU database to retrieve GNPS chemical annotation results (spectral library matches with m/z, retention time, and compound identities) for each selected MassIVE file. Organize the annotations into a matrix where rows represent individual files and columns represent unique annotated chemical features. Handle missing values—where a file lacks an annotation for a feature—by either removing features below a frequency threshold or imputing with zero abundance. Validate matrix dimensions and consistency (e.g., no negative values, correct data types). The resulting matrix is then compatible with scikit-learn PCA and Emperor visualization import, enabling unsupervised exploration of chemical annotation structure across samples.

Related tools

• ReDU (Query and retrieve GNPS-annotated chemical features from public MassIVE files for selected cohort) — https://github.com/mwang87/ReDU-MS2-GNPS
• GNPS (Perform spectral library matching to generate chemical annotations (m/z, retention time, compound identities) for each file's MS2 spectra) — https://github.com/CCMS-UCSD/GNPSDocumentation
• MassIVE (Public repository storing raw and processed tandem MS data files that serve as input to ReDU and GNPS)
• scikit-learn (Apply PCA to the constructed feature-by-sample matrix to compute principal component loadings and scores)

Evaluation signals

• Matrix dimensions (rows = number of selected files, columns = number of unique annotated features) are consistent with input file count and annotation inventory.
• No negative or invalid values in the matrix; all entries are non-negative abundances or presence/absence flags.
• Sparsity metric (percentage of zero/missing entries) is documented and reasonable for the annotation scope.
• Metadata file row count matches matrix row count; all file identifiers are traceable to MassIVE IDs.
• Matrix imports successfully into scikit-learn and Emperor without format errors; PCA computes and produces valid PC scores.

Limitations

• GNPS annotations are level 2 or 3 by the 2007 metabolomics standard initiative (putative or putatively characterized, not confirmed identifications), limiting biological interpretability.
• The same chemical may have multiple GNPS annotations due to slight variation in MS2 spectra (m/z or abundance differences), leading to feature duplication; deduplication strategy must be specified.
• Missing annotations for a file–feature pair can introduce bias if imputation method is not documented or if removal threshold is too aggressive (loss of rare features).
• Matrix construction does not account for batch effects, instrumental drift, or sample heterogeneity across files from different laboratories or collection dates.

Evidence

• [methods] Query the ReDU database to retrieve chemical annotation matrices for selected MassIVE files, with rows as files and columns as annotated chemical features (m/z, retention time, GNPS annotations).: "Query the ReDU database to retrieve chemical annotation matrices for selected MassIVE files, with rows as files and columns as annotated chemical features (m/z, retention time, GNPS annotations)."
• [methods] Construct a feature-by-sample matrix from the annotation data, handling missing values by imputation or removal.: "Construct a feature-by-sample matrix from the annotation data, handling missing values by imputation or removal."
• [abstract] The same chemical can have multiple GNPS annotations. Slight variation in the MS2 spectra (m/z or abundance) cause the pattern to match different reference MS2 spectra for the same chemical: "The same chemical can have multiple GNPS annotations. Slight variation in the MS2 spectra cause patterns to match different reference spectra"
• [abstract] GNPS annotations via spectral reference matching are considered level 2 (putative annotation based on spectral library similarity) or level 3 (putatively characterized compound class): "GNPS annotations via spectral reference matching are considered level 2 (putative annotation based on spectral library similarity) or level 3"
• [readme] Download batch template for GNPS at https://redu.ucsd.edu/metabatchdump Run Batch Workflow for Spectral Library Search: "Download batch template for GNPS at https://redu.ucsd.edu/metabatchdump Run Batch Workflow for Spectral Library Search"