metabolite-database-embedding-lookup
Summary
Train a spectrum transformer model using contrastive learning to align tandem MS spectrum embeddings with molecular structure fingerprint embeddings, enabling structure annotation through nearest-neighbor lookup in a reference metabolite database. This approach replaces database-dependent fragmentation tree matching with learned joint embedding spaces.
When to use
When you have paired tandem MS spectra and known molecular structures (SMILES or fingerprints) and want to annotate novel spectra by retrieving similar structures from a reference database without relying on spectral database matching. Use this when you need to generalize beyond existing spectral libraries or when contrastive embedding-based retrieval outperforms traditional cosine/spectral similarity metrics.
When NOT to use
- Input consists only of MS1 precursor masses without fragmentation spectra; use MIST-CF for chemical formula prediction instead.
- Reference database is very small (< 100 metabolites) or highly specialized; contrastive learning may not yield robust embeddings.
- Negative mode spectra or multiple adduct types are required; original MIST focuses on positive mode [M+H]+ only.
Inputs
- Tandem MS spectra (MGF format or .ms files with m/z and intensity pairs)
- Molecular structures as SMILES strings or known chemical identifiers
- Reference metabolite database with both spectra and structures
Outputs
- Joint embedding space (spectrum embeddings and fingerprint embeddings aligned)
- Ranked candidate structures for query spectra with similarity scores
- Structure annotations with relevance scores
How to apply
Load paired tandem MS spectra and corresponding molecular structures as training data. Preprocess spectra by normalizing intensities and filtering noise; encode structures into molecular fingerprints using a reference fingerprint scheme (e.g., ECFP). Initialize a MIST transformer with separate spectrum encoder and fingerprint encoder modules. Train end-to-end using contrastive loss (with negative sampling from unpaired spectra-structure pairs in each batch) to align spectrum embeddings with fingerprint embeddings in a joint space. After training convergence, encode a reference metabolite database (all spectra and structures) into the learned joint embedding space. For novel query spectra, generate embeddings via the trained spectrum encoder and perform k-nearest-neighbor lookup in the reference database embeddings to retrieve candidate structures, ranked by similarity scores (e.g., cosine distance in embedding space).
Related tools
Examples
. quickstart/00_download_models.sh && . quickstart/01_run_models.sh
Evaluation signals
- Retrieval rank metric: fraction of query spectra for which the correct structure ranks in top-k (e.g., top-1, top-5, top-10)
- Embedding space alignment: verify that paired spectrum-fingerprint embeddings are closer than randomly sampled spectrum-fingerprint pairs; measure via contrastive loss convergence
- Cosine similarity distribution: confirm that true positive pairs have higher cosine similarity in the joint embedding space than false positives from the same batch
- Database lookup precision: for a held-out test set of spectra, verify that nearest-neighbor candidates match known ground-truth structures with acceptable error rates
- Embedding space stability: re-encode the same spectra multiple times and confirm embeddings are reproducible (e.g., correlation > 0.99)
Limitations
- Method requires paired spectrum-structure training data; performance degrades if training set lacks chemical diversity or is biased toward certain instrument types or ionization modes.
- Contrastive learning depends on quality negative sampling; poor negative sampling (e.g., similar but incorrect structures in a batch) can hurt embedding alignment.
- Original MIST focuses on positive mode MS/MS with [M+H]+ adducts; multiple adduct types and negative mode support require model extension (as noted in MIST-CF README).
- Reference database size and composition affect retrieval performance; small or non-representative databases yield lower annotation accuracy.
- Joint embedding space is specific to the fingerprint encoding scheme used during training; transferability to different fingerprint schemes is limited.
Evidence
- [intro] When trained in a contrastive learning framework, MIST enables embedding and structure annotation by database lookup.: "when trained in a contrastive learning framework, MIST enables embedding and structure annotation by database lookup"
- [other] Train MIST end-to-end using contrastive loss to align spectrum embeddings with fingerprint embeddings, with negative sampling from unpaired spectra-structure pairs in each batch.: "Train MIST end-to-end using contrastive loss to align spectrum embeddings with fingerprint embeddings, with negative sampling from unpaired spectra-structure pairs in each batch"
- [other] For novel spectra, generate embeddings via the trained spectrum encoder and perform nearest-neighbor lookup in the reference database embeddings to retrieve candidate structures with similarity scores.: "For novel spectra, generate embeddings via the trained spectrum encoder and perform nearest-neighbor lookup in the reference database embeddings to retrieve candidate structures with similarity scores"
- [readme] MIST models can be used to predict molecular fingerprints from tandem mass spectrometry data and, when trained in a contrastive learning framework, enable embedding and structure annotation by database lookup.: "MIST models can be used to predict molecular fingerprints from tandem mass spectrometry data and, when trained in a contrastive learning framework, enable embedding and structure annotation by"
- [other] Preprocess spectra (normalize intensities, filter noise) and encode chemical structures into fingerprints using a reference fingerprint scheme.: "Preprocess spectra (normalize intensities, filter noise) and encode chemical structures into fingerprints using a reference fingerprint scheme"
1---2name: metabolite-database-embedding-lookup3description: Use when when you have paired tandem MS spectra and known molecular structures (SMILES or fingerprints) and want to annotate novel spectra by retrieving similar structures from a reference database without relying on spectral database matching.4license: CC-BY-4.05---67# metabolite-database-embedding-lookup89## Summary1011Train a spectrum transformer model using contrastive learning to align tandem MS spectrum embeddings with molecular structure fingerprint embeddings, enabling structure annotation through nearest-neighbor lookup in a reference metabolite database. This approach replaces database-dependent fragmentation tree matching with learned joint embedding spaces.1213## When to use1415When you have paired tandem MS spectra and known molecular structures (SMILES or fingerprints) and want to annotate novel spectra by retrieving similar structures from a reference database without relying on spectral database matching. Use this when you need to generalize beyond existing spectral libraries or when contrastive embedding-based retrieval outperforms traditional cosine/spectral similarity metrics.1617## When NOT to use1819- Input consists only of MS1 precursor masses without fragmentation spectra; use MIST-CF for chemical formula prediction instead.20- Reference database is very small (< 100 metabolites) or highly specialized; contrastive learning may not yield robust embeddings.21- Negative mode spectra or multiple adduct types are required; original MIST focuses on positive mode [M+H]+ only.2223## Inputs2425- Tandem MS spectra (MGF format or .ms files with m/z and intensity pairs)26- Molecular structures as SMILES strings or known chemical identifiers27- Reference metabolite database with both spectra and structures2829## Outputs3031- Joint embedding space (spectrum embeddings and fingerprint embeddings aligned)32- Ranked candidate structures for query spectra with similarity scores33- Structure annotations with relevance scores3435## How to apply3637Load paired tandem MS spectra and corresponding molecular structures as training data. Preprocess spectra by normalizing intensities and filtering noise; encode structures into molecular fingerprints using a reference fingerprint scheme (e.g., ECFP). Initialize a MIST transformer with separate spectrum encoder and fingerprint encoder modules. Train end-to-end using contrastive loss (with negative sampling from unpaired spectra-structure pairs in each batch) to align spectrum embeddings with fingerprint embeddings in a joint space. After training convergence, encode a reference metabolite database (all spectra and structures) into the learned joint embedding space. For novel query spectra, generate embeddings via the trained spectrum encoder and perform k-nearest-neighbor lookup in the reference database embeddings to retrieve candidate structures, ranked by similarity scores (e.g., cosine distance in embedding space).3839## Related tools4041- **MIST** (Transformer architecture for encoding spectra and fingerprints; provides the spectrum and fingerprint encoder modules trained via contrastive loss) — https://github.com/samgoldman97/mist42- **MIST-CF** (Extension for chemical formula prediction from MS/MS data; complements structure annotation with formula constraints) — https://github.com/samgoldman97/mist-cf4344## Examples4546```47. quickstart/00_download_models.sh && . quickstart/01_run_models.sh48```4950## Evaluation signals5152- Retrieval rank metric: fraction of query spectra for which the correct structure ranks in top-k (e.g., top-1, top-5, top-10)53- Embedding space alignment: verify that paired spectrum-fingerprint embeddings are closer than randomly sampled spectrum-fingerprint pairs; measure via contrastive loss convergence54- Cosine similarity distribution: confirm that true positive pairs have higher cosine similarity in the joint embedding space than false positives from the same batch55- Database lookup precision: for a held-out test set of spectra, verify that nearest-neighbor candidates match known ground-truth structures with acceptable error rates56- Embedding space stability: re-encode the same spectra multiple times and confirm embeddings are reproducible (e.g., correlation > 0.99)5758## Limitations5960- Method requires paired spectrum-structure training data; performance degrades if training set lacks chemical diversity or is biased toward certain instrument types or ionization modes.61- Contrastive learning depends on quality negative sampling; poor negative sampling (e.g., similar but incorrect structures in a batch) can hurt embedding alignment.62- Original MIST focuses on positive mode MS/MS with [M+H]+ adducts; multiple adduct types and negative mode support require model extension (as noted in MIST-CF README).63- Reference database size and composition affect retrieval performance; small or non-representative databases yield lower annotation accuracy.64- Joint embedding space is specific to the fingerprint encoding scheme used during training; transferability to different fingerprint schemes is limited.6566## Evidence6768- [intro] When trained in a contrastive learning framework, MIST enables embedding and structure annotation by database lookup.: "when trained in a contrastive learning framework, MIST enables embedding and structure annotation by database lookup"69- [other] Train MIST end-to-end using contrastive loss to align spectrum embeddings with fingerprint embeddings, with negative sampling from unpaired spectra-structure pairs in each batch.: "Train MIST end-to-end using contrastive loss to align spectrum embeddings with fingerprint embeddings, with negative sampling from unpaired spectra-structure pairs in each batch"70- [other] For novel spectra, generate embeddings via the trained spectrum encoder and perform nearest-neighbor lookup in the reference database embeddings to retrieve candidate structures with similarity scores.: "For novel spectra, generate embeddings via the trained spectrum encoder and perform nearest-neighbor lookup in the reference database embeddings to retrieve candidate structures with similarity scores"71- [readme] MIST models can be used to predict molecular fingerprints from tandem mass spectrometry data and, when trained in a contrastive learning framework, enable embedding and structure annotation by database lookup.: "MIST models can be used to predict molecular fingerprints from tandem mass spectrometry data and, when trained in a contrastive learning framework, enable embedding and structure annotation by"72- [other] Preprocess spectra (normalize intensities, filter noise) and encode chemical structures into fingerprints using a reference fingerprint scheme.: "Preprocess spectra (normalize intensities, filter noise) and encode chemical structures into fingerprints using a reference fingerprint scheme"
HolobiomicsLab 