chemical-formula-subformula-classification
Summary
Assign tandem mass spectrum fragments to chemical subformulae using structural decomposition, enabling formula-level spectrum prediction via autoregressive reconstruction. This preprocessing step converts raw spectra into subformula-labeled training data for models like SCARF.
When to use
Apply this skill when you have tandem mass spectra with known molecular structures (SMILES, InChI, or chemical formula) and aim to train or evaluate a formula-level spectrum predictor. Subformula classification is essential before training SCARF or similar models that predict spectra autoregressively at the chemical formula level rather than at the molecular fragment level.
When NOT to use
- Input spectra lack associated molecular structures or SMILES — subformula assignment requires known chemistry.
- Goal is fragment-level prediction (e.g., ICEBERG, which models molecular fragments) rather than formula-level prediction.
- Spectra are already annotated with subformulae or you are only performing retrieval without model training.
Inputs
- tandem mass spectra (MGF or HDF5 format with peak m/z and intensity)
- molecular structures (SMILES strings, InChI keys, or chemical formulae)
- raw spectrum dataset (e.g., NIST20 .SDF export or MassSpecGym)
Outputs
- subformula-annotated spectrum dataset (HDF5 with subformula labels per peak)
- two preprocessed variants: magma_subform_50 and no_subform
- structured metadata linking spectra to subformulae and collision energy
How to apply
Use the MAGMa algorithm or permissive subformula assignment to decompose each molecule into all chemically feasible subformulae. Two variants are produced: magma_subform_50 strictly labels subformulae based on SMILES structure using MAGMa, while no_subform permissively allows all entries to pass through. Each fragment peak in the spectrum is mapped to its corresponding subformula, creating a labeled dataset. Run the assignment pipeline via data_scripts/forms/assign_subformulae.py or the batch script data_scripts/all_assign_subform.sh, which preprocesses the data into HDF5 or structured format compatible with downstream model training. Verify that subformula assignments are consistent with the molecular structure and that all peaks have valid formula labels.
Related tools
- ms-pred (SCARF model and MAGMa integration) (Loads pre-trained SCARF model weights and executes subformula classification pipeline; stores preprocessed subformula-annotated data) — https://github.com/coleygroup/ms-pred
- MAGMa algorithm (Performs strict structural subformula decomposition by SMILES fragmentation rules)
- NIST20 (tandem MS database) (Source of raw spectra and molecular structures for preprocessing)
Examples
python data_scripts/forms/assign_subformulae.py --dataset nist20 --output-dir data/spec_datasets/nist20/subformulae
Evaluation signals
- All spectrum peaks are assigned to valid chemical subformulae consistent with the parent molecular formula.
- Subformula-annotated dataset successfully loads into training pipeline without missing or malformed labels.
- magma_subform_50 variant is stricter (fewer assigned peaks) than no_subform; both are non-empty.
- Downstream SCARF model training converges and produces spectrum predictions with comparable or improved accuracy vs. baselines.
- HDF5 output structure matches the expected schema (labels.tsv, spec_files.hdf5 with subformula keys).
Limitations
- MAGMa-based assignment may fail or produce incomplete labelings for unusual or exotic chemical structures not well-represented in training chemistry.
- Permissive no_subform variant may introduce noise by labeling peaks that do not correspond to chemically feasible fragments, reducing model precision.
- Processing large datasets (e.g., full PubChem) is computationally expensive and requires hours even with parallelization; formula subset generation has been observed to take several hours.
- Subformula classification depends on accurate SMILES or InChI input; incorrect or ambiguous structures will propagate errors downstream.
Evidence
- [other] SCARF is a spectrum predictor model that operates by performing subformula classification to autoregressively reconstruct fragmentations and predict tandem mass spectra at the chemical formula level.: "SCARF is a spectrum predictor model that operates by performing subformula classification to autoregressively reconstruct fragmentations and predict tandem mass spectra at the chemical formula level."
- [readme] Data should then be assigned to subformulae files using
data_scripts/forms/assign_subformulae.py, which will preprocess the data. We produce two fragment versions of the molecule, magma_subform_50 and no_subform. The former strictly labels subformula based upon smiles structure and the latter is permissive and allows all entries to pass.: "Data should then be assigned to subformulae files using data_scripts/forms/assign_subformulae.py, which will preprocess the data. We produce two fragment versions of the molecule,"
- [readme] Making formula subsets takes longer (on the order of several hours, even parallelized) as it requires converting each molecule in PubChem to a mol / InChI.: "Making formula subsets takes longer (on the order of several hours, even parallelized) as it requires converting each molecule in PubChem to a mol / InChI."
- [intro] ICEBERG predicts spectra at the level of molecular fragments, whereas SCARF predicts spectra at the level of chemical formula.: "ICEBERG predicts spectra at the level of molecular fragments, whereas SCARF predicts spectra at the level of chemical formula."
1---2name: chemical-formula-subformula-classification3description: Use when you have tandem mass spectra with known molecular structures (SMILES, InChI, or chemical formula) and aim to train or evaluate a formula-level spectrum predictor.4license: CC-BY-4.05---67# chemical-formula-subformula-classification89## Summary1011Assign tandem mass spectrum fragments to chemical subformulae using structural decomposition, enabling formula-level spectrum prediction via autoregressive reconstruction. This preprocessing step converts raw spectra into subformula-labeled training data for models like SCARF.1213## When to use1415Apply this skill when you have tandem mass spectra with known molecular structures (SMILES, InChI, or chemical formula) and aim to train or evaluate a formula-level spectrum predictor. Subformula classification is essential before training SCARF or similar models that predict spectra autoregressively at the chemical formula level rather than at the molecular fragment level.1617## When NOT to use1819- Input spectra lack associated molecular structures or SMILES — subformula assignment requires known chemistry.20- Goal is fragment-level prediction (e.g., ICEBERG, which models molecular fragments) rather than formula-level prediction.21- Spectra are already annotated with subformulae or you are only performing retrieval without model training.2223## Inputs2425- tandem mass spectra (MGF or HDF5 format with peak m/z and intensity)26- molecular structures (SMILES strings, InChI keys, or chemical formulae)27- raw spectrum dataset (e.g., NIST20 .SDF export or MassSpecGym)2829## Outputs3031- subformula-annotated spectrum dataset (HDF5 with subformula labels per peak)32- two preprocessed variants: magma_subform_50 and no_subform33- structured metadata linking spectra to subformulae and collision energy3435## How to apply3637Use the MAGMa algorithm or permissive subformula assignment to decompose each molecule into all chemically feasible subformulae. Two variants are produced: `magma_subform_50` strictly labels subformulae based on SMILES structure using MAGMa, while `no_subform` permissively allows all entries to pass through. Each fragment peak in the spectrum is mapped to its corresponding subformula, creating a labeled dataset. Run the assignment pipeline via `data_scripts/forms/assign_subformulae.py` or the batch script `data_scripts/all_assign_subform.sh`, which preprocesses the data into HDF5 or structured format compatible with downstream model training. Verify that subformula assignments are consistent with the molecular structure and that all peaks have valid formula labels.3839## Related tools4041- **ms-pred (SCARF model and MAGMa integration)** (Loads pre-trained SCARF model weights and executes subformula classification pipeline; stores preprocessed subformula-annotated data) — https://github.com/coleygroup/ms-pred42- **MAGMa algorithm** (Performs strict structural subformula decomposition by SMILES fragmentation rules)43- **NIST20 (tandem MS database)** (Source of raw spectra and molecular structures for preprocessing)4445## Examples4647```48python data_scripts/forms/assign_subformulae.py --dataset nist20 --output-dir data/spec_datasets/nist20/subformulae49```5051## Evaluation signals5253- All spectrum peaks are assigned to valid chemical subformulae consistent with the parent molecular formula.54- Subformula-annotated dataset successfully loads into training pipeline without missing or malformed labels.55- magma_subform_50 variant is stricter (fewer assigned peaks) than no_subform; both are non-empty.56- Downstream SCARF model training converges and produces spectrum predictions with comparable or improved accuracy vs. baselines.57- HDF5 output structure matches the expected schema (labels.tsv, spec_files.hdf5 with subformula keys).5859## Limitations6061- MAGMa-based assignment may fail or produce incomplete labelings for unusual or exotic chemical structures not well-represented in training chemistry.62- Permissive no_subform variant may introduce noise by labeling peaks that do not correspond to chemically feasible fragments, reducing model precision.63- Processing large datasets (e.g., full PubChem) is computationally expensive and requires hours even with parallelization; formula subset generation has been observed to take several hours.64- Subformula classification depends on accurate SMILES or InChI input; incorrect or ambiguous structures will propagate errors downstream.6566## Evidence6768- [other] SCARF is a spectrum predictor model that operates by performing subformula classification to autoregressively reconstruct fragmentations and predict tandem mass spectra at the chemical formula level.: "SCARF is a spectrum predictor model that operates by performing subformula classification to autoregressively reconstruct fragmentations and predict tandem mass spectra at the chemical formula level."69- [readme] Data should then be assigned to subformulae files using `data_scripts/forms/assign_subformulae.py`, which will preprocess the data. We produce two fragment versions of the molecule, `magma_subform_50` and `no_subform`. The former strictly labels subformula based upon smiles structure and the latter is permissive and allows all entries to pass.: "Data should then be assigned to subformulae files using `data_scripts/forms/assign_subformulae.py`, which will preprocess the data. We produce two fragment versions of the molecule,"70- [readme] Making formula subsets takes longer (on the order of several hours, even parallelized) as it requires converting each molecule in PubChem to a mol / InChI.: "Making formula subsets takes longer (on the order of several hours, even parallelized) as it requires converting each molecule in PubChem to a mol / InChI."71- [intro] ICEBERG predicts spectra at the level of molecular fragments, whereas SCARF predicts spectra at the level of chemical formula.: "ICEBERG predicts spectra at the level of molecular fragments, whereas SCARF predicts spectra at the level of chemical formula."
HolobiomicsLab