nmr-modality-ablation-analysis
Summary
Systematically evaluate the individual and joint contributions of ¹H NMR, ¹³C NMR, and combined 1D NMR inputs to molecular structure prediction accuracy by running inference under each modality condition independently and comparing performance metrics. This skill isolates which spectroscopic modalities drive structure elucidation performance and quantifies synergy between them.
When to use
Apply this skill when you have a trained multitask machine learning model for structure prediction, test set molecules with paired ¹H and ¹³C NMR spectra, and need to understand the marginal contribution of each NMR modality or justify multimodal input design. Use it when stakeholders ask 'which NMR experiment is actually necessary?' or when you suspect one modality is redundant with the other.
When NOT to use
- Input lacks paired ¹H and ¹³C NMR spectra for the same molecules (modality ablation requires both modalities present to be meaningful).
- The model is not yet trained or checkpoint is not available; ablation analysis requires a frozen, fixed model.
- Goal is to optimize model architecture or hyperparameters; this skill measures performance, not tunes the model.
Inputs
- Trained NMR2Struct model checkpoint (weights and architecture)
- Test set molecular structures (SMILES or graph representation)
- ¹H NMR spectra (CNN-compatible tensor, e.g., shape [batch, channels, frequency_bins])
- ¹³C NMR spectra (CNN-compatible tensor, same shape as ¹H)
- Ground truth molecular formula and connectivity labels
Outputs
- Per-modality accuracy metrics (exact-match formula accuracy, structure connectivity F1 or graph edit distance)
- Modality-specific error distribution (confusion matrix by atom type, bond type, molecular weight bins)
- Synergy quantification (combined accuracy minus single-modality baseline)
- Tabulated results by modality showing accuracy, error gaps, and per-molecule predictions
- Comparison summary (which modality(s) dominate for different structural classes)
How to apply
Load the trained NMR2Struct model checkpoint and prepare the test set with aligned molecular ground truth labels and corresponding ¹H and ¹³C NMR spectra. Create three separate inference batches: (1) ¹H-only spectra masked to zeros in ¹³C channels, (2) ¹³C-only spectra masked to zeros in ¹H channels, and (3) both modalities combined. Run the convolutional neural network encoder and transformer decoder on each batch independently, generating molecular formula and connectivity predictions for each condition. Compute accuracy metrics (exact-match accuracy for formula, graph edit distance or connectivity F1 for structure) separately for each modality. Quantify synergy by subtracting the maximum single-modality accuracy from the combined accuracy; compare error distributions across conditions to identify which structural features (connectivity, functional groups, molecular weight) each modality resolves best. Tabulate results by modality with accuracy, confidence intervals, and per-molecule error analysis.
Related tools
- Convolutional Neural Network (CNN) encoder (Encodes raw 1D NMR spectra (¹H and/or ¹³C) into learned feature representations for transformer input)
- Transformer architecture (Decodes CNN feature representations and assembled molecular fragments into predicted structure (formula and connectivity))
- NMR2Struct multitask framework (End-to-end model combining CNN encoder and transformer decoder for structure prediction from 1D NMR)
Evaluation signals
- Exact-match accuracy for predicted molecular formula matches ground truth for ≥80% of test molecules in single-modality and combined conditions.
- Synergy value (combined minus best single-modality accuracy) is non-negative and quantifies the benefit of multimodal input; if synergy ≈ 0, modalities are redundant.
- Error distribution differs qualitatively across modalities (e.g., ¹H alone fails on isomers with identical ¹H shifts but different ¹³C shifts, while ¹³C alone struggles with quaternary carbons); this confirms complementarity.
- Graph edit distance or connectivity F1 score for structure prediction increases monotonically from single modalities to combined input across molecule classes.
- Per-molecule predictions show same connectivity for all three conditions on molecules where the model achieves high confidence; large disagreements flag ambiguous spectra or modality-specific artifacts.
Limitations
- Ablation measures marginal contribution but does not explain which spectroscopic features (e.g., splitting patterns, chemical shift ranges) drive predictions; interpretation requires additional feature attribution analysis.
- Performance reported on molecules with up to 19 heavy atoms; ablation results may not generalize to larger structures or rare functional groups outside the training distribution.
- Masking one modality to zeros assumes independence; in a real multitask model, the two modalities may share learned representations in early layers, so true single-modality performance may differ from zero-masking.
- Exact-match accuracy is a strict metric; two predictions differing by a single atom or bond will be marked as failure, potentially underestimating practical utility of lower-accuracy modality combinations.
Evidence
- [intro] multitask machine learning framework capable of predicting molecular structure using 1D ¹H NMR, ¹³C NMR, or both modalities combined as input: "we introduce a multitask machine learning framework that predicts the molecular structure (formula and connectivity) of an unknown compound solely based on its 1D ¹H and/or ¹³C NMR spectra"
- [intro] transformer architecture assembles molecular fragments into structures; CNN encodes spectra: "a transformer architecture can be constructed to efficiently solve the task, traditionally performed by chemists, of assembling large numbers of molecular fragments into molecular structures"
- [intro] CNN and transformer integrated for end-to-end prediction: "Integrating this capability with a convolutional neural network, we build an end-to-end model for predicting structure from spectra"
- [intro] 1D NMR structure elucidation is challenging due to combinatorial explosion: "elucidating structure using only one-dimensional (1D) NMR spectra, the most readily accessible data, remains an extremely challenging problem because of the combinatorial explosion"
- [intro] Framework tested on molecules up to 19 heavy atoms: "We demonstrate the effectiveness of this framework on molecules with up to 19 heavy (non-hydrogen) atoms"
1---2name: nmr-modality-ablation-analysis3description: Use when you have a trained multitask machine learning model for structure prediction, test set molecules with paired ¹H and ¹³C NMR spectra, and need to understand the marginal contribution of each NMR modality or justify multimodal input design.4license: CC-BY-4.05---67# nmr-modality-ablation-analysis89## Summary1011Systematically evaluate the individual and joint contributions of ¹H NMR, ¹³C NMR, and combined 1D NMR inputs to molecular structure prediction accuracy by running inference under each modality condition independently and comparing performance metrics. This skill isolates which spectroscopic modalities drive structure elucidation performance and quantifies synergy between them.1213## When to use1415Apply this skill when you have a trained multitask machine learning model for structure prediction, test set molecules with paired ¹H and ¹³C NMR spectra, and need to understand the marginal contribution of each NMR modality or justify multimodal input design. Use it when stakeholders ask 'which NMR experiment is actually necessary?' or when you suspect one modality is redundant with the other.1617## When NOT to use1819- Input lacks paired ¹H and ¹³C NMR spectra for the same molecules (modality ablation requires both modalities present to be meaningful).20- The model is not yet trained or checkpoint is not available; ablation analysis requires a frozen, fixed model.21- Goal is to optimize model architecture or hyperparameters; this skill measures performance, not tunes the model.2223## Inputs2425- Trained NMR2Struct model checkpoint (weights and architecture)26- Test set molecular structures (SMILES or graph representation)27- ¹H NMR spectra (CNN-compatible tensor, e.g., shape [batch, channels, frequency_bins])28- ¹³C NMR spectra (CNN-compatible tensor, same shape as ¹H)29- Ground truth molecular formula and connectivity labels3031## Outputs3233- Per-modality accuracy metrics (exact-match formula accuracy, structure connectivity F1 or graph edit distance)34- Modality-specific error distribution (confusion matrix by atom type, bond type, molecular weight bins)35- Synergy quantification (combined accuracy minus single-modality baseline)36- Tabulated results by modality showing accuracy, error gaps, and per-molecule predictions37- Comparison summary (which modality(s) dominate for different structural classes)3839## How to apply4041Load the trained NMR2Struct model checkpoint and prepare the test set with aligned molecular ground truth labels and corresponding ¹H and ¹³C NMR spectra. Create three separate inference batches: (1) ¹H-only spectra masked to zeros in ¹³C channels, (2) ¹³C-only spectra masked to zeros in ¹H channels, and (3) both modalities combined. Run the convolutional neural network encoder and transformer decoder on each batch independently, generating molecular formula and connectivity predictions for each condition. Compute accuracy metrics (exact-match accuracy for formula, graph edit distance or connectivity F1 for structure) separately for each modality. Quantify synergy by subtracting the maximum single-modality accuracy from the combined accuracy; compare error distributions across conditions to identify which structural features (connectivity, functional groups, molecular weight) each modality resolves best. Tabulate results by modality with accuracy, confidence intervals, and per-molecule error analysis.4243## Related tools4445- **Convolutional Neural Network (CNN) encoder** (Encodes raw 1D NMR spectra (¹H and/or ¹³C) into learned feature representations for transformer input)46- **Transformer architecture** (Decodes CNN feature representations and assembled molecular fragments into predicted structure (formula and connectivity))47- **NMR2Struct multitask framework** (End-to-end model combining CNN encoder and transformer decoder for structure prediction from 1D NMR)4849## Evaluation signals5051- Exact-match accuracy for predicted molecular formula matches ground truth for ≥80% of test molecules in single-modality and combined conditions.52- Synergy value (combined minus best single-modality accuracy) is non-negative and quantifies the benefit of multimodal input; if synergy ≈ 0, modalities are redundant.53- Error distribution differs qualitatively across modalities (e.g., ¹H alone fails on isomers with identical ¹H shifts but different ¹³C shifts, while ¹³C alone struggles with quaternary carbons); this confirms complementarity.54- Graph edit distance or connectivity F1 score for structure prediction increases monotonically from single modalities to combined input across molecule classes.55- Per-molecule predictions show same connectivity for all three conditions on molecules where the model achieves high confidence; large disagreements flag ambiguous spectra or modality-specific artifacts.5657## Limitations5859- Ablation measures marginal contribution but does not explain which spectroscopic features (e.g., splitting patterns, chemical shift ranges) drive predictions; interpretation requires additional feature attribution analysis.60- Performance reported on molecules with up to 19 heavy atoms; ablation results may not generalize to larger structures or rare functional groups outside the training distribution.61- Masking one modality to zeros assumes independence; in a real multitask model, the two modalities may share learned representations in early layers, so true single-modality performance may differ from zero-masking.62- Exact-match accuracy is a strict metric; two predictions differing by a single atom or bond will be marked as failure, potentially underestimating practical utility of lower-accuracy modality combinations.6364## Evidence6566- [intro] multitask machine learning framework capable of predicting molecular structure using 1D ¹H NMR, ¹³C NMR, or both modalities combined as input: "we introduce a multitask machine learning framework that predicts the molecular structure (formula and connectivity) of an unknown compound solely based on its 1D ¹H and/or ¹³C NMR spectra"67- [intro] transformer architecture assembles molecular fragments into structures; CNN encodes spectra: "a transformer architecture can be constructed to efficiently solve the task, traditionally performed by chemists, of assembling large numbers of molecular fragments into molecular structures"68- [intro] CNN and transformer integrated for end-to-end prediction: "Integrating this capability with a convolutional neural network, we build an end-to-end model for predicting structure from spectra"69- [intro] 1D NMR structure elucidation is challenging due to combinatorial explosion: "elucidating structure using only one-dimensional (1D) NMR spectra, the most readily accessible data, remains an extremely challenging problem because of the combinatorial explosion"70- [intro] Framework tested on molecules up to 19 heavy atoms: "We demonstrate the effectiveness of this framework on molecules with up to 19 heavy (non-hydrogen) atoms"
HolobiomicsLab