# Baseline Model Implementation For Comparison

> Use when you are introducing a novel spectrum prediction model and need to demonstrate that performance improvements come from architectural innovation rather than experimental advantage.

- Skill: `holobiomicslab/baseline-model-implementation-for-comparison` (Agent Skill)
- Install (CLI): `npx skillmds@latest add holobiomicslab/baseline-model-implementation-for-comparison`
- Raw SKILL.md: https://api.skillmd.com/api/skills/holobiomicslab/baseline-model-implementation-for-comparison/raw
- Safety review: PASS (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- License: CC-BY-4.0
- Author: HolobiomicsLab (https://skillmd.com/u/holobiomicslab)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/holobiomicslab/baseline-model-implementation-for-comparison

---


# baseline-model-implementation-for-comparison

## Summary

Implement reference baseline models (e.g., NEIMS with FFN and GNN encoders) under strictly equivalent settings—same covariates, hyperparameter sweeps, train/test splits—to enable fair quantitative comparison with novel spectrum prediction methods. This skill ensures reproducible benchmarking and isolates model architectural differences from confounding experimental factors.

## When to use

You are introducing a novel spectrum prediction model and need to demonstrate that performance improvements come from architectural innovation rather than experimental advantage. Baseline implementation is required when comparing across NEIMS, MassFormer, 3DMolMS, GrAFF-MS, or CFM-ID on the same dataset (e.g., NIST'20) with tandem mass spectrum prediction as the evaluation task.

## When NOT to use

- Input dataset or splits differ from the novel model's (comparison requires identical data provenance and preprocessing).
- Hyperparameter sweeps for the baseline are not exhaustively identical to the novel model (unfair advantage to one method).
- Baseline source code or paper is unavailable or ambiguously specified (use only published methods with reproducible details).

## Inputs

- Molecular structures as SMILES strings or chemical graphs (PubChem records)
- Experimental tandem mass spectra in standardized format (.SDF, .mgf, .hdf5)
- Dataset split specification (train/validation/test indices with seed)
- Hyperparameter sweep configuration (learning rate, batch size, encoder hidden dims, num_layers)

## Outputs

- Trained baseline model checkpoint (.pt or equivalent)
- Spectrum predictions on test set (m/z, intensity tuples or ranking scores)
- Evaluation metrics (top-1/top-k retrieval accuracy, spectral similarity, cosine distance)
- Metadata log (hyperparameters, random seed, training time, hardware used)

## How to apply

Select a reference baseline architecture from the literature (e.g., NEIMS FFN or GNN encoder from Goldman et al. 2009). Load the molecular dataset and chemical structures from PubChem or a commercial source (NIST'20) with associated experimental mass spectra in standardized format (.SDF or .hdf5). Construct the encoder (FFN or GNN) and train end-to-end on the spectrum prediction task using the exact same dataset splits, hyperparameter sweep grid, learning rate schedule, and batch settings as your novel model. Evaluate on the held-out test set computing prediction accuracy and comparison metrics (e.g., top-1/top-k retrieval accuracy for structural elucidation, spectral similarity cosine scores). Save model weights, predictions, and evaluation metrics to output artifacts. Document all hyperparameters and random seeds (e.g., split_1_rnd1) to enable replication.

## Related tools

- **PubChem** (Source of molecular structures and chemical formula lookups for dataset construction and retrieval experiments)
- **ms-pred (coleygroup)** (Reference implementation of NEIMS and other baseline models under equivalent-settings framework for spectrum prediction) — https://github.com/coleygroup/ms-pred
- **MAGMa** (Annotation tool for generating fragmentation trees and substructure labels used in baseline and novel model training pipelines)

## Evaluation signals

- Hyperparameter sweep configuration is identical between baseline and novel model (same ranges, grid resolution, learning rate schedules).
- Train/validation/test split indices and random seeds are reproducible and identical across both implementations.
- Baseline model achieves published or expected performance on the reference test set (e.g., NIST'20 top-1 accuracy ~30–40% for NEIMS GNN).
- Evaluation metrics (top-k retrieval accuracy, cosine similarity scores) are computed on the same held-out test set using identical metric definitions.
- Model checkpoint, predictions, and metadata are saved and can be reloaded to reproduce results without retraining.

## Limitations

- NIST'20 is a commercial dataset requiring purchase; open-source alternatives (MassSpecGym) have less manual curation and yield different results.
- Baseline code availability and clarity vary; some published methods (e.g., CFM-ID 4.0) are not retrained in the comparison—only reference results are provided.
- GPU memory and training time scale with dataset size and hyperparameter sweep breadth; smaller GPUs (<24 GB) require reduced batch sizes and may not replicate published results.
- Contrastive finetuning steps (e.g., for ICEBERG intensity models) are optional and affect final performance; equivalent-settings comparison requires explicit choices about which optional steps to include.

## Evidence

- [readme] In order to fairly compare various spectra models, we implement a number of baselines and alternative models using equivalent settings across models (i.e., same covariates, hyperparameter sweeps for each, etc.): "In order to fairly compare various spectra models, we implement a number of baselines and alternative models using equivalent settings across models (i.e., same covariates, hyperparameter sweeps for"
- [other] NEIMS baseline is implemented with both FFN and GNN encoder variants as part of an equivalent-settings comparison framework where all models use the same covariates and hyperparameter sweeps.: "NEIMS baseline is implemented with both FFN and GNN encoder variants as part of an equivalent-settings comparison framework where all models use the same covariates and hyperparameter sweeps."
- [other] Construct a graph neural network (GNN) encoder to learn molecular representations from chemical graphs. 3. Train the GNN encoder end-to-end on the spectrum prediction task using the dataset split and hyperparameters specified for the baseline comparison. 4. Evaluate the trained GNN-encoder NEIMS model on the held-out test set, computing prediction accuracy and comparison metrics against ms-pred reference results.: "Construct a graph neural network (GNN) encoder to learn molecular representations from chemical graphs. 3. Train the GNN encoder end-to-end on the spectrum prediction task using the dataset split and"
- [other] Load the molecular dataset and chemical structures from PubChem with associated experimental mass spectra.: "Load the molecular dataset and chemical structures from PubChem with associated experimental mass spectra."
- [readme] 1. *NEIMS* using both FFN and GNN encoders from Rapid prediction of electron–ionization mass spectrometry using neural networks: "1. *NEIMS* using both FFN and GNN encoders from Rapid prediction of electron–ionization mass spectrometry using neural networks"