# Spectrum Preprocessing Binning Normalization

> Use when when preparing raw MS/MS spectra for input to a Siamese neural network trained to predict structural similarity scores (Tanimoto).

- Skill: `holobiomicslab/spectrum-preprocessing-binning-normalization` (Agent Skill)
- Install (CLI): `npx skillmds@latest add holobiomicslab/spectrum-preprocessing-binning-normalization`
- Raw SKILL.md: https://api.skillmd.com/api/skills/holobiomicslab/spectrum-preprocessing-binning-normalization/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- License: CC-BY-4.0
- Author: HolobiomicsLab (https://skillmd.com/u/holobiomicslab)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/holobiomicslab/spectrum-preprocessing-binning-normalization

---


# spectrum-preprocessing-binning-normalization

## Summary

Preprocesses tandem mass spectra by removing low-intensity peaks, applying intensity transformations, and binning peak data into fixed m/z intervals to prepare inputs for deep learning similarity prediction. This standardization ensures consistent feature representation across spectra of varying quality and intensity distributions.

## When to use

When preparing raw MS/MS spectra for input to a Siamese neural network trained to predict structural similarity scores (Tanimoto). Apply this skill before computing spectral embeddings or training a deep learning model on spectrum pairs, particularly when spectra have variable peak counts, intensity ranges, or quality levels that could bias learning.

## When NOT to use

- Input spectra are already in pre-binned or normalized vector form (e.g., from a prior preprocessing step).
- The analysis requires preservation of the original peak intensities or m/z values for downstream interpretation (e.g., fragment assignment or peak annotation).
- Spectra come from instruments or methods where the 10–1000 m/z range does not cover the relevant chemical space (e.g., intact protein MS, ion mobility experiments).

## Inputs

- Raw MS/MS spectrum data (peaks with m/z and intensity values)
- Spectrum metadata (e.g., maximum peak intensity per spectrum)

## Outputs

- Preprocessed spectrum vectors (10,000-dimensional binned intensity arrays)
- Optionally: filtered peak lists before binning

## How to apply

Remove peaks with intensities below 0.1% of the maximum peak intensity in each spectrum, then retain only the 1,000 highest-intensity peaks to reduce noise and excessive dimensionality. Apply square-root transformation to peak intensities to reduce bias toward the highest peaks and improve feature balance. Finally, bin the transformed peaks into 10,000 equally-sized m/z intervals spanning 10–1000 m/z. This produces a fixed-length feature vector per spectrum suitable for neural network input. The square-root transformation is critical because it prevents the model from over-weighting dominant peaks; the binning converts variable-length peak lists into dense, comparable vectors.

## Related tools

- **matchms** (Spectrum data cleaning, metadata extraction, and filtering pipeline; applies intensity thresholds and peak count limits) — https://github.com/matchms/matchms
- **RDKit** (Generates molecular fingerprints (Daylight, 2048 bits) from chemical structures to compute ground-truth Tanimoto scores for training labels)
- **MS2DeepScore** (Siamese network that receives preprocessed binned spectra as input and outputs structural similarity predictions) — https://github.com/matchms/ms2deepscore

## Examples

```
from matchms.Pipeline import Pipeline, create_workflow
from matchms.filtering.default_pipelines import DEFAULT_FILTERS
from ms2deepscore import MS2DeepScore
from ms2deepscore.models import load_model

model = load_model('ms2deepscore_model.pt')
pipeline = Pipeline(create_workflow(query_filters=DEFAULT_FILTERS, score_computations=[[MS2DeepScore, {'model': model}]]))
report = pipeline.run('spectra.mgf')
preprocessed_spectra = pipeline.spectra_queries
```

## Evaluation signals

- Output vectors have shape (n_spectra, 10000) with no missing values after binning.
- Peak intensity values are in the range [0, sqrt(max_original_intensity)] after square-root transformation.
- All spectra with more than 1,000 peaks are correctly truncated; those with fewer peaks retain their non-zero bins.
- Peaks below 0.1% of maximum intensity are removed; verify by checking that minimum non-zero intensity > 0.001 × max_intensity.
- RMSE on held-out test set matches reported values (~0.15 without uncertainty filtering, ~0.10 with IQR < 0.025) when preprocessed spectra are fed to the trained model.

## Limitations

- The fixed 10–1000 m/z binning range may lose information from spectra with significant peaks outside this range (e.g., high-mass fragments, small molecular ions).
- Square-root transformation reduces but does not eliminate bias toward high-intensity peaks; very weak peaks may be further suppressed and lose discriminative power.
- Binning into 10,000 fixed intervals can blur fine m/z distinctions near the detection limit; spectra with very sparse peaks may have insufficient feature density.
- The 0.1% intensity threshold and 1,000-peak limit are fixed hyperparameters; data with atypical noise profiles or peak distributions may require empirical re-tuning.
- No explicit handling of instrument artifacts, isotope peaks, or adducts; preprocessing assumes prior removal of such features or tolerance to their presence.

## Evidence

- [methods] The spectra underwent basic filtering to remove excessive amounts of peaks, by removing peaks with intensities < 0.1% of the maximum peak intensity and limiting the maximum number of peaks to the 1000 highest intensity peaks: "removing peaks with intensities < 0.1% of the maximum peak intensity and limiting the maximum number of peaks to the 1000 highest intensity peaks"
- [methods] Peak intensities were square root transformed to avoid a too strong focus on the highest intensity peaks only: "Peak intensities were square root transformed to avoid a too strong focus on the highest intensity peaks only"
- [methods] Spectrum peaks were binned in 10,000 equally-sized bins ranging from 10 to 1000 m/z: "Spectrum peaks were binned in 10,000 equally-sized bins ranging from 10 to 1000 m/z"
- [other] Prepare test spectra by applying the same preprocessing: remove peaks with intensity < 0.1% of maximum, keep top 1,000 peaks, apply square-root transformation to intensities, and bin into 10,000 equally-sized bins (10–1000 m/z).: "remove peaks with intensity < 0.1% of maximum, keep top 1,000 peaks, apply square-root transformation to intensities, and bin into 10,000 equally-sized bins"
- [methods] Metadata was cleaned and checked using matchms [18] version 0.8.2, which included cleaning compound names, extracting adduct information from the given metadata, moving metadata to consistent fields: "Metadata was cleaned and checked using matchms [18] version 0.8.2, which included cleaning compound names, extracting adduct information"