peak-list-filtering-and-cleaning
Summary
Remove electronic and chemical noise ions from MS/MS peak lists to improve spectral quality and compound identification confidence. This skill applies domain-specific filters (intensity-based for electronic noise, formula-based for chemical noise) to a 2D array of m/z and intensity values, returning a deduplicated, curated peak list suitable for library matching.
When to use
Apply this skill when you have acquired MS/MS spectra containing suspect noise ions—either electronic noise (ions with identical intensities occurring >4 times in a single peak list, a signature of detector artifacts) or chemical noise (fragment ions chemically implausible given the precursor molecule's structure). Use it before spectral library searching or when entropy similarity between replicate analyses is low.
When NOT to use
- Input spectrum has already been denoised or validated against a reference standard—applying the filter again may remove legitimate low-abundance peaks.
- Peak list contains fewer than 5 peaks total—the intensity-frequency threshold (>4 occurrences) is calibrated for typical-sized spectra and may not be meaningful for sparse data.
- Chemical structure (SMILES or formula) is unknown or unreliable—formula denoising cannot proceed without accurate precursor composition.
Inputs
- 2D numpy array with shape [n, 2] containing m/z (column 0) and intensity (column 1) values
- SMILES string or chemical formula (for formula denoising)
- Ion adduct (e.g., '[M+H]+', '[M+Na]+') (for formula denoising)
Outputs
- 2D numpy array with shape [m, 2] (m ≤ n) containing deduplicated m/z and intensity values
- Denoised peak list suitable for spectral library matching
How to apply
Load your peak list as a 2D numpy array with shape [n, 2] where columns are m/z and intensity. Apply electronic denoising first by counting frequency of each unique intensity value and filtering peaks whose intensity occurs more than 4 times (an empirically validated threshold from NIST23 where such occurrences are <0.05% in genuine spectra). Then apply formula denoising by (1) preparing the master formula from SMILES and adduct information; (2) generating all chemically plausible subformula losses; (3) checking whether each observed peak corresponds to a valid fragment; (4) removing peaks without plausible assignments. Finally, sanitize the spectrum by removing zero-intensity ions and sorting by m/z. Return the deduplicated array in the same [n, 2] shape as input.
Related tools
- spectral_denoising Python package (Implements electronic_denoising, formula_denoising, and spectral_denoising_batch functions for noise removal) — https://github.com/FanzhouKong/spectral_denoising
- numpy (Core array manipulation for loading, filtering, and returning peak lists)
- RDkit (Parses SMILES strings and computes chemical formulas for formula denoising)
- ms_entropy (Computes spectral entropy and entropy similarity to evaluate denoising effectiveness)
Examples
import numpy as np
import spectral_denoising as sd
peak = np.array([[48.99, 154.0], [63.01, 265.0], [79.02, 521.0]], dtype=np.float32)
peak_denoised = sd.electronic_denoising(peak)
peak_denoised = sd.formula_denoising(peak_denoised, 'O=c1nc[nH]c2nc[nH]c12', '[M+Na]+')
Evaluation signals
- Intensity-frequency histogram post-denoising shows no intensity value occurring >4 times (electronic noise removed).
- Peak count decreases from input to output (m < n), confirming noise removal; verify the removed peaks were either high-frequency intensities or chemically implausible fragments.
- Entropy similarity between denoised spectrum and a clean reference improves relative to the raw spectrum (computed via entropy_similarity function).
- Denoised spectrum matches expected fragment pattern for the precursor molecule (e.g., common neutral losses like H2O, CO2 are present; artifact peaks are absent).
- Output array is sorted by m/z, contains no zero-intensity rows, and all m/z and intensity values are non-negative floats or integers within expected mass and intensity ranges.
Limitations
- Electronic denoising threshold (>4 identical intensities) is empirically validated on NIST23 but may need adjustment for other instrument types, acquisition methods, or noise profiles not represented in NIST23.
- Formula denoising requires accurate SMILES or chemical formula input; misspecified precursor composition will cause false positives (legitimate fragments labeled as noise) or false negatives (noise not removed).
- Does not handle 'chemical noise' arising from in-source fragmentation, solvent adducts, or multiply-charged ions unless the precursor adduct is correctly specified.
- Spectral entropy and similarity metrics are relative; denoising success is best judged in conjunction with independent validation (e.g., replicate analysis, library search improvement) rather than entropy alone.
Evidence
- [other] The electronic_denoising function removes obvious electronic noise ions in MS/MS spectra: "The
electronic_denoising function removes obvious electronic noise ions in MS/MS spectra"
- [other] Identify intensity values that occur more than 4 times (threshold empirically validated on NIST23): "Identify intensity values that occur more than 4 times (a threshold empirically validated on NIST23 database where such occurrences are <0.05% in genuine spectra)"
- [other] Load a peak list as a 2D numpy array with shape [n, 2] containing m/z and intensity columns: "Load a peak list as a 2D numpy array with shape [n, 2] containing m/z and intensity columns"
- [intro] Noise ions in MS/MS spectra are largely categorized as electronic noises and chemical noises: "Noise ions in MS/MS spectra are largely categorized as 1. electronic noises and 2. chemical noises."
- [other] The formula_denoising function removes chemical noise ions by evaluating if a fragment could be formed from a chemically plausible subformula loss: "The
formula_denoising function removes chemical noise ions in MS/MS spectra by evaluating if it could be formed from a chemically plausible subformula loss"
- [other] sanitize_spectrum is a wrapper function that removes zero-intensity ions and sorts the spectrum by mass: "
spectral_operations.sanitize_spectrum is just a wrapper function for spectral_operations.remove_zero_ions and spectral_operations.sort_spectrum"
- [readme] Python version requirement for spectral denoising: "Spectral denoising requires
Python >= 3.8 installed on your system"
1---2name: peak-list-filtering-and-cleaning3description: Use when you have acquired MS/MS spectra containing suspect noise ions—either electronic noise (ions with identical intensities occurring >4 times in a single peak list, a signature of detector artifacts) or chemical noise (fragment ions chemically implausible given the precursor molecule's.4license: CC-BY-4.05---67# peak-list-filtering-and-cleaning89## Summary1011Remove electronic and chemical noise ions from MS/MS peak lists to improve spectral quality and compound identification confidence. This skill applies domain-specific filters (intensity-based for electronic noise, formula-based for chemical noise) to a 2D array of m/z and intensity values, returning a deduplicated, curated peak list suitable for library matching.1213## When to use1415Apply this skill when you have acquired MS/MS spectra containing suspect noise ions—either electronic noise (ions with identical intensities occurring >4 times in a single peak list, a signature of detector artifacts) or chemical noise (fragment ions chemically implausible given the precursor molecule's structure). Use it before spectral library searching or when entropy similarity between replicate analyses is low.1617## When NOT to use1819- Input spectrum has already been denoised or validated against a reference standard—applying the filter again may remove legitimate low-abundance peaks.20- Peak list contains fewer than 5 peaks total—the intensity-frequency threshold (>4 occurrences) is calibrated for typical-sized spectra and may not be meaningful for sparse data.21- Chemical structure (SMILES or formula) is unknown or unreliable—formula denoising cannot proceed without accurate precursor composition.2223## Inputs2425- 2D numpy array with shape [n, 2] containing m/z (column 0) and intensity (column 1) values26- SMILES string or chemical formula (for formula denoising)27- Ion adduct (e.g., '[M+H]+', '[M+Na]+') (for formula denoising)2829## Outputs3031- 2D numpy array with shape [m, 2] (m ≤ n) containing deduplicated m/z and intensity values32- Denoised peak list suitable for spectral library matching3334## How to apply3536Load your peak list as a 2D numpy array with shape [n, 2] where columns are m/z and intensity. Apply electronic denoising first by counting frequency of each unique intensity value and filtering peaks whose intensity occurs more than 4 times (an empirically validated threshold from NIST23 where such occurrences are <0.05% in genuine spectra). Then apply formula denoising by (1) preparing the master formula from SMILES and adduct information; (2) generating all chemically plausible subformula losses; (3) checking whether each observed peak corresponds to a valid fragment; (4) removing peaks without plausible assignments. Finally, sanitize the spectrum by removing zero-intensity ions and sorting by m/z. Return the deduplicated array in the same [n, 2] shape as input.3738## Related tools3940- **spectral_denoising Python package** (Implements electronic_denoising, formula_denoising, and spectral_denoising_batch functions for noise removal) — https://github.com/FanzhouKong/spectral_denoising41- **numpy** (Core array manipulation for loading, filtering, and returning peak lists)42- **RDkit** (Parses SMILES strings and computes chemical formulas for formula denoising)43- **ms_entropy** (Computes spectral entropy and entropy similarity to evaluate denoising effectiveness)4445## Examples4647```48import numpy as np49import spectral_denoising as sd50peak = np.array([[48.99, 154.0], [63.01, 265.0], [79.02, 521.0]], dtype=np.float32)51peak_denoised = sd.electronic_denoising(peak)52peak_denoised = sd.formula_denoising(peak_denoised, 'O=c1nc[nH]c2nc[nH]c12', '[M+Na]+')53```5455## Evaluation signals5657- Intensity-frequency histogram post-denoising shows no intensity value occurring >4 times (electronic noise removed).58- Peak count decreases from input to output (m < n), confirming noise removal; verify the removed peaks were either high-frequency intensities or chemically implausible fragments.59- Entropy similarity between denoised spectrum and a clean reference improves relative to the raw spectrum (computed via entropy_similarity function).60- Denoised spectrum matches expected fragment pattern for the precursor molecule (e.g., common neutral losses like H2O, CO2 are present; artifact peaks are absent).61- Output array is sorted by m/z, contains no zero-intensity rows, and all m/z and intensity values are non-negative floats or integers within expected mass and intensity ranges.6263## Limitations6465- Electronic denoising threshold (>4 identical intensities) is empirically validated on NIST23 but may need adjustment for other instrument types, acquisition methods, or noise profiles not represented in NIST23.66- Formula denoising requires accurate SMILES or chemical formula input; misspecified precursor composition will cause false positives (legitimate fragments labeled as noise) or false negatives (noise not removed).67- Does not handle 'chemical noise' arising from in-source fragmentation, solvent adducts, or multiply-charged ions unless the precursor adduct is correctly specified.68- Spectral entropy and similarity metrics are relative; denoising success is best judged in conjunction with independent validation (e.g., replicate analysis, library search improvement) rather than entropy alone.6970## Evidence7172- [other] The electronic_denoising function removes obvious electronic noise ions in MS/MS spectra: "The ``electronic_denoising`` function removes obvious electronic noise ions in MS/MS spectra"73- [other] Identify intensity values that occur more than 4 times (threshold empirically validated on NIST23): "Identify intensity values that occur more than 4 times (a threshold empirically validated on NIST23 database where such occurrences are <0.05% in genuine spectra)"74- [other] Load a peak list as a 2D numpy array with shape [n, 2] containing m/z and intensity columns: "Load a peak list as a 2D numpy array with shape [n, 2] containing m/z and intensity columns"75- [intro] Noise ions in MS/MS spectra are largely categorized as electronic noises and chemical noises: "Noise ions in MS/MS spectra are largely categorized as 1. electronic noises and 2. chemical noises."76- [other] The formula_denoising function removes chemical noise ions by evaluating if a fragment could be formed from a chemically plausible subformula loss: "The ``formula_denoising`` function removes chemical noise ions in MS/MS spectra by evaluating if it could be formed from a chemically plausible subformula loss"77- [other] sanitize_spectrum is a wrapper function that removes zero-intensity ions and sorts the spectrum by mass: "``spectral_operations.sanitize_spectrum`` is just a wrapper function for ``spectral_operations.remove_zero_ions`` and ``spectral_operations.sort_spectrum``"78- [readme] Python version requirement for spectral denoising: "Spectral denoising requires ``Python >= 3.8`` installed on your system"
HolobiomicsLab 