Peak Detection Accuracy Assessment

peak-detection-accuracy-assessment

Summary

Validates the accuracy and completeness of chromatographic peak detection in non-targeted metabolomics preprocessing by comparing detected peaks against a gold-standard benchmark dataset with known molecular composition, retention time boundaries, and isotopologue information. This skill quantifies detection rate, isotopologue ratio fidelity, and peak quality to benchmark the reliability of different preprocessing tools (XCMS, MZmine 2, MS-DIAL, etc.).

When to use

Apply this skill after running a non-targeted preprocessing (NPP) pipeline (e.g., XCMS, MZmine 2, OpenMS) on the same set of centroided mzML files that were used to generate a validated benchmark dataset. You want to measure what fraction of known peaks the NPP tool successfully detected, whether isotopologue ratios remain unbiased (within 30% of predicted), and whether detected peaks exhibit acceptable chromatographic quality (correlation ≥ 0.85 to the most abundant isotopologue).

When NOT to use

• NPP output is from a different set of mzML files than the benchmark; matching will be unreliable.
• Target molecules lack known retention time boundaries (user.rtmin/user.rtmax); benchmark generation will fail or produce low-confidence peaks.
• Input mzML files are not centroided; peak extraction and isotopologue detection will be inaccurate.
• You are only assessing one NPP tool on one dataset without a validated reference; you need a benchmark to enable relative comparison.

Inputs

• Centroided mzML files (same set used for benchmark generation)
• Benchmark dataset CSV (generated by mzRAPP, containing 47 molecules, 157 features, 2870 peaks for MTBLS267 example)
• NPP output table (peak/feature table from XCMS, MZmine 2, MS-DIAL, or equivalent, with m/z, retention time, intensity columns)
• Target file CSV (molecule name, molecular composition SumForm_c, main_adduct, user.rtmin, user.rtmax)
• Sample group file CSV (sample_name, sample_group columns)

Outputs

• Peak detection performance metrics (% detected, % not-found, % split peaks)
• Isotopologue ratio (IR) metric table (% degenerated IR per sample or tool)
• Missing value classification (reason for non-detection: missed by NPP, failed isotopologue validation, etc.)
• Alignment error count (peaks with retention time or m/z drift beyond tolerance)
• Per-tool and per-sample comparison report (e.g., XCMS run 1 vs. run 3 vs. MZmine 2)

How to apply

First, ensure you have generated a benchmark dataset from target molecules with known molecular formulas, retention time boundaries, and main adducts (e.g., M+H or M-H) by running mzRAPP on the same mzML files. The benchmark extraction filters isotopologues to only those where the theoretically most abundant and at least one additional isotopologue are found, and removes peaks with area/height deviation >30% from predicted values or Pearson correlation <0.85 relative to the highest isotopologue. Then, export the NPP output (peak table or feature list with m/z, retention time, and intensity values) and import both the benchmark and NPP output into mzRAPP's assessment module. The tool matches peaks between benchmark and NPP output by m/z and retention time, then calculates detection rate (found vs. not-found peaks), classification of missing values (e.g., missed by NPP vs. failed isotopologue validation), degenerated isotopologue ratio frequency (IR-metric), and alignment errors. Compare observed performance metrics against acceptable thresholds: aim for ≥90% peak detection rate and <20% degenerated isotopologue ratio in high-quality preprocessing runs.

Related tools

• mzRAPP (Benchmark generation and NPP reliability assessment; orchestrates peak extraction from mzML files, isotopologue prediction (via enviPat), and comparison of NPP outputs against benchmark to compute detection, alignment, and isotopologue quality metrics.) — https://github.com/YasinEl/mzRAPP
• enviPat (Predicts isotopologue patterns and masses for target molecules given their molecular formula and selected adducts; mzRAPP uses these predictions to define which isotopologues to extract and validate.)
• XCMS (Example non-targeted preprocessing tool being assessed; mzRAPP quantifies its peak detection accuracy, isotopologue ratio bias, and alignment performance.)
• MZmine 2 (Example non-targeted preprocessing tool being assessed; mzRAPP quantifies its peak detection accuracy and isotopologue ratio bias.)
• Skyline (Manual peak curation tool from which retention time boundaries (user.rtmin/user.rtmax) can be exported and imported into mzRAPP's target file.)
• R (Programming environment for running mzRAPP via library(mzRAPP) and for scripted benchmark generation and assessment workflows.)

Examples

library(mzRAPP); callmzRAPP()

Evaluation signals

• Benchmark dataset row count matches expected: 47 unique molecules, 157 unique features (molecule + adduct + isotopologue combinations), 2870 total peaks (all chromatographic peak detections) for MTBLS267 reference.
• Peak detection rate ≥ 83–94% for XCMS and ≥ 82–92% for MZmine 2 indicates acceptable NPP performance; <80% suggests preprocessing issues.
• Degenerated isotopologue ratio (IR) metric ≤ 20% is high-quality preprocessing; > 30% indicates significant isotopic pattern distortion.
• Alignment error count is low (few peaks with m/z or retention time drift beyond the defined tolerance); high error count indicates poor NPP feature alignment.
• Matched peak m/z and retention time fall within defined tolerance windows (default 5 ppm, ±5% of user-provided peak boundaries); unmatched peaks are logged as missed detections.

Limitations

• Benchmark quality depends critically on the accuracy of user-provided retention time boundaries (user.rtmin/user.rtmax); poor curation limits assessment reliability.
• Only isotopologues for which both the theoretically most abundant and at least one additional isotopologue are detected are retained; sparse or low-intensity isotopologue patterns may be excluded, reducing the completeness of the benchmark.
• Isotopologue peaks with area/height deviation > 30% from predicted or Pearson correlation < 0.85 are filtered out; this threshold may exclude real but weak isotopologues in low-abundance compounds.
• mzRAPP requires centroided mzML files; profile-mode or uncalibrated data will lead to inaccurate isotopologue and adduct matching.
• Assessment is tool-specific and dataset-specific; performance metrics from one NPP tool or study cohort may not generalize to other tools, ionization modes, or metabolite classes.

Evidence

• [methods] benchmark dataset containing 47 different molecules with 157 different features (including all adducts and isotopologues), resulting in 2870 peaks: "If you processed all 30 mzML files, you should have generated a benchmark containing 47 different molecules with 157 different features (including all adducts and isotopologues), resulting in 2870"
• [methods] isotopologue filtering by peak shape correlation and abundance ratio bias: "Sufficient quality of low peaks was ensured by removing isotopologues that do not satisfy criteria in peak shape (peak shape correlation with most abundant isotopologue)"
• [methods] isotopologue ratio bias < 30% and Pearson correlation cutoff: "removing isotopologues that do not satisfy criteria in peak shape and abundance (Isotopologue ratio bias < 30%)"
• [methods] XCMS and MZmine 2 performance comparison: "XCMS run 1 detected 83-94% of peaks with 28-53% degenerated isotopologue ratio"
• [readme] mzRAPP's assessment purpose and scope: "The goal of mzRAPP is to allow reliability assessment of non-targeted data pre-processing (NPP; XCMS, XCMS3, MetaboanalystR 3.0, SLAW, XCMS-online, MZmine 2, MZmine 3, MS-DIAL, OpenMS, El-MAVEN,..)"
• [readme] enviPat isotopologue prediction in mzRAPP: "mzRAPP extracts and validates chromatographic peaks for which boundaries are provided for all (enviPat predicted) isotopologues of those target molecules directly from mzML files"