false-discovery-rate-interpretation
Summary
Interpretation of false discovery rates (FDR) from database search pipelines to distinguish genuine natural product identifications from spurious matches in mass spectrometry data. This skill involves extracting FDR values from pipeline outputs and using them as a quality filter to rank and validate spectral matches.
When to use
When you have executed database search pipelines (Dereplicator, VarQuest, or Dereplicator+) on centroided LC-MS/MS spectra in MGF format and obtained match results with associated p-values and false discovery rates. Use this skill when you need to discriminate between high-confidence and low-confidence identifications across multiple pipelines or search modes (e.g., standard vs. blind/PTM-tolerant searches).
When NOT to use
- Input spectra are not centroided or are in unsupported formats (non-MGF/mzML/mzXML/mzData).
- Database search has not yet been executed; raw spectra alone without match results cannot be analyzed for FDR.
- FDR values are not calculated by the pipeline or are missing from output files.
Inputs
- significant_matches.tsv files from MetaMiner, Dereplicator, VarQuest, or Dereplicator+
- Scan identifiers from LC-MS/MS spectra (MGF or mzML format)
- Match scores from database search results
- P-values from spectral matching
- False discovery rate values per match
Outputs
- Tabulated comparison of FDR values across pipelines
- Filtered match sets ranked by FDR threshold
- Cross-pipeline hit pattern comparison matrix
- Tool-specific and common identification sets
- FDR-stratified sensitivity analysis by search mode
How to apply
Parse the significant_matches.tsv output files generated by each database search pipeline, extracting scan identifiers, match scores, p-values, and false discovery rates. Sort and filter matches by FDR threshold (typically stricter FDR values indicate higher-confidence identifications). Cross-compare FDR distributions and hit patterns across pipelines (e.g., Dereplicator vs. VarQuest vs. Dereplicator+) to identify tool-specific detections and common identifications. Record FDR alongside match scores to quantify sensitivity differences between search modes (standard parameters vs. --blind flag for arbitrary PTM detection). Use FDR as the primary ranking criterion when reporting identifications and when evaluating which pipeline or search mode is most effective for your spectral dataset.
Related tools
- Dereplicator (Database search pipeline that matches tandem mass spectra against post-translationally modified RiPP structure databases and outputs FDR-qualified match results) — https://github.com/ablab/npdtools
- VarQuest (Modification-tolerant variant database search pipeline that identifies novel PNP variants with p-values and FDR scores) — https://github.com/ablab/npdtools
- Dereplicator+ (Extended database search pipeline for both peptidic and non-peptidic metabolite identification with FDR-qualified results) — https://github.com/ablab/npdtools
- MetaMiner (Metabologenomic pipeline that integrates genomic and metabolomic data; uses Dereplicator for spectral matching and generates significant_matches.tsv with FDR values) — https://github.com/ablab/npdtools
- NPDtools 2.5.0 (Toolkit containing Dereplicator, VarQuest, Dereplicator+, and MetaMiner; provides unified interface and output format for FDR interpretation across pipelines) — https://github.com/ablab/npdtools
Examples
python parse_matches.py --dereplicator-tsv significant_matches_derep.tsv --varquest-tsv significant_matches_vq.tsv --dereplicator-plus-tsv significant_matches_derep_plus.tsv --fdr-threshold 0.05 --output comparison_table.csv
Evaluation signals
- FDR values are present for all reported matches and are numeric (typically 0–1 or 0–100 scale depending on pipeline output format).
- FDR distributions differ meaningfully across search modes (standard vs. --blind) or across pipelines (Dereplicator vs. VarQuest vs. Dereplicator+), indicating mode-dependent sensitivity.
- Matches filtered by FDR threshold show expected trade-offs: stricter thresholds reduce false positives but may miss true identifications; permissive thresholds increase hit counts but include lower-confidence matches.
- Cross-pipeline comparison reveals both tool-specific detections (high FDR in one pipeline, absent in others) and common identifications (low FDR across multiple pipelines), validating consistency.
- Scan identifiers are correctly preserved and traceable from raw spectra through match results to final FDR-ranked tables.
Limitations
- FDR computation assumes that the reference chemical structure database is comprehensive and representative; incomplete databases may systematically inflate FDRs for true identifications not represented in the database.
- Different pipelines (Dereplicator, VarQuest, Dereplicator+) may use different statistical models or thresholds to compute FDR, making cross-pipeline FDR comparison non-quantitative without normalization.
- The article does not provide explicit FDR cutoff thresholds (e.g., FDR < 0.05) for accepting identifications; interpretation depends on prior knowledge and downstream validation.
- FDR does not account for sample-specific factors (e.g., contamination, instrument drift) that may inflate or deflate confidence in a match within a particular run.
Evidence
- [methods] Parse and tabulate significant_matches.tsv outputs from all three tools, recording scan identifiers, match scores, p-values, and false discovery rates.: "Parse and tabulate significant_matches.tsv outputs from all three tools, recording scan identifiers, match scores, p-values, and false discovery rates."
- [methods] Cross-compare hit sets to identify tool-specific detections, common identifications, and mode-dependent sensitivity differences.: "Cross-compare hit sets to identify tool-specific detections, common identifications, and mode-dependent sensitivity differences."
- [readme] matches tandem mass spectra against the constructed post-translationally modified RiPPs structure database using Dereplicator: "matches tandem mass spectra against the constructed post-translationally modified RiPPs structure database using Dereplicator"
- [methods] Execute MetaMiner again with --blind flag enabled to search for arbitrary post-translational modifications.: "Execute MetaMiner again with --blind flag enabled to search for arbitrary post-translational modifications."
1---2name: false-discovery-rate-interpretation3description: Use when when you have executed database search pipelines (Dereplicator, VarQuest, or Dereplicator+) on centroided LC-MS/MS spectra in MGF format and obtained match results with associated p-values and false discovery rates.4license: CC-BY-4.05---67# false-discovery-rate-interpretation89## Summary1011Interpretation of false discovery rates (FDR) from database search pipelines to distinguish genuine natural product identifications from spurious matches in mass spectrometry data. This skill involves extracting FDR values from pipeline outputs and using them as a quality filter to rank and validate spectral matches.1213## When to use1415When you have executed database search pipelines (Dereplicator, VarQuest, or Dereplicator+) on centroided LC-MS/MS spectra in MGF format and obtained match results with associated p-values and false discovery rates. Use this skill when you need to discriminate between high-confidence and low-confidence identifications across multiple pipelines or search modes (e.g., standard vs. blind/PTM-tolerant searches).1617## When NOT to use1819- Input spectra are not centroided or are in unsupported formats (non-MGF/mzML/mzXML/mzData).20- Database search has not yet been executed; raw spectra alone without match results cannot be analyzed for FDR.21- FDR values are not calculated by the pipeline or are missing from output files.2223## Inputs2425- significant_matches.tsv files from MetaMiner, Dereplicator, VarQuest, or Dereplicator+26- Scan identifiers from LC-MS/MS spectra (MGF or mzML format)27- Match scores from database search results28- P-values from spectral matching29- False discovery rate values per match3031## Outputs3233- Tabulated comparison of FDR values across pipelines34- Filtered match sets ranked by FDR threshold35- Cross-pipeline hit pattern comparison matrix36- Tool-specific and common identification sets37- FDR-stratified sensitivity analysis by search mode3839## How to apply4041Parse the significant_matches.tsv output files generated by each database search pipeline, extracting scan identifiers, match scores, p-values, and false discovery rates. Sort and filter matches by FDR threshold (typically stricter FDR values indicate higher-confidence identifications). Cross-compare FDR distributions and hit patterns across pipelines (e.g., Dereplicator vs. VarQuest vs. Dereplicator+) to identify tool-specific detections and common identifications. Record FDR alongside match scores to quantify sensitivity differences between search modes (standard parameters vs. --blind flag for arbitrary PTM detection). Use FDR as the primary ranking criterion when reporting identifications and when evaluating which pipeline or search mode is most effective for your spectral dataset.4243## Related tools4445- **Dereplicator** (Database search pipeline that matches tandem mass spectra against post-translationally modified RiPP structure databases and outputs FDR-qualified match results) — https://github.com/ablab/npdtools46- **VarQuest** (Modification-tolerant variant database search pipeline that identifies novel PNP variants with p-values and FDR scores) — https://github.com/ablab/npdtools47- **Dereplicator+** (Extended database search pipeline for both peptidic and non-peptidic metabolite identification with FDR-qualified results) — https://github.com/ablab/npdtools48- **MetaMiner** (Metabologenomic pipeline that integrates genomic and metabolomic data; uses Dereplicator for spectral matching and generates significant_matches.tsv with FDR values) — https://github.com/ablab/npdtools49- **NPDtools 2.5.0** (Toolkit containing Dereplicator, VarQuest, Dereplicator+, and MetaMiner; provides unified interface and output format for FDR interpretation across pipelines) — https://github.com/ablab/npdtools5051## Examples5253```54python parse_matches.py --dereplicator-tsv significant_matches_derep.tsv --varquest-tsv significant_matches_vq.tsv --dereplicator-plus-tsv significant_matches_derep_plus.tsv --fdr-threshold 0.05 --output comparison_table.csv55```5657## Evaluation signals5859- FDR values are present for all reported matches and are numeric (typically 0–1 or 0–100 scale depending on pipeline output format).60- FDR distributions differ meaningfully across search modes (standard vs. --blind) or across pipelines (Dereplicator vs. VarQuest vs. Dereplicator+), indicating mode-dependent sensitivity.61- Matches filtered by FDR threshold show expected trade-offs: stricter thresholds reduce false positives but may miss true identifications; permissive thresholds increase hit counts but include lower-confidence matches.62- Cross-pipeline comparison reveals both tool-specific detections (high FDR in one pipeline, absent in others) and common identifications (low FDR across multiple pipelines), validating consistency.63- Scan identifiers are correctly preserved and traceable from raw spectra through match results to final FDR-ranked tables.6465## Limitations6667- FDR computation assumes that the reference chemical structure database is comprehensive and representative; incomplete databases may systematically inflate FDRs for true identifications not represented in the database.68- Different pipelines (Dereplicator, VarQuest, Dereplicator+) may use different statistical models or thresholds to compute FDR, making cross-pipeline FDR comparison non-quantitative without normalization.69- The article does not provide explicit FDR cutoff thresholds (e.g., FDR < 0.05) for accepting identifications; interpretation depends on prior knowledge and downstream validation.70- FDR does not account for sample-specific factors (e.g., contamination, instrument drift) that may inflate or deflate confidence in a match within a particular run.7172## Evidence7374- [methods] Parse and tabulate significant_matches.tsv outputs from all three tools, recording scan identifiers, match scores, p-values, and false discovery rates.: "Parse and tabulate significant_matches.tsv outputs from all three tools, recording scan identifiers, match scores, p-values, and false discovery rates."75- [methods] Cross-compare hit sets to identify tool-specific detections, common identifications, and mode-dependent sensitivity differences.: "Cross-compare hit sets to identify tool-specific detections, common identifications, and mode-dependent sensitivity differences."76- [readme] matches tandem mass spectra against the constructed post-translationally modified RiPPs structure database using Dereplicator: "matches tandem mass spectra against the constructed post-translationally modified RiPPs structure database using Dereplicator"77- [methods] Execute MetaMiner again with --blind flag enabled to search for arbitrary post-translational modifications.: "Execute MetaMiner again with --blind flag enabled to search for arbitrary post-translational modifications."
HolobiomicsLab 