metadata-normalization-and-reconciliation
Summary
Harmonize and reconcile feature metadata (identifiers, retention time, m/z, and other fields) from multiple mass spectrometry feature lists generated by different acquisition methods or processing software, then perform cross-list comparison to identify overlapping and unique features. This skill is essential when integrating heterogeneous MS datasets that must be unified before comparative analysis.
When to use
You have multiple CSV feature lists from different acquisition methods (e.g., LC-MS vs LC-IMS-MS) or processing software, each using different naming conventions, retention time scales, or m/z precision; you need to identify which features are the same across lists and which are unique to each method before performing downstream analysis or meta-analysis.
When NOT to use
- Input data are already from a single acquisition method or processing pipeline (no cross-method reconciliation needed)
- Feature lists lack common measurable fields such as m/z or retention time (comparison cannot be performed reliably)
- The goal is quality control of a single feature list rather than integration of multiple lists
Inputs
- CSV files containing feature lists from different acquisition methods or processing software
- Each CSV must include columns for feature identifiers, m/z values, retention time, and other metadata fields
Outputs
- Harmonized feature table (CSV or tabular format) with unified feature IDs and normalized metadata
- Comparison report documenting feature overlap statistics, source method tags, and presence/absence across input lists
How to apply
Load all CSV feature lists using pandas, then apply harmonization by matching on common fields (m/z, retention time, feature identifiers) and resolving naming conflicts through field mapping and tolerance-based matching (e.g., m/z and RT windows). Cross-list comparison logic identifies overlapping features across all input lists by finding matches within specified tolerances, while tagging each feature with its source method. Generate a structured comparison report documenting feature overlap statistics, harmonized feature IDs, source method annotations, and presence/absence patterns. The rationale is that direct string matching or ID comparison fails when software or instruments use different scales or nomenclature; tolerancing on continuous fields (m/z, RT) enables robust reconciliation even when nomenclature differs.
Related tools
- Comparador (Primary tool for harmonizing and comparing feature lists from different acquisition methods or processing software; ingests CSV files and applies reconciliation logic) — https://github.com/pnnl/IonToolPack
- IonToolPack (Software suite housing Comparador; provides GUI interface for feature list comparison and supports multiple MS data formats and omics domains) — https://github.com/pnnl/IonToolPack
Evaluation signals
- Harmonized feature IDs are consistent and resolvable across all input lists; no duplicate or conflicting IDs in the output
- Feature overlap statistics are symmetric or near-symmetric (if list A matches feature X from list B, then list B's feature X is tagged as matching list A)
- All input features are accounted for in the output; presence/absence patterns match the input data
- Retention time and m/z values in the harmonized output fall within the specified tolerance windows of their matched counterparts
- Source method tags correctly identify the origin of each feature and reflect input list boundaries
Limitations
- Harmonization accuracy depends on the choice of tolerance windows for m/z and retention time matching; overly tight tolerances may miss true matches across methods; overly loose tolerances may create false matches
- Naming conflicts in feature identifiers may not be fully resolved if software systems use orthogonal naming schemes with no common measurable anchor
- The workflow assumes CSV format; other feature list formats (e.g., mzTab, proprietary binary formats) must be converted to CSV first
- Cross-method comparison is limited to common fields present in all input lists; method-specific metadata may be lost during harmonization
Evidence
- [other] Comparador ingests feature lists in CSV format from different acquisition methods or processing software, applies harmonization procedures, and performs comparative analysis on the results.: "Comparador ingests feature lists in CSV format from different acquisition methods or processing software, applies harmonization procedures, and performs comparative analysis on the results."
- [other] Harmonize feature identifiers, retention time, m/z, and other metadata across lists by matching on common fields and resolving naming conflicts.: "Harmonize feature identifiers, retention time, m/z, and other metadata across lists by matching on common fields and resolving naming conflicts."
- [other] Identify overlapping and unique features across all input lists using cross-list comparison logic.: "Identify overlapping and unique features across all input lists using cross-list comparison logic."
- [other] Generate a structured comparison report (CSV or tabular format) documenting feature overlap statistics, harmonized feature IDs, source method tags, and analysis results.: "Generate a structured comparison report (CSV or tabular format) documenting feature overlap statistics, harmonized feature IDs, source method tags, and analysis results."
- [readme] Tool to compare lists of features (CSV files) from different acquisition methods or processing software, by harmonizing and analyzing results.: "Tool to compare lists of features (CSV files) from different acquisition methods or processing software, by harmonizing and analyzing results."
1---2name: metadata-normalization-and-reconciliation3description: Use when you have multiple CSV feature lists from different acquisition methods (e.g., LC-MS vs LC-IMS-MS) or processing software, each using different naming conventions, retention time scales, or m/z precision;4license: CC-BY-4.05---67# metadata-normalization-and-reconciliation89## Summary1011Harmonize and reconcile feature metadata (identifiers, retention time, m/z, and other fields) from multiple mass spectrometry feature lists generated by different acquisition methods or processing software, then perform cross-list comparison to identify overlapping and unique features. This skill is essential when integrating heterogeneous MS datasets that must be unified before comparative analysis.1213## When to use1415You have multiple CSV feature lists from different acquisition methods (e.g., LC-MS vs LC-IMS-MS) or processing software, each using different naming conventions, retention time scales, or m/z precision; you need to identify which features are the same across lists and which are unique to each method before performing downstream analysis or meta-analysis.1617## When NOT to use1819- Input data are already from a single acquisition method or processing pipeline (no cross-method reconciliation needed)20- Feature lists lack common measurable fields such as m/z or retention time (comparison cannot be performed reliably)21- The goal is quality control of a single feature list rather than integration of multiple lists2223## Inputs2425- CSV files containing feature lists from different acquisition methods or processing software26- Each CSV must include columns for feature identifiers, m/z values, retention time, and other metadata fields2728## Outputs2930- Harmonized feature table (CSV or tabular format) with unified feature IDs and normalized metadata31- Comparison report documenting feature overlap statistics, source method tags, and presence/absence across input lists3233## How to apply3435Load all CSV feature lists using pandas, then apply harmonization by matching on common fields (m/z, retention time, feature identifiers) and resolving naming conflicts through field mapping and tolerance-based matching (e.g., m/z and RT windows). Cross-list comparison logic identifies overlapping features across all input lists by finding matches within specified tolerances, while tagging each feature with its source method. Generate a structured comparison report documenting feature overlap statistics, harmonized feature IDs, source method annotations, and presence/absence patterns. The rationale is that direct string matching or ID comparison fails when software or instruments use different scales or nomenclature; tolerancing on continuous fields (m/z, RT) enables robust reconciliation even when nomenclature differs.3637## Related tools3839- **Comparador** (Primary tool for harmonizing and comparing feature lists from different acquisition methods or processing software; ingests CSV files and applies reconciliation logic) — https://github.com/pnnl/IonToolPack40- **IonToolPack** (Software suite housing Comparador; provides GUI interface for feature list comparison and supports multiple MS data formats and omics domains) — https://github.com/pnnl/IonToolPack4142## Evaluation signals4344- Harmonized feature IDs are consistent and resolvable across all input lists; no duplicate or conflicting IDs in the output45- Feature overlap statistics are symmetric or near-symmetric (if list A matches feature X from list B, then list B's feature X is tagged as matching list A)46- All input features are accounted for in the output; presence/absence patterns match the input data47- Retention time and m/z values in the harmonized output fall within the specified tolerance windows of their matched counterparts48- Source method tags correctly identify the origin of each feature and reflect input list boundaries4950## Limitations5152- Harmonization accuracy depends on the choice of tolerance windows for m/z and retention time matching; overly tight tolerances may miss true matches across methods; overly loose tolerances may create false matches53- Naming conflicts in feature identifiers may not be fully resolved if software systems use orthogonal naming schemes with no common measurable anchor54- The workflow assumes CSV format; other feature list formats (e.g., mzTab, proprietary binary formats) must be converted to CSV first55- Cross-method comparison is limited to common fields present in all input lists; method-specific metadata may be lost during harmonization5657## Evidence5859- [other] Comparador ingests feature lists in CSV format from different acquisition methods or processing software, applies harmonization procedures, and performs comparative analysis on the results.: "Comparador ingests feature lists in CSV format from different acquisition methods or processing software, applies harmonization procedures, and performs comparative analysis on the results."60- [other] Harmonize feature identifiers, retention time, m/z, and other metadata across lists by matching on common fields and resolving naming conflicts.: "Harmonize feature identifiers, retention time, m/z, and other metadata across lists by matching on common fields and resolving naming conflicts."61- [other] Identify overlapping and unique features across all input lists using cross-list comparison logic.: "Identify overlapping and unique features across all input lists using cross-list comparison logic."62- [other] Generate a structured comparison report (CSV or tabular format) documenting feature overlap statistics, harmonized feature IDs, source method tags, and analysis results.: "Generate a structured comparison report (CSV or tabular format) documenting feature overlap statistics, harmonized feature IDs, source method tags, and analysis results."63- [readme] Tool to compare lists of features (CSV files) from different acquisition methods or processing software, by harmonizing and analyzing results.: "Tool to compare lists of features (CSV files) from different acquisition methods or processing software, by harmonizing and analyzing results."
HolobiomicsLab