GCF-MF hierarchical aggregation
Summary
Generalizes pairwise BGC-spectrum link scores to Gene Cluster Family (GCF) to Molecular Family (MF) level by computing the maximum score over all constituent BGC-spectrum pairs. This hierarchical aggregation enables scoring of multi-BGC/multi-spectrum links in natural product discovery workflows.
When to use
When you have predicted BGC-spectrum IOKR scores or other pairwise linking scores, and need to rank genomic clusters (GCFs from BiG-SCAPE) against metabolomic clusters (MFs from MS/MS spectra grouping), particularly in NPLinker workflows where one GCF may contain multiple BGCs and one MF may contain multiple spectra.
When NOT to use
- When input scores are already aggregated at the GCF-MF level; hierarchical aggregation is a one-way transformation.
- When GCF or MF membership is undefined or empty (no BGCs in a GCF or no spectra in an MF).
- When you need to preserve information about which specific BGC-spectrum pair drove the max score but do not also record the pair ID; track the winning pair separately if forensic detail is required.
Inputs
- BGC-spectrum IOKR score table (columns: BGC_ID, spectrum_ID, σ_IOKR)
- GCF membership table (columns: GCF_ID, BGC_ID)
- MF membership table (columns: MF_ID, spectrum_ID)
Outputs
- GCF-MF link score table (columns: GCF_ID, MF_ID, max_IOKR_score, num_pairs_evaluated)
- Ranked GCF-MF pairs sorted by max_IOKR_score descending
How to apply
Load the table of BGC-spectrum IOKR scores σ_IOKR(m, g) for all predicted pairs. For each candidate GCF-MF pair, retrieve all BGCs assigned to the GCF and all spectra assigned to the MF. Filter the BGC-spectrum score table to only pairs where both the BGC and spectrum are present in the input data. Compute the maximum IOKR score over all qualifying (BGC, spectrum) combinations: σ_IOKR(G, M) = max{σ_IOKR(m, g) : m ∈ M, g ∈ G}. Output a ranked table with columns GCF_ID, MF_ID, max_IOKR_score, and num_pairs_evaluated. The max aggregation rationale is that if even one BGC-spectrum pair shows strong evidence of linkage, the GCF-MF pair warrants investigation; this avoids diluting weak average signals from heterogeneous clusters.
Related tools
- IOKR (Generates BGC-spectrum pairwise link scores (σ_IOKR) that are aggregated by this skill)
- NPLinker (Framework orchestrating genomic-metabolomic linking; implements hierarchical aggregation workflow) — https://github.com/sdrogers/nplinker
- BiG-SCAPE (Produces Gene Cluster Family (GCF) assignments from antiSMASH BGCs; output GCF_ID is input to aggregation)
Evaluation signals
- Output table has exactly as many rows as there are unique (GCF_ID, MF_ID) pairs tested; no duplicates.
- Each max_IOKR_score value lies within the range [min, max] of the input BGC-spectrum scores; no score should exceed the global maximum or fall below the global minimum.
- For each row, num_pairs_evaluated ≥ 1 and ≤ (|BGCs in GCF| × |spectra in MF|); verify no false negatives from filtering logic.
- Spot-check 5–10 rows: manually retrieve the constituent BGC-spectrum pairs and confirm the reported max_IOKR_score equals the observed maximum.
- Validated links (from PoDP or other ground truth) should rank significantly higher (p < 0.05 by Mann–Whitney U test) than random GCF-MF pairs, mirroring the p-value of 2.633 × 10⁻⁴ reported for combined IOKR + standardised correlation linkage.
Limitations
- Max aggregation discards information about the full distribution of BGC-spectrum scores within a GCF-MF pair; a GCF-MF pair with one strong match and many weak matches looks identical to one with all strong matches.
- IOKR scoring itself relies on MIBiG homology to assign molecular structures to BGCs, restricting use to BGCs showing considerable homology; BGCs without characterized homologs will have no IOKR scores and will be omitted from aggregation.
- If a GCF contains BGCs not in the input IOKR score table (e.g. novel, unannotated BGCs), those BGCs are silently filtered out, potentially underestimating the true number of candidate pairs and biasing the aggregation toward characterized BGCs.
- Kernel function and parameter selection for IOKR significantly affects performance but choices are not fully characterized; aggregation quality is downstream of IOKR calibration choices.
Evidence
- [other] The IOKR score is generalised from BGC-spectrum pairs to GCF-MF pairs by computing the maximum IOKR score over all (BGC in GCF, spectrum in MF) combinations: for a GCF G and MF M, σ_IOKR(M, G) = max{σ_IOKR(m, g) : m ∈ M, g ∈ G}: "The IOKR score is generalised from BGC-spectrum pairs to GCF-MF pairs by computing the maximum IOKR score over all (BGC in GCF, spectrum in MF) combinations: for a GCF G and MF M, σ_IOKR(M, G) ="
- [other] Workflow: (1) Load the set of predicted BGC-spectrum IOKR scores (σ_IOKR values for individual pairs). (2) For each GCF-MF pair, retrieve all associated BGCs in the GCF and all spectra in the MF. (3) Filter to (BGC, spectrum) pairs where both elements are present in the input IOKR score table.: "1. Load the set of predicted BGC-spectrum IOKR scores (σ_IOKR values for individual pairs). 2. For each GCF-MF pair, retrieve all associated BGCs in the GCF and all spectra in the MF. 3. Filter to"
- [other] For each GCF-MF pair, compute the maximum IOKR score over all matching (BGC, spectrum) pairs: σ_IOKR(G,M) = max σ_IOKR(m, g) for m ∈ M, g ∈ G. Output a table with columns: GCF_ID, MF_ID, max_IOKR_score, num_pairs_evaluated.: "For each GCF-MF pair, compute the maximum IOKR score over all matching (BGC, spectrum) pairs: σ_IOKR(G,M) = max σ_IOKR(m, g) for m ∈ M, g ∈ G. 5. Output a table with columns: GCF_ID, MF_ID,"
- [readme] NPLinker is a python framework for data mining microbial natural products by integrating genomics and metabolomics data.: "NPLinker is a python framework for data mining microbial natural products by integrating genomics and metabolomics data."
- [discussion] A drawback of the current IOKR scoring method is its reliance on MIBiG homology to assign molecular structures to BGCs, which is needed to compute the molecular fingerprint. This restricts its use to: "A drawback of the current IOKR scoring method is its reliance on MIBiG homology to assign molecular structures to BGCs, which is needed to compute the molecular fingerprint. This restricts its use to"
1---2name: gcf-mf-hierarchical-aggregation3description: Use when when you have predicted BGC-spectrum IOKR scores or other pairwise linking scores, and need to rank genomic clusters (GCFs from BiG-SCAPE) against metabolomic clusters (MFs from MS/MS spectra grouping), particularly in NPLinker workflows where one GCF may contain multiple BGCs and one MF.4license: CC-BY-4.05---67# GCF-MF hierarchical aggregation89## Summary1011Generalizes pairwise BGC-spectrum link scores to Gene Cluster Family (GCF) to Molecular Family (MF) level by computing the maximum score over all constituent BGC-spectrum pairs. This hierarchical aggregation enables scoring of multi-BGC/multi-spectrum links in natural product discovery workflows.1213## When to use1415When you have predicted BGC-spectrum IOKR scores or other pairwise linking scores, and need to rank genomic clusters (GCFs from BiG-SCAPE) against metabolomic clusters (MFs from MS/MS spectra grouping), particularly in NPLinker workflows where one GCF may contain multiple BGCs and one MF may contain multiple spectra.1617## When NOT to use1819- When input scores are already aggregated at the GCF-MF level; hierarchical aggregation is a one-way transformation.20- When GCF or MF membership is undefined or empty (no BGCs in a GCF or no spectra in an MF).21- When you need to preserve information about which specific BGC-spectrum pair drove the max score but do not also record the pair ID; track the winning pair separately if forensic detail is required.2223## Inputs2425- BGC-spectrum IOKR score table (columns: BGC_ID, spectrum_ID, σ_IOKR)26- GCF membership table (columns: GCF_ID, BGC_ID)27- MF membership table (columns: MF_ID, spectrum_ID)2829## Outputs3031- GCF-MF link score table (columns: GCF_ID, MF_ID, max_IOKR_score, num_pairs_evaluated)32- Ranked GCF-MF pairs sorted by max_IOKR_score descending3334## How to apply3536Load the table of BGC-spectrum IOKR scores σ_IOKR(m, g) for all predicted pairs. For each candidate GCF-MF pair, retrieve all BGCs assigned to the GCF and all spectra assigned to the MF. Filter the BGC-spectrum score table to only pairs where both the BGC and spectrum are present in the input data. Compute the maximum IOKR score over all qualifying (BGC, spectrum) combinations: σ_IOKR(G, M) = max{σ_IOKR(m, g) : m ∈ M, g ∈ G}. Output a ranked table with columns GCF_ID, MF_ID, max_IOKR_score, and num_pairs_evaluated. The max aggregation rationale is that if even one BGC-spectrum pair shows strong evidence of linkage, the GCF-MF pair warrants investigation; this avoids diluting weak average signals from heterogeneous clusters.3738## Related tools3940- **IOKR** (Generates BGC-spectrum pairwise link scores (σ_IOKR) that are aggregated by this skill)41- **NPLinker** (Framework orchestrating genomic-metabolomic linking; implements hierarchical aggregation workflow) — https://github.com/sdrogers/nplinker42- **BiG-SCAPE** (Produces Gene Cluster Family (GCF) assignments from antiSMASH BGCs; output GCF_ID is input to aggregation)4344## Evaluation signals4546- Output table has exactly as many rows as there are unique (GCF_ID, MF_ID) pairs tested; no duplicates.47- Each max_IOKR_score value lies within the range [min, max] of the input BGC-spectrum scores; no score should exceed the global maximum or fall below the global minimum.48- For each row, num_pairs_evaluated ≥ 1 and ≤ (|BGCs in GCF| × |spectra in MF|); verify no false negatives from filtering logic.49- Spot-check 5–10 rows: manually retrieve the constituent BGC-spectrum pairs and confirm the reported max_IOKR_score equals the observed maximum.50- Validated links (from PoDP or other ground truth) should rank significantly higher (p < 0.05 by Mann–Whitney U test) than random GCF-MF pairs, mirroring the p-value of 2.633 × 10⁻⁴ reported for combined IOKR + standardised correlation linkage.5152## Limitations5354- Max aggregation discards information about the full distribution of BGC-spectrum scores within a GCF-MF pair; a GCF-MF pair with one strong match and many weak matches looks identical to one with all strong matches.55- IOKR scoring itself relies on MIBiG homology to assign molecular structures to BGCs, restricting use to BGCs showing considerable homology; BGCs without characterized homologs will have no IOKR scores and will be omitted from aggregation.56- If a GCF contains BGCs not in the input IOKR score table (e.g. novel, unannotated BGCs), those BGCs are silently filtered out, potentially underestimating the true number of candidate pairs and biasing the aggregation toward characterized BGCs.57- Kernel function and parameter selection for IOKR significantly affects performance but choices are not fully characterized; aggregation quality is downstream of IOKR calibration choices.5859## Evidence6061- [other] The IOKR score is generalised from BGC-spectrum pairs to GCF-MF pairs by computing the maximum IOKR score over all (BGC in GCF, spectrum in MF) combinations: for a GCF G and MF M, σ_IOKR(M, G) = max{σ_IOKR(m, g) : m ∈ M, g ∈ G}: "The IOKR score is generalised from BGC-spectrum pairs to GCF-MF pairs by computing the maximum IOKR score over all (BGC in GCF, spectrum in MF) combinations: for a GCF G and MF M, σ_IOKR(M, G) ="62- [other] Workflow: (1) Load the set of predicted BGC-spectrum IOKR scores (σ_IOKR values for individual pairs). (2) For each GCF-MF pair, retrieve all associated BGCs in the GCF and all spectra in the MF. (3) Filter to (BGC, spectrum) pairs where both elements are present in the input IOKR score table.: "1. Load the set of predicted BGC-spectrum IOKR scores (σ_IOKR values for individual pairs). 2. For each GCF-MF pair, retrieve all associated BGCs in the GCF and all spectra in the MF. 3. Filter to"63- [other] For each GCF-MF pair, compute the maximum IOKR score over all matching (BGC, spectrum) pairs: σ_IOKR(G,M) = max σ_IOKR(m, g) for m ∈ M, g ∈ G. Output a table with columns: GCF_ID, MF_ID, max_IOKR_score, num_pairs_evaluated.: "For each GCF-MF pair, compute the maximum IOKR score over all matching (BGC, spectrum) pairs: σ_IOKR(G,M) = max σ_IOKR(m, g) for m ∈ M, g ∈ G. 5. Output a table with columns: GCF_ID, MF_ID,"64- [readme] NPLinker is a python framework for data mining microbial natural products by integrating genomics and metabolomics data.: "NPLinker is a python framework for data mining microbial natural products by integrating genomics and metabolomics data."65- [discussion] A drawback of the current IOKR scoring method is its reliance on MIBiG homology to assign molecular structures to BGCs, which is needed to compute the molecular fingerprint. This restricts its use to: "A drawback of the current IOKR scoring method is its reliance on MIBiG homology to assign molecular structures to BGCs, which is needed to compute the molecular fingerprint. This restricts its use to"
HolobiomicsLab 