HolobiomicsLab 
sub-cluster-assignment-extraction
Summary
Extract and structure gene-to-sub-cluster assignments and topic-assignment weights from Latent Dirichlet Allocation topic modelling results. This skill transforms the raw output of topic models into a queryable assignment table that links gene identifiers to detected sub-cluster IDs with associated confidence scores.
When to use
Apply this skill after PRESTO-TOP topic modelling has been run on redundancy-filtered tokenised BGCs and raw topic assignments have been generated. Use it when you need to convert unstructured topic model output into a relational format suitable for downstream linkage to natural product substructures or for validation against known biosynthetic pathways.
When NOT to use
- Input has not yet been tokenised and redundancy-filtered; run tokenization and Adjacency Index-based filtering first.
- Topic modelling has not been performed; this skill assumes LDA output already exists.
- Assignment table already exists in downstream format; extraction is redundant.
Inputs
- Raw topic model output from PRESTO-TOP (LDA posterior distributions)
- Redundancy-filtered tokenised BGCs with gene identifiers
- Topic-gene membership matrix or similar LDA output format
Outputs
- Sub-cluster assignment table (gene ID, sub-cluster ID, topic-assignment weight)
- Structured sub-cluster membership with topic scores
- Gene-to-sub-cluster mapping suitable for linkage to natural product substructures
How to apply
After PRESTO-TOP topic modelling completes, parse the topic-gene membership matrix and extract three key components: (1) gene identifiers from the input tokenised BGCs; (2) topic assignments (the detected sub-cluster IDs inferred by LDA); (3) topic-assignment weights (posterior probabilities or membership scores reflecting the strength of each gene–sub-cluster association). Structure these into a table where each row represents one gene, with columns for gene ID, assigned sub-cluster ID, and corresponding topic score. The topic score indicates confidence: higher scores reflect stronger coherence with the sub-cluster's domain co-occurrence pattern. Filter or flag assignments with low confidence scores if downstream analysis requires high-confidence links only.
Related tools
- iPRESTO (Command-line tool that performs tokenization, redundancy filtering, and PRESTO-TOP topic modelling; sub-cluster extraction operates on its output) — https://github.com/medema-group/iPRESTO
Evaluation signals
- All genes from the input tokenised BGCs appear exactly once in the output assignment table.
- Topic-assignment weights are valid probabilities or membership scores in the expected range (e.g., 0–1 or 0–100); no NaN or infinite values.
- Sub-cluster IDs are consistent with the number of topics K inferred by the LDA model; no orphaned or duplicate assignments.
- Gene identifiers in the output table match identifiers from the input redundancy-filtered tokenised BGCs exactly (no truncation or corruption).
- Assignment table is joinable with tokenised BGC domain annotations and natural product substructure labels without reconciliation errors.
Limitations
- Topic assignment quality depends on the quality of tokenisation (Pfam/subPfam resolution) and redundancy filtering (Adjacency Index threshold); poor upstream choices degrade output.
- LDA is stochastic; replicate runs with the same K may produce different topic orderings or label permutations; post-hoc matching or consensus strategies may be needed for reproducibility.
- Sub-cluster biological interpretability requires downstream validation (e.g., linkage to known natural product substructures); high topic scores do not guarantee functional relevance.
- Genes assigned to multiple topics with similar weights reflect genuine ambiguity in domain co-occurrence patterns; hard assignment (single sub-cluster per gene) may lose information.
Evidence
- [other] Extract and structure detected sub-clusters with gene membership and topic-assignment weights: "Extract and structure detected sub-clusters with gene membership and topic-assignment weights."
- [other] Produce a sub-cluster output table linking gene identifiers to assigned sub-cluster IDs and topic scores: "Produce a sub-cluster output table linking gene identifiers to assigned sub-cluster IDs and topic scores."
- [other] PRESTO-TOP is a novel sub-cluster detection method that applies topic modelling with Latent Dirichlet Allocation to identify gene sub-clusters from redundancy-filtered tokenised BGCs: "PRESTO-TOP is a novel sub-cluster detection method that applies topic modelling with Latent Dirichlet Allocation to identify gene sub-clusters from redundancy-filtered tokenised BGCs."
- [intro] The sub-clusters found with iPRESTO can then be linked to Natural Product substructures: "The sub-clusters found with iPRESTO can then be linked to Natural Product substructures"