Bio Comparative Genomics Hgt Detection

Version Compatibility

Reference examples tested with: HGTector 2.0b3+, AvP 1.0.4+, HGTphyloDetect 1.0+, ALE 1.0+ (ssolo/ALE github), GeneRax 2.1.3+, AleRax 1.2.0+ (Morel 2024), RANGER-DTL 2.0+, IslandViewer 4 (web), mobileOG-db 1.0+, MetaCHIP 1.10+, IQ-TREE 2.3.6+, BioPython 1.84+, DIAMOND 2.1.10+. Open Tree of Life and NCBI Taxonomy reference databases updated 2024-Q3 minimum for HGTector/AvP.

Before using code patterns, verify installed versions match. If versions differ:

• Python: pip show hgtector then hgtector search --help
• CLI: ALEml_undated --help, generax --help, alerax --help
• DB: hgtector database --check for taxonomy version

If code throws Taxonomy ID not found, database version mismatch, or KeyError on NCBI taxids, refresh the local taxonomy dump (NCBI updates monthly). ALE/GeneRax expect newick gene trees with bootstraps; AleRax expects gene-tree distributions (uniform bootstrap samples or UFBoot trees).

Horizontal Gene Transfer Detection

"Are these genes horizontally acquired, and from where?" -> HGT signal lives in three orthogonal signal classes: composition (recent transfers carry donor codon usage; erodes by Lawrence-Ochman 1998 amelioration in ~50-200 Myr), phylogeny (gene tree nests within distant clade), and phyletic distribution (patchy taxonomic presence). No single class proves HGT; claims require concordance across at least two classes plus mandatory exclusion of contamination and differential gene loss (DGL). The most consequential failure mode in eukaryotic HGT detection is contamination passing all three classes silently (Boothby 2015 tardigrade "17% HGT" refuted by Koutsovoulos 2016; Crisp 2015 human "145 HGTs" refuted by Salzberg 2017 GB 18:85).

• Python: hgtector search -> hgtector analyze for BLAST-distribution screen
• Python: AvP (Koutsovoulos 2022 PLoS Comp Biol 18:e1010686) for eukaryotic phylogenetic HGT with automated tree workflow
• CLI: ALEml_undated (Szöllősi 2013 Syst Biol 62:901), generax (Morel 2020 MBE 37:2763), alerax (Morel 2024 Bioinformatics 40:btae162) for prokaryote DTL reconciliation
• Web: IslandViewer 4 (Bertelli 2017 NAR 45:W30) for bacterial genomic islands
• CLI: metachip (Song 2019 Microbiome 7:36) for metagenomic HGT inference

Algorithmic Taxonomy

Methodology evolves; verify the current AleRax / AvP documentation before committing to a single approach. ALE/GeneRax/AleRax have largely superseded older parsimony reconciliation for bacterial phylogenomics.

Decision Tree by Experimental Scenario

Per-Tool Failure Modes

Contamination masquerading as eukaryotic HGT (THE critical failure)

Trigger: Eukaryote genome assembly, especially from non-axenic culture, microbiome-associated organism, or low-coverage shotgun.

Mechanism: Bacterial DNA in the sample is assembled as contigs separate from the eukaryote nuclear contigs but is reported as part of the assembly. Genes on contaminant contigs phylogenetically nest within bacteria, compositionally differ from the eukaryote, and have patchy phyletic distribution -- triggering all three HGT signal classes simultaneously.

Symptom: "HGT" genes are concentrated on short, low-coverage contigs; have GC% dramatically different from the bulk genome; show codon usage indistinguishable from bacteria; cluster on contigs lacking eukaryotic gene order; the genome assembly's BUSCO completeness anomaly indicates contamination (e.g. tardigrade Hypsibius dujardini: Boothby 2015 PNAS 112:15976 -> Koutsovoulos 2016 PNAS 113:5053 contamination refutation).

Fix: MANDATORY contamination filter before any eukaryotic HGT analysis. Use BlobTools2 (Challis 2020 G3) coverage-vs-GC visualization to identify contaminant blobs; Kraken2 (Wood 2019 GB 20:257) or Conterminator (Steinegger 2020 GB 21:115) to taxonomically classify contigs; FCS-GX (Astashyn 2024 GB 25:60) is the NCBI tool now required for GenBank submission. Apply these BEFORE running AvP / HGTector / Alien Index. After cleaning, re-screen. Crisp 2015 Genome Biol 16:50 "145 HGT in humans" was nearly all refuted by Salzberg 2017 GB 18:85 contamination-aware reanalysis (only known mitochondrial and retroviral insertions survive).

Amelioration eroding compositional signal

Trigger: Compositional-only HGT calls on bacterial genomes diverged > 50 Myr from donor.

Mechanism: After transfer, point mutations occur under host mutation bias (e.g. AT bias in obligate symbionts), driving codon usage and GC% toward host values (Lawrence & Ochman 1997 J Mol Evol 44:383; 1998 PNAS 95:9413). Half-life of compositional signal in bacteria is roughly 50-200 Myr depending on mutation rate and selection. Ancient transfers are compositionally indistinguishable from native genes.

Symptom: Phylogenetic methods detect transfer but composition-based tools (SIGI, IslandPath) miss it; GC anomaly is weak (|z| < 2); codon adaptation index resembles native genes.

Fix: For HGT older than ~50 Myr, rely on phylogenetic methods (ALE / AvP); composition is informative only as supporting evidence for recent transfers. Report transfer age estimate from ALE branch posterior alongside composition.

Differential gene loss mimicking HGT

Trigger: Gene present in distantly related taxa but absent in immediate sister lineages.

Mechanism: An ancestral gene is independently lost in multiple intermediate lineages; the surviving taxa appear "incompatible" with the species tree. Phylogenetic incongruence and patchy taxonomic distribution are identical to HGT signatures (Maddison 1997 Syst Biol 46:523).

Symptom: Gene-tree topology actually matches species-tree topology of the surviving taxa, just with intermediate taxa missing. Composition matches the recipient genome (no HGT amelioration to explain). ALE infers high loss rate at branches and may favor a "transfer + loss" or "duplication + losses" event class depending on cost weights.

Fix: Inspect ALE event-class posteriors (D vs T vs L); when loss rate inference is non-negligible at sister branches, prefer the loss-explanation. Check independently sequenced relatives if available. Quantify Dollo-parsimony loss vs ML-DTL event likelihoods. RANGER-DTL with loss cost = 1, transfer cost = 3 favors loss; rerun with transfer cost = 1.5 to see sensitivity.

Hybridization / reticulate evolution

Trigger: Sister species suspected to have hybridized or share recent introgression; rapid radiations.

Mechanism: Gene flow between closely related lineages produces gene-tree species-tree discordance indistinguishable from HGT at short divergence times. Phylogenetic networks (not trees) are required.

Symptom: Multiple genes show identical pattern of incongruence (same direction, same sister); D-statistic (see [[introgression-detection]]) significantly different from zero between candidate hybrid and its inferred sister.

Fix: Run ABBA-BABA / Dsuite for the candidate (see [[introgression-detection]]); if introgression signal is uniform across the genome, prefer hybridization explanation. ALE on bacterial trees can handle this via transfer-with-replacement; for eukaryotes, phylogenetic networks (e.g. PhyloNetworks / SNaQ via Solis-Lemus & Ane 2016 PLoS Genet 12:e1005896, or NakhlehLab PhyloNet) properly model reticulation.

gBGC-driven AT->GC substitution bias

Trigger: Mammalian / vertebrate gene with apparent unusual nucleotide composition triggering compositional HGT call.

Mechanism: GC-biased gene conversion (gBGC) in regions of high recombination drives Weak->Strong (AT->GC) fixation independently of selection (Galtier & Duret 2007 Trends Genet 23:273). Sub-telomeric and recombination-hot regions show elevated GC indistinguishable from a GC-rich HGT donor at the composition level.

Symptom: "HGT" candidates cluster in sub-telomeric regions or high-recombination zones; W->S substitution bias is elevated; phylogenetic placement is consistent with host species; selection scans (BUSTED) show no signal.

Fix: Test for gBGC explicitly via W->S vs S->W substitution ratios (Capra 2013 PLoS Genet 9:e1003684); require non-zero phylogenetic incongruence in addition to composition; exclude regions with recombination rate > 90th percentile.

Close-relative donor making BLAST methods fail

Trigger: Transfer between species in the same genus or family (genus-level transfer).

Mechanism: HGTector / DarkHorse / Alien Index compare close vs distal BLAST hits; when donor is closely related, hits cluster in "close" and the method reports no anomaly.

Symptom: HGTector hgt_score near zero, but the gene shows clear phylogenetic placement within a sister taxon rather than expected vertical inheritance.

Fix: Use phylogenetic methods (ALE on orthogroup trees) for genus-level HGT; BLAST-distribution methods are designed for inter-phylum transfer. Restrict HGTector taxonomic comparison level (--rank order or higher).

Poor sampling of donor lineage

Trigger: Putative HGT phylogenetically places "near" a poorly sampled taxon (e.g. Archaea or candidate phyla).

Mechanism: Long branches in the donor clade attract the query gene by LBA (Felsenstein 1978 Syst Zool 27:401); poor sampling means real intermediate relatives are absent, and the gene appears nested within the wrong clade.

Symptom: AU/SH topology test rejects alternative placements only marginally (p ~ 0.05); adding any newly available genome from the candidate donor lineage changes placement substantially.

Fix: Use site-heterogeneous models (CAT-PMSF, PhyloBayes-MPI) to mitigate LBA at deep nodes; add taxa from undersampled lineages where possible; report donor inference with explicit support-level caveats. Cross-check with ALE branch posterior, which integrates over alternative donor lineages probabilistically.

Tetranucleotide z-score thresholding in heterogeneous genomes

Trigger: Applying generic |z| > 2 cutoff to genomes with naturally high composition variance (Streptomyces, Burkholderia, Halophiles).

Mechanism: rRNA operons, prophage remnants, and CRISPR arrays have native composition anomalies; generic z-score thresholds flag them as HGT.

Symptom: "HGT" calls concentrate in rRNA-flanking regions, ribosomal protein operons, or annotated prophages.

Fix: Mask known native-anomaly features (rRNA, tRNA, CRISPR) before composition analysis; use genome-specific thresholds calibrated against confirmed native genes; raise |z| threshold to 3 for high-variance genomes.

Quantitative Thresholds

HGTector v2 Workflow

Goal: Identify HGT candidates in a prokaryote genome via taxonomic BLAST hit distribution.

Approach: Build / download taxonomy database -> hgtector search to run DIAMOND against reference proteomes -> hgtector analyze to score close-vs-distal taxonomic distribution -> rank candidates by score.

'''HGTector v2 wrapper with quality filters'''

import subprocess
import pandas as pd


def run_hgtector(proteome_faa, out_dir, db_dir, threads=8, rank='order'):
    '''HGTector v2 search + analyze.

    rank: NCBI taxonomic rank below which hits are "close"; "order" is default.
    Use "family" to detect family-level transfers, "phylum" only for ancient inter-phylum HGT.
    '''
    subprocess.run([
        'hgtector', 'search', '-i', proteome_faa, '-o', f'{out_dir}/search',
        '-m', 'diamond', '-d', f'{db_dir}/db.dmnd', '-t', str(threads),
        '--queries-per-batch', '200'
    ], check=True)
    subprocess.run([
        'hgtector', 'analyze', '-i', f'{out_dir}/search', '-o', f'{out_dir}/analyze',
        '-t', f'{db_dir}/taxdump', '--rank-self', rank,
        '--rank-close', rank, '--bandwidth', 'auto'
    ], check=True)
    return f'{out_dir}/analyze'


def parse_hgtector(analyze_dir, min_score=0.5):
    '''HGTector outputs scores.tsv with: gene, hits, close, distal, ...
    distal > close indicates likely foreign origin.
    '''
    df = pd.read_csv(f'{analyze_dir}/scores.tsv', sep='\t')
    df['hgt_score'] = df['distal'] - df['close']
    df['confidence'] = pd.cut(df['hgt_score'], bins=[-1e9, 0.5, 1.0, 1e9],
                              labels=['low', 'moderate', 'strong'])
    return df[df['hgt_score'] > min_score].sort_values('hgt_score', ascending=False)

ALE Probabilistic DTL Reconciliation

Goal: Quantify D / T / L events with branch-wise posterior on a bacterial / archaeal species tree.

Approach: Infer per-orthogroup gene-tree bootstrap distributions (UFBoot from IQ-TREE) -> observe (encode as .ale files) -> run ALEml_undated to reconcile against species tree -> extract per-branch transfer count posteriors.

# 1. Build per-orthogroup UFBoot trees
for og in orthogroups/*.fa; do
    iqtree2 -s $og -m TEST -B 1000 -nt 2 --prefix ${og%.fa}_ufb
done

# 2. Observe and reconcile
for tree in orthogroups/*_ufb.ufboot; do
    ALEobserve $tree
    ALEml_undated species_tree.nwk ${tree}.ale separators="|" sample=100
done

# 3. Aggregate D/T/L counts per species-tree branch
python aggregate_ale.py orthogroups/*_uTs > dtl_branchwise.tsv

'''Aggregate ALE outputs into branch-wise event tables.'''
import re
from collections import defaultdict
import pandas as pd

def parse_ale_uts(uts_file):
    '''ALE _uTs format: per-branch D/T/L counts (mean over sampled reconciliations).'''
    branch_events = {}
    with open(uts_file) as fh:
        for line in fh:
            if line.startswith('#') or not line.strip():
                continue
            parts = line.split()
            if len(parts) >= 5:
                branch_id = parts[0]
                # columns: branch  duplications  transfers  losses  originations
                branch_events[branch_id] = {
                    'duplications': float(parts[1]),
                    'transfers': float(parts[2]),
                    'losses': float(parts[3]),
                    'originations': float(parts[4])
                }
    return branch_events

Bacterial datasets characteristically show transfers >> duplications, while opisthokonts (eukaryotes) show the opposite (Szöllősi 2013). ALE-rooting of species trees (Williams 2017 PNAS 114:E4602) is now a standard rooting approach for deep archaeal/prokaryote phylogenomics.

AvP Eukaryotic HGT Phylogenetic Workflow

Goal: Detect HGT in eukaryotic proteomes with automated phylogenetic-tree validation.

Approach: avp prepare builds taxonomic database from candidate hits -> avp detect builds gene trees per candidate -> avp classify runs AU/SH tests on alternative placements.

avp prepare -t taxonomy.tsv -d nr.fasta -o database
avp detect -i query_proteome.fasta -d database -o avp_results -threads 8
avp classify -i avp_results -d database --au-test --bootstrap 1000

The output avp_results/classification.tsv reports per-gene: HGT / putative-HGT / unknown / non-HGT, donor lineage assignment, gene-tree support, and AU-test p-value. Apply only after FCS-GX or BlobTools contamination filter.

IslandViewer 4 + mobileOG-db Integration

For single-genome bacterial analysis, the combined workflow is: IslandViewer 4 (web; submit assembly, returns islands via IslandPath-DIMOB / SIGI-HMM / IslandPick consensus) -> annotate genes with mobileOG-db (Brown 2022 Appl Environ Microbiol 88:e0099122) for mobile element families -> cross-check candidate islands for integrase / transposase / phage signatures -> if AMR is annotated, cross-reference with AMRFinderPlus or CARD.

The five most reliable island indicators in order of strength: (1) integrase / transposase / phage protein co-located; (2) tRNA gene at one boundary (integration hotspot); (3) GC% deviation > 2 SD from genome mean; (4) absence in close relatives; (5) DNA recognition sites for known integrases (e.g. attL/attR).

Reconciliation: When Methods Disagree

Operational rule for publication: Concordant evidence from 2+ signal classes + contamination-cleared assembly + donor lineage with > 5 sampled relatives + ALE/AvP phylogenetic support with AU p < 0.05 + DGL ruled out by sister-lineage inspection. Single-method (especially composition-only) claims should be downgraded to "candidate" status.

Cohort Gotchas

• Genome MAGs from metagenomes: completeness and contamination matter more than for isolates. Apply CheckM2 first; HGT detection on incomplete MAGs is unreliable.
• Streptomyces, Borrelia, Mollicutes: genomes with extreme GC variance or unusual replication; composition methods produce many false positives. Raise z-threshold to 3 and rely more on phylogenetic methods.
• Endosymbionts (Buchnera, Wolbachia): strong AT bias from genome reduction; gene from external donor with normal composition appears as HGT but may be ancestral. Use AleRax with reduced-effective-population-size aware priors.
• Plant genomes: plastid-to-nucleus EGT is widespread (~5-18% of nuclear plant proteome plastid-derived per Martin 2002 PNAS 99:12246); standard HGT tools detect these but they are not "horizontal" in the prokaryote sense.
• Animal microbiome-associated organisms: consortium contamination is common in non-axenic cultures; pre-2010 published "HGTs" in such organisms require FCS-GX re-screening.
• Single-cell amplified genomes (SAGs): chimeric MDA artifacts mimic HGT; require co-assembly validation before HGT calls.

Anticipated Reviewer Pushback

Common Errors

Tool Installation Notes

# HGTector v2
pip install hgtector
hgtector database --output ~/hgtector_db --reference refseq --rank species

# AvP
git clone https://github.com/GDKO/AvP && cd AvP && pip install .
# Requires DIAMOND, IQ-TREE, mafft on PATH

# ALE + GeneRax + AleRax
conda install -c bioconda ale generax
# AleRax: build from source per https://github.com/BenoitMorel/AleRax
git clone --recursive https://github.com/BenoitMorel/AleRax && cd AleRax && ./install.sh

# Contamination tools (REQUIRED for eukaryote HGT)
conda install -c bioconda fcs-gx blobtoolkit kraken2
pip install conterminator

# Mobile elements
git clone https://github.com/clb21565/mobileOG-db
conda install -c bioconda amrfinder

NCBI now mandates FCS-GX screening for new eukaryote genome submissions to GenBank (Astashyn 2024 GB 25:60); apply it to any pre-2023 published eukaryote assembly before HGT analysis.

References

• Lawrence JG & Ochman H 1997 J Mol Evol 44:383 (amelioration concept)
• Lawrence JG & Ochman H 1998 PNAS 95:9413 (amelioration quantification)
• Maddison WP 1997 Syst Biol 46:523 (gene tree species tree discordance causes)
• Felsenstein J 1978 Syst Zool 27:401 (LBA)
• Galtier N & Duret L 2007 Trends Genet 23:273 (gBGC review)
• Vernikos GS & Parkhill J 2006 Bioinformatics 22:2196 (AlienHunter)
• Waack S et al 2006 BMC Bioinf 7:142 (SIGI-HMM)
• Podell S & Gaasterland T 2007 Genome Biol 8:R16 (DarkHorse)
• Gladyshev EA et al 2008 Science 320:1210 (Alien Index; bdelloid rotifer HGT)
• Szöllősi GJ et al 2013 Syst Biol 62:901 (ALE undated)
• Bansal MS et al 2018 Bioinformatics 34:3214 (RANGER-DTL 2.0)
• Bertelli C et al 2017 NAR 45:W30 (IslandViewer 4)
• Williams TA et al 2017 PNAS 114:E4602 (ALE rooting of the archaeal tree of life)
• Boothby TC et al 2015 PNAS 112:15976 (tardigrade HGT claim)
• Koutsovoulos G et al 2016 PNAS 113:5053 (tardigrade contamination refutation)
• Crisp A et al 2015 Genome Biol 16:50 (human HGT claim)
• Salzberg SL 2017 Genome Biol 18:85 (human HGT contamination refutation)
• Zhu Q, Kosoy M & Dittmar K 2014 BMC Genomics 15:717 (HGTector)
• Koutsovoulos G et al 2022 PLoS Comp Biol 18:e1010686 (AvP)
• Yuan L et al 2023 Brief Bioinform 24:bbad035 (HGTphyloDetect)
• Morel B et al 2020 MBE 37:2763 (GeneRax)
• Morel B et al 2024 Bioinformatics 40:btae162 (AleRax co-estimation)
• Brown CL et al 2022 Appl Environ Microbiol 88:e0099122 (mobileOG-db)
• Song W et al 2019 Microbiome 7:36 (MetaCHIP)
• Astashyn A et al 2024 GB 25:60 (FCS-GX)
• Wood DE et al 2019 GB 20:257 (Kraken2)
• Steinegger M et al 2020 GB 21:115 (Conterminator)
• Challis R et al 2020 G3 10:1361 (BlobTools2)
• Ravenhall M et al 2015 PLoS Comp Biol 11:e1004095 (HGT inference review)
• Martin W et al 2002 PNAS 99:12246 (plastid EGT)
• Capra JA et al 2013 PLoS Genet 9:e1003684 (gBGC quantification)
• Solis-Lemus C & Ane C 2016 PLoS Genet 12:e1005896 (SNaQ / PhyloNetworks reticulation)
• Shimodaira H 2002 Syst Biol 51:492 (AU test)

Related Skills

• comparative-genomics/gene-tree-species-tree-reconciliation - Deep DTL methodology (ALE / GeneRax / AleRax)
• comparative-genomics/ortholog-inference - Orthogroup construction for ALE input
• comparative-genomics/introgression-detection - Distinguish HGT from hybridization via D-statistic
• comparative-genomics/genome-distance-and-species-delineation - ANI / dDDH for donor-recipient distance
• comparative-genomics/pangenome-analysis - Bacterial pangenome assembly for accessory-gene HGT analysis
• phylogenetics/modern-tree-inference - Gene-tree inference with UFBoot for ALE
• phylogenetics/bayesian-inference - PhyloBayes-MPI CAT-GTR for deep-node LBA mitigation
• alignment/multiple-alignment - Codon-aware alignment for compositional analyses
• metagenomics/amr-detection - Mobile element / resistance gene context for HGT candidates
• genome-annotation/prokaryotic-annotation - Mobile-element annotation alongside gene annotation