Bio Comparative Genomics Whole Genome Alignment

Version Compatibility

Reference examples tested with: Progressive Cactus 2.9.1+ (ComparativeGenomicsToolkit/cactus; Armstrong 2020 Nature 587:246), Minigraph-Cactus (Hickey 2024 Nat Biotech 42:663; bundled with Cactus 2.5+), HAL toolkit 2.3+ (Hickey 2013 Bioinformatics 29:1341), LASTZ 1.04.22+, UCSC kentUtils for chain/net (Kent 2003 PNAS 100:11484), MUMmer 4.0.0+, minimap2 2.28+, AnchorWave 1.2.5+, progressiveMauve 2.4.0+, sibeliaz 1.2.5+, winnowmap 2.03+ (Jain 2022 Nat Methods 19:705). Toil workflow runner 6.0+ for Cactus on HPC/cloud.

Before using code patterns, verify installed versions match. If versions differ:

• CLI: cactus --help, cactus-pangenome --help, halStats --help, lastz --version, nucmer --version, minimap2 --version
• Python: pip show toil, toil --version

If code throws Toil workflow restart failure, HAL file corrupted, WDL workflow missing, the Cactus pipeline is Toil-based and requires careful checkpointing; failed runs must be restarted with --restart. HAL file versions differ across hal-toolkit releases; pin the version that produced the file.

Whole Genome Alignment

"Align these multiple genomes at the base-pair level" -> Choose between reference-free progressive alignment (Cactus / Minigraph-Cactus: produces HAL, no privileged reference) and reference-anchored pairwise alignment (LASTZ chains/nets, MUMmer, minimap2: one genome is the reference, queries align to it). The fundamental tradeoff is scale vs structure: pairwise pipelines scale linearly per pair but lose multi-way relationships; progressive Cactus scales linearly with a tree but quadratically without and produces ancestrally-coherent alignments. For comparative genomics at vertebrate / mammal scale, Cactus is now the standard substrate (Zoonomia, Christmas 2023 Science 380:eabn3943; Bird10000 Genomes); for pangenome graph construction, Minigraph-Cactus (Hickey 2024) is the production pipeline.

• CLI: cactus jobStore seqFile.txt output.hal --binariesMode local -- reference-free progressive WGA
• CLI: cactus-pangenome --reference ref name --vcf -- pangenome graph from genomes
• CLI: lastz target.fa[multiple] query.fa then UCSC chain-net pipeline -- pairwise to a reference
• CLI: minimap2 -ax asm5 ref.fa query.fa | samtools sort -- fast pairwise for closely related
• CLI: nucmer --maxmatch ref.fa query.fa then dnadiff -- MUMmer4 pairwise
• CLI: anchorwave proali --ploidy 4 -- WGD-aware sequence-level synteny alignment

Algorithmic Taxonomy

Methodology evolves; Cactus / Minigraph-Cactus / HAL are now the dominant production pipelines for vertebrate / mammal / plant WGA. LASTZ chains/nets remain the standard for adding a single new species to a reference-anchored ecosystem (UCSC, Ensembl).

Decision Tree by Experimental Scenario

Per-Tool Failure Modes

Cactus running quadratically without species tree

Trigger: Running cactus without --guide-tree or with a poorly-resolved guide tree.

Mechanism: Cactus is designed as a progressive aligner: it aligns siblings, then their ancestor, recursively up the species tree. Without a guide tree, it tries all-vs-all alignment, which is O(N^2) and explodes for > 10 genomes.

Symptom: Cactus job time scales as N^2; cluster runs hit wall time before completing.

Fix: Always provide a guide tree in seqFile.txt:

(((human:0.05, chimp:0.05):0.05, macaque:0.1):0.2, mouse:0.5);
human    /path/to/human.fa
chimp    /path/to/chimp.fa
macaque  /path/to/macaque.fa
mouse    /path/to/mouse.fa

Branch lengths in substitutions per site; need not be precise but must reflect relative divergence. For poorly-resolved trees, use STAR (concatenation) tree from a few hundred concatenated single-copy orthologs (cf. [[ortholog-inference]]).

Branch-scale parameter mis-tuned

Trigger: Cactus alignment producing very few or very many alignment columns relative to expected.

Mechanism: Cactus uses --branchScale to scale internal-branch lengths for LASTZ chaining parameters. Default scaling assumes vertebrate-like substitution rates; plants, bacteria, and viruses need adjustment. Wrong scaling produces under-aligned (too low) or over-merged (too high) blocks.

Symptom: HAL file shows alignment column count vastly different from expected; halStats reports unusual coverage; comparative annotation downstream fails.

Fix: Adjust --branchScale per clade: vertebrates default 1.0; plants 0.5-0.7; bacteria 0.3-0.5; viruses 0.1-0.3. Validate against known orthologs (BUSCO single-copies should align across all genomes).

Toil checkpoint restart failures

Trigger: Cactus job interrupted (cluster timeout, OOM, node failure) and restart.

Mechanism: Cactus uses Toil's job store for checkpointing. Restart requires identical jobStore path, identical config, and pointing to the same machine type (some cloud-stored jobs require AWS / Google Cloud credentials).

Symptom: Cactus restart errors with "job store inconsistent" or "missing intermediate file."

Fix: Use cactus --restart jobStore seqFile.txt output.hal. For HPC, ensure the jobStore path is on shared storage accessible to all nodes. For cloud, use S3 or Google Cloud Storage with proper credentials. If restart fails, restart from scratch with a new jobStore.

Reference bias in LASTZ chain/net

Trigger: Reporting "human-chimp synteny" from LASTZ chains/nets with human as reference.

Mechanism: Chain/net is reference-anchored; the alignment is biased toward the reference's genomic context. Repeat-content differences and assembly-quality differences between reference and query produce alignment artifacts.

Symptom: Different chains/nets from different reference choices show inconsistent SV calls; e.g. mouse-as-reference and human-as-reference produce different INV counts.

Fix: For multi-genome questions, use Cactus (reference-free). For pairwise questions, run chain/net both directions (A-on-B and B-on-A) and report consistent calls. Document reference choice in methods. SyRI on minimap2 alignments has similar issues; cross-validate.

minimap2 sensitivity loss at < 70% identity

Trigger: Running minimap2 -x asm10 or -x asm20 between distantly related species.

Mechanism: Minimap2's minimizer-based seeding loses sensitivity at lower identity; many regions never anchor and produce gaps in alignment.

Symptom: Alignment coverage low (< 50% of query mapped); SyRI / SV-calling on this alignment reports few or no SVs; high-identity regions align but mid-identity regions don't.

Fix: Switch to LASTZ for < 70% identity; LASTZ uses 6-bp seeds with scoring matrices designed for distant comparison (HoxD55 for human-fish). For minimap2, try -x asm20 with manual sensitivity tuning (-k 19 -w 19) but expect inferior performance vs LASTZ. minimap2 documentation recommends presets up to 20% divergence.

Cactus producing wrong topology in ancestor assembly

Trigger: Cactus is asked to assemble an ancestral sequence at an internal node, but the input tree has wrong topology.

Mechanism: Cactus uses the guide tree to determine which descendants align to which ancestor. Wrong topology produces incorrect ancestor sequence at internal nodes.

Symptom: Ancestor sequences inconsistent with biological expectation; downstream phylogenetic analyses on Cactus ancestor sequences disagree with concatenation phylogenies.

Fix: Validate species tree before Cactus alignment using phylogenetics/modern-tree-inference on concatenated single-copy orthologs. If topology is uncertain, use a more conservative resolution (e.g. polytomy resolved via STRIDE).

Repeats inflating LASTZ chain count

Trigger: Running LASTZ on unmasked or weakly masked genomes.

Mechanism: Unmasked TEs produce millions of paralogous LASTZ alignments; chains form from these, creating false-positive synteny / alignment.

Symptom: Chain count >> 100,000 per pair; many short chains in TE-rich regions; chain/net pipeline produces excessive chains where biology predicts few.

Fix: Softmask both genomes (RepeatModeler2 + RepeatMasker) before LASTZ. LASTZ honors softmasked sequences in its seeding step (lowercase = ignored at seeding, full alignment after extension). Hardmasking (N's) is too aggressive and breaks LASTZ scoring.

HAL version mismatch breaking downstream tools

Trigger: Using HAL files generated with one version of Cactus / HAL toolkit with downstream tools using a different version.

Mechanism: HAL file format is versioned; minor version mismatch usually OK, major version mismatch causes silent or loud failures.

Symptom: halStats / TOGA / halLiftover errors about "unrecognized HAL version."

Fix: Use the HAL toolkit version that wrote the file (check via halStats --filter halVersion or git log of the run). For shared HAL files, pin Cactus version in documentation.

Insufficient masking before Cactus

Trigger: Cactus on unmasked vertebrate genomes.

Mechanism: Cactus's CAF/BAR algorithms identify candidate alignment columns; without masking, TE-derived blocks dominate. Memory consumption explodes and alignment results are mostly TE pseudo-alignments.

Symptom: Cactus runs out of memory; HAL file is huge (> 100 GB for what should be 10 GB); halStats shows enormous column count.

Fix: Mask all genomes with RepeatMasker (or species-specific RepeatModeler2 library) before Cactus. Soft-masking is acceptable; Cactus handles lowercase appropriately. Document masking strategy.

AnchorWave WGD parameter mis-set

Trigger: AnchorWave proali with wrong --ploidy value for the query genome.

Mechanism: AnchorWave's WGD-aware alignment matches each query CDS to ploidy-many reference CDS regions; wrong ploidy causes either undermatching (low ploidy) or overmatching (high ploidy).

Symptom: Reported alignment count inconsistent with biology (e.g. hexaploid wheat reports diploid-level matches).

Fix: Set --ploidy to query genome's ploidy (1 for haploid reference, 2 for diploid query when reference is diploid, 4 for tetraploid query vs diploid reference). Check with python -m jcvi.compara.synteny depth on output anchors.

Quantitative Thresholds

Progressive Cactus Standard Workflow

Goal: Build a multi-species reference-free WGA from N genomes.

Approach: Provide guide tree + per-species genome FASTA in seqFile.txt -> run cactus with Toil job store -> produce HAL -> extract MAF / synteny / annotation as needed.

# 1. Pre-mask genomes (softmask)
for fa in genomes/*.fa; do
    species=$(basename $fa .fa)
    BuildDatabase -name ${species}_DB $fa
    RepeatModeler -database ${species}_DB -threads 16
    RepeatMasker -lib ${species}_DB-families.fa -xsmall -pa 16 $fa
done

# 2. Prepare seqFile (tree + paths)
cat > seqFile.txt << 'EOF'
(((human:0.05, chimp:0.05):0.05, macaque:0.1):0.2, mouse:0.5);
human    genomes/human.fa.masked
chimp    genomes/chimp.fa.masked
macaque  genomes/macaque.fa.masked
mouse    genomes/mouse.fa.masked
EOF

# 3. Run Cactus
cactus jobStore_path seqFile.txt output.hal \
    --binariesMode local \
    --workDir /tmp/cactus_work \
    --maxCores 64 \
    --logFile cactus.log

# 4. Inspect HAL
halStats output.hal
# Reports: genome count, chromosome / contig count per genome, alignment column count

# 5. Extract MAF for downstream phyloP / conservation
hal2maf output.hal reference_genome ucsc.maf --refGenome human --chunkSize 1000000

For HPC deployment, use Toil's --batchSystem slurm or --batchSystem kubernetes for cluster-scale runs.

LASTZ + UCSC Chain/Net Standard Pipeline

Goal: Pairwise WGA of a query genome to a reference, producing UCSC-format chain and net files.

Approach: LASTZ pairwise -> axtChain -> chainNet -> netSyntenic.

# 1. LASTZ pairwise alignment per chromosome
for chr in $(cut -f1 reference.fa.fai); do
    lastz reference.fa[multiple,unmask] query.fa[multiple,unmask] \
        --hspthresh=3000 --ydrop=9400 --gappedthresh=3000 \
        --ambiguous=iupac --inner=2000 --score=$LASTZ_PARAMS \
        --format=axt \
        > pairwise/${chr}.axt
done

# 2. Build chains. `axtChain` reads PSL when given `-psl`; for AXT input from lastz
# (`--format=axt`) drop the `-psl` flag, or emit PSL from LASTZ with `--format=psl` upstream.
for axt in pairwise/*.axt; do
    chr=$(basename $axt .axt)
    axtChain -linearGap=loose $axt reference.2bit query.2bit chains/${chr}.chain
done

# 3. Merge chains
chainMergeSort chains/*.chain > merged.chain
chainPreNet merged.chain reference.size query.size pre.chain

# 4. Build net (one-to-one selection)
chainNet pre.chain reference.size query.size \
    target.net query.net
netSyntenic target.net target.syntenic.net

# 5. Filter and convert
netToAxt target.syntenic.net pre.chain reference.2bit query.2bit final.axt
axtToMaf final.axt reference.size query.size final.maf

Minigraph-Cactus for Pangenome Construction

Goal: Build a pangenome graph from related genomes (intra-species or sister-species).

Approach: Provide reference + alternative genomes -> minigraph for SV graph -> Cactus for base-level alignment.

# Prepare seqFile (similar to Cactus but designates reference)
cat > pangenome_seqs.txt << 'EOF'
reference   human.fa
hap1        haplotype1.fa
hap2        haplotype2.fa
hap3        haplotype3.fa
EOF

# Run minigraph-cactus
cactus-pangenome jobStore_path pangenome_seqs.txt \
    --outDir output_pangenome \
    --outName pangenome \
    --reference reference \
    --vcf \
    --gfa \
    --gbz \
    --indexCores 32 \
    --mapCores 32

# Outputs:
#   pangenome.gbz       Genome Browser Z compressed graph
#   pangenome.gfa.gz    Graph Fragment Assembly format
#   pangenome.vcf.gz    VCF for short variants from graph
#   pangenome.full.hal  Full Cactus HAL for downstream tools

For HPRC-scale (90 haplotypes), use --mapCores 64 --indexCores 64 on a high-memory node. See [[pangenome-analysis]] for downstream graph analysis.

Reconciliation: When Methods Disagree

Operational rule for publication: Multi-species WGA for comparative genomics uses Progressive Cactus with documented guide tree, masking, and Toil configuration. Pairwise reference alignments use LASTZ chain/net for UCSC tracks or minimap2 for fast pangenomics. Document choice of alignment tool, reference (if applicable), parameter values, and any non-default settings.

Cohort Gotchas

• Highly repetitive genomes (large mammals, plants): memory consumption explodes; ensure >= 256 GB RAM per node
• Centromeric / telomeric regions: unalignable in most assemblies; Winnowmap2 for explicit telomere-to-telomere work; otherwise mask
• Sex chromosomes: different content; Cactus aligns where possible; downstream analyses must respect sex-chromosome specifics
• Recently diverged inbred strains (e.g. mouse strains): > 99% identity; minimap2 -x asm5 is fine; Cactus overkill
• Pangenome with structural variants: Minigraph-Cactus preferred over straight Cactus
• Multi-region polyploids: AnchorWave proali with ploidy specification; subgenome-aware

Anticipated Reviewer Pushback

Common Errors

Tool Installation Notes

# Progressive Cactus (Docker / Singularity preferred for reproducibility)
docker pull quay.io/comparative-genomics-toolkit/cactus:latest
# Or via virtualenv
python3 -m venv cactus_env && source cactus_env/bin/activate && pip install cactus

# HAL toolkit (usually bundled with Cactus)
git clone https://github.com/ComparativeGenomicsToolkit/hal && cd hal && make

# LASTZ + UCSC kentUtils
conda install -c bioconda lastz ucsc-axt-chain ucsc-chain-merge-sort ucsc-chain-pre-net ucsc-chain-net ucsc-net-syntenic ucsc-net-to-axt

# MUMmer4
conda install -c bioconda mummer

# minimap2 + winnowmap2
conda install -c bioconda minimap2 winnowmap

# AnchorWave
conda install -c bioconda anchorwave

# progressiveMauve
conda install -c bioconda mauve

# SibeliaZ
conda install -c bioconda sibeliaz

# RepeatMasker / RepeatModeler2 (for masking)
conda install -c bioconda repeatmasker repeatmodeler

# Toil for HPC / cloud
pip install toil[all]

For Cactus on HPC, set up Toil with the appropriate batch system (--batchSystem slurm); for cloud, use Toil's S3/GCS support and a job store on shared cloud storage.

References

• Armstrong J et al 2020 Nature 587:246 (Progressive Cactus)
• Hickey G et al 2024 Nat Biotech 42:663 (Minigraph-Cactus)
• Hickey G et al 2013 Bioinformatics 29:1341 (HAL toolkit)
• Kent WJ et al 2003 PNAS 100:11484 (UCSC chain/net)
• Schwartz S et al 2003 Genome Res 13:103 (LASTZ)
• Marçais G et al 2018 PLoS Comp Biol 14:e1005944 (MUMmer4)
• Li H 2018 Bioinformatics 34:3094 (minimap2)
• Li H 2021 Bioinformatics 37:4572 (minimap2 long-read updates)
• Song B et al 2022 PNAS 119:e2113075119 (AnchorWave)
• Darling AE et al 2010 PLoS One 5:e11147 (progressiveMauve)
• Minkin I & Medvedev P 2020 Nat Commun 11:6327 (SibeliaZ)
• Jain C et al 2022 Nat Methods 19:705 (Winnowmap2)
• Blanchette M et al 2004 Genome Res 14:708 (Multiz)
• Brudno M et al 2003 GR 13:721 (LAGAN)
• Liao W-W et al 2023 Nature 617:312 (HPRC draft pangenome)
• Sirén J et al 2024 Nat Methods 21:2017 (vg personalized pangenome references). The vg-Giraffe pangenome mapper is Sirén J et al 2021 Science 374:abg8871.
• Garrison E et al 2024 Nat Methods 21:2008 (PGGB)
• Paten B et al 2011 Genome Res 21:1512 (Cactus algorithm)
• ComparativeGenomicsToolkit Cactus + HAL toolkit documentation (https://github.com/ComparativeGenomicsToolkit)

Related Skills

• comparative-genomics/synteny-analysis - Synteny detection from WGA (Cactus -> halSynteny)
• comparative-genomics/comparative-annotation-projection - TOGA + CESAR uses Cactus HAL
• comparative-genomics/pangenome-analysis - Minigraph-Cactus / PGGB for pangenome graph construction
• comparative-genomics/whole-genome-duplication - Ks-dating uses WGA-derived gene pair alignments
• alignment/multiple-alignment - MAFFT / MUSCLE for protein MSA from WGA-derived orthologs
• alignment/pairwise-alignment - LASTZ / minimap2 / MUMmer for sequence-level comparison
• alignment/structural-alignment - WGA-extracted protein orthologs for downstream alignment
• genome-assembly/assembly-qc - BUSCO / Compleasm checks before WGA
• variant-calling/structural-variant-calling - WGA-based SV detection from genome assemblies
• causal-genomics/heritability-partitioning - LDSC partitioning using phyloP scores from Cactus HAL