Musals Eval

musals-eval

MuSAlS: A Fast Multiple Sequence Alignment Approach Using Hierarchical Clustering — Light et al. (2026) (arXiv:2601.15458, 2026)

What this evaluates

Evaluates the computational efficiency and alignment quality of a multiple sequence alignment algorithm across genomic and protein datasets. It probes the trade-off between runtime scalability and evolutionary accuracy metrics like distance distortion and gap percentage.

Datasets

• Greengenes 12.10 — total ?; splits: test (-1)
• Greengenes 13.5 — total ?; splits: test (-1)
• PDB — total ?; splits: test (-1)
• PFam-10k — total 10000; splits: test (10000)
• PFam-100k — total 100000; splits: test (100000)
• PFam-1M — total 1000000; splits: test (1000000)

Metrics

• runtime (primary) — range: other
  • Wall-clock time in seconds required to complete the alignment process.
• alignment width — range: other
  • Total number of columns in the resulting multiple sequence alignment.
• alignment stretch — range: other
  • Ratio measuring the expansion of the alignment relative to the original sequences.
• percentage of gaps — range: percent
  • Proportion of gap characters in the alignment matrix.
• distance distortion (primary) — range: other
  • Metric quantifying the deviation of pairwise evolutionary distances in the alignment from reference distances.
• minimum p distance — range: [0, 1]
  • Minimum pairwise p-distance (proportion of differing sites) across all sequence pairs.
• mean p distance — range: [0, 1]
  • Average pairwise p-distance across all sequence pairs.
• maximum p distance — range: [0, 1]
  • Maximum pairwise p-distance across all sequence pairs.

Input / output format

Input: Raw nucleotide or protein sequence datasets for benchmarking.

Output: Aligned multiple sequence alignment (MSA) output, plus computed statistics for runtime, width, stretch, gap percentage, distance distortion, and p-distances.

Scoring recipe

def score_msa(alignments, runtimes):
    results = {}
    for alg, (msa, time) in alignments.items():
        results[alg] = {
            'runtime': time,
            'width': msa.num_columns,
            'stretch': msa.width / msa.max_seq_length,
            'gap_pct': msa.count_gaps() / msa.total_cells * 100,
            'distortion': compute_distance_distortion(msa),
            'p_min': min(pairwise_pdist(s1, s2) for s1, s2 in msa.pairs),
            'p_avg': mean(pairwise_pdist(s1, s2) for s1, s2 in msa.pairs),
            'p_max': max(pairwise_pdist(s1, s2) for s1, s2 in msa.pairs)
        }
    return results

Common pitfalls

• Several baseline algorithms fail silently or abort on specific datasets (e.g., MAGUS on proteins, KAlign on non-standard PDB residues), so missing table entries indicate compatibility limits rather than poor performance.
• Optimizing for compactness (low width and gap percentage) can trade off with evolutionary accuracy, as seen when MuSAlS achieves tighter alignments but higher distance distortion on Greengenes datasets.

Evidence (verbatim from paper)

We measured performance across various datasets (Greengenes 12.10, Greengenes 13.5, PDB, and PFam subsets) using the evaluation metrics runtime, alignment width, alignment stretch, percentage of gaps, distance distortion, minimum p distance, mean p distance, and maximum p distance.

Citation

@misc{light2026musals,
  title={MuSAlS: A Fast Multiple Sequence Alignment Approach Using Hierarchical Clustering},
  author={Light et al. (2026)},
  year={2026},
  note={arXiv:2601.15458}
}

• arXiv: 2601.15458