Virus Host Prediction Eval

virus-host-prediction-eval

Computational approaches for virus host prediction: A review of methods and applications — Shang et al. (2025) (arXiv:2509.00349, 2025)

What this evaluates

Evaluates computational tools and genomic features for predicting prokaryotic virus-host interactions. It probes the ability of models to correctly link viral sequences to their host taxa using either pairwise link prediction or taxonomic classification formulations.

Datasets

• RefSeq-VHDB — total 4698; splits: test (4698)
• MetaHiC-VHDB — total ?; splits: test (-1)

Metrics

• Top-1 accuracy (primary) — range: [0, 1]
  • The proportion of viruses for which the single highest-scoring predicted host matches the true host label.
• Top-N accuracy — range: [0, 1]
  • The proportion of viruses for which the true host appears anywhere in the top-N predicted hosts.
• alignment proportion — range: [0, 1]
  • The fraction of total viruses that yield at least one valid alignment or prediction.
• correct match ratio — range: [0, 1]
  • The ratio of correctly predicted host hits to the total number of alignment hits returned, estimating errors from Top-N predictions.

Input / output format

Input: Viral genomic sequence, optionally paired with candidate host genomes/MAGs or specific biological features (CRISPR spacers, prophage regions).

Output: Link prediction: probability score p ∈ [0,1] per virus-host pair. Classification: single taxonomic label (e.g., genus/family). Feature evaluation: list of alignment hits with pid and coverage/length.

Scoring recipe

def evaluate(predictions, gold, top_n=5):
    top1_correct = sum(1 for p, g in zip(predictions, gold) if p == g)
    top1_acc = top1_correct / len(gold)
    topn_correct = sum(1 for p_list, g in zip(predictions, gold) if g in p_list[:top_n])
    topn_acc = topn_correct / len(gold)
    aligned = sum(1 for p in predictions if len(p) > 0)
    align_prop = aligned / len(gold)
    total_hits = sum(len(p) for p in predictions)
    correct_hits = sum(1 for p_list, g in zip(predictions, gold) for p in p_list if p == g)
    correct_ratio = correct_hits / total_hits if total_hits > 0 else 0
    return top1_acc, topn_acc, align_prop, correct_ratio

Common pitfalls

• Extreme class imbalance in link prediction causes high false positive rates without stringent alignment thresholds.
• Top-N accuracy can be misleadingly high while yielding a low correct match ratio due to many incorrect hits.
• Closed-set classification cannot predict hosts from taxa absent in the training or reference database.
• Threshold sensitivity: minor changes in percentage identical matches (pid) drastically trade off accuracy against alignment proportion.

Evidence (verbatim from paper)

The performance was quantified using four metrics (see Methods): Top-1 accuracy (accuracy on the best hit), Top-N accuracy (accuracy on all alignment), alignment proportion (proportion of aligned viruses), and correct match ratio (estimate the errors introduced by using Top-N accuracy).

Citation

@misc{shang2025computational,
  title={Computational approaches for virus host prediction: A review of methods and applications},
  author={Shang et al. (2025)},
  year={2025},
  note={arXiv:2509.00349}
}

• arXiv: 2509.00349