Fuximt Xxzh Eval

fuximt-xxzh-eval

FuxiMT: Sparsifying Large Language Models for Chinese-Centric Multilingual Machine Translation — Shaolin Zhu et al. (2025) (arXiv:2505.14256, 2025)

What this evaluates

Evaluates multilingual machine translation capability for Chinese-involved pairs (xx-zh), measuring translation quality across varying levels of parallel data availability. It probes how well models leverage cross-lingual knowledge transfer and handle data scarcity in low-resource settings.

Datasets

• xx-zh translation pairs — total ?; splits: (unstated)

Metrics

• BLEU (primary) — range: percent
  • Standard sentence-level BLEU score (n-gram precision with brevity penalty) computed on xx-zh translation pairs.

Input / output format

Input: Source sentence in a target language (xx)

Output: Translated sentence in Chinese (zh)

Scoring recipe

def compute_bleu(predictions, references):
    # predictions and references are lists of strings
    # Compute sentence-level BLEU (typically BLEU-4 with brevity penalty)
    # Average BLEU scores across all test instances
    return average_bleu(predictions, references)

Common pitfalls

• BLEU is reported exclusively for the xx-zh direction; bidirectional or zh-xx scores are not provided.
• Resource categories (High/Medium/Low/Very Low) are defined by the volume of available parallel data, not by language family or typology.
• Ablation results report a single 'Average BLEU' across pairs, while main results are stratified by resource level.

Evidence (verbatim from paper)

Table [1] presents the performance of FuxiMT and the baselines across different resource levels of languages, categorized as High, Medium, Low, and Very Low resource in terms of the amount of parallel data available (as described in Appendix Table [5]).

Table 1: BLEU scores for different resource languages xx-zh.

Citation

@misc{zhu2025fuximt,
  title={FuxiMT: Sparsifying Large Language Models for Chinese-Centric Multilingual Machine Translation},
  author={Shaolin Zhu et al. (2025)},
  year={2025},
  note={arXiv:2505.14256}
}

• arXiv: 2505.14256