M Absa Eval

m-absa-eval

M-ABSA: A Multilingual Dataset for Aspect-Based Sentiment Analysis — Wu et al. (2025) (arXiv:2502.11824, 2025)

What this evaluates

Evaluates multilingual aspect-based sentiment analysis (ABSA) models on triplet extraction (aspect term, category, sentiment) and pairwise extraction (aspect term, sentiment) across 21 languages and 7 domains. Probes cross-lingual transfer, cross-domain adaptation, and zero-shot LLM prompting capabilities.

Datasets

• M-ABSA — total ?; splits: train (-1), test (-1); repo https://github.com/swaggy66/M-ABSA

Metrics

• Micro-F1 (primary) — range: percent
  • Exact match F1: a prediction is correct if and only if all predicted sentiment elements in the pair or triplet match the gold standard. Reported as the average over 5 random seeds.

Input / output format

Input: Text reviews in 21 languages across 7 domains (Coursera, Food, Hotel, Laptop, Phone, Restaurant, Sightseeing). For LLM zero-shot evaluation, structured prompts are used to extract aspect terms, categories, and sentiment polarities.

Output: Extracted tuples: (aspect term, sentiment polarity) for UABSA, or (aspect term, aspect category, sentiment polarity) for TASD.

Scoring recipe

correct = 0
total = 0
for pred, gold in zip(predictions, gold_labels):
    if set(pred) == set(gold):
        correct += 1
    total += 1
return (correct / total) * 100

Common pitfalls

• Confusing TASD (triplet extraction) with UABSA (pairwise extraction); TASD is significantly harder and yields lower scores across all models.
• Assuming cross-lingual transfer performance scales linearly with resource availability; typological similarity (e.g., West Germanic) often matters more than raw resource count.
• Not averaging over 5 random seeds, which the authors explicitly state is required for reporting results.

Evidence (verbatim from paper)

We adopt Micro-F1 scores as the main evaluation metrics for all tasks. A prediction is correct if and only if all its predicted sentiment elements in the pair or triplet are correct. All the experimental results are reported using the average of 5 random seeds.

Citation

@misc{wu2025mabsa,
  title={M-ABSA: A Multilingual Dataset for Aspect-Based Sentiment Analysis},
  author={Wu et al. (2025)},
  year={2025},
  note={arXiv:2502.11824}
}

• arXiv: 2502.11824