qhjqhj00 thai-ser-eval
THAI Speech Emotion Recognition (THAI-SER) corpus — Wongpithayadisai et al. (2025) (arXiv:2507.09618, 2025)
What this evaluates
Evaluates speech emotion recognition models on a culturally grounded Thai speech corpus, testing their ability to classify utterances into five emotion categories (neutral, angry, happy, sad, frustrated) across different recording environments and cross-corpus settings.
Datasets
- THAI-SER — total 27854; splits: train (-1), val (-1), test (-1); repo https://github.com/vistec-AI/dataset-releases
Metrics
weighted accuracy(primary) — range: percent- Computed by dividing the number of correctly classified samples by the total number of samples.
unweighted accuracy— range: percent- Calculated by taking the average of per-class accuracy across all emotion categories.
Input / output format
Input: 64 mel-scale filterbank coefficients (25ms frame length, 10ms shift), split into 3-second intervals (padded if shorter), normalized via CMVN, with VTLP augmentation applied to training samples.
Output: Discrete emotion class label from {neutral, angry, happy, sad, frustrated} (or a 4-class subset excluding frustrated).
Scoring recipe
def compute_metrics(predictions, gold_labels):
total = len(gold_labels)
correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g)
weighted_acc = correct / total
classes = set(gold_labels)
per_class_acc = []
for cls in classes:
cls_mask = [g == cls for g in gold_labels]
if sum(cls_mask) > 0:
cls_correct = sum(1 for p, g in zip(predictions, gold_labels) if p == g and g == cls)
per_class_acc.append(cls_correct / sum(cls_mask))
unweighted_acc = sum(per_class_acc) / len(per_class_acc)
return weighted_acc, unweighted_acc
Common pitfalls
- Zoom recordings are explicitly excluded from the baseline k-fold cross-validation due to domain mismatch and audio compression artifacts; they are reserved as a separate out-of-domain challenge set.
- Training data must be filtered using an agreement score threshold of 0.71 to remove samples with low inter-annotator reliability, which otherwise confuses the model.
- Cross-corpus comparisons (e.g., with IEMOCAP) require pruning both datasets to match speaker counts and total hours per fold to avoid unfair size imbalances.
Evidence (verbatim from paper)
We chose weighted accuracy and unweighted accuracy as the evaluation metrics. These two metrics have been used in various literature (Satt, Rozenberg, and Hoory (2017), Ando et al. (2018), and Etienne et al. (2018)). The weighted accuracy is computed by dividing the number of correctly classified samples by the total number of samples. On the other hand, unweighted accuracy is calculated by taking an average of per-class accuracy.
Citation
@misc{wongpithayadisai2025thai,
title={THAI Speech Emotion Recognition (THAI-SER) corpus},
author={Wongpithayadisai et al. (2025)},
year={2025},
note={arXiv:2507.09618}
}
- arXiv: 2507.09618