Iemocap Eval

iemocap-eval

Modelling Emotions is an Elusive Pursuit in Affective Computing — Larsen et al. (2026) (arXiv:2603.23017, 2026)

What this evaluates

Evaluates how categorical and continuous label ambiguity impacts the performance of unimodal emotion recognition models (text, audio, facial) on the IEMOCAP dataset. It tests whether filtering data by annotator agreement or VAD score dispersion yields cleaner evaluation signals. The protocol highlights the disconnect between rigid single-label benchmarks and the inherent ambiguity of affective data.

Datasets

• IEMOCAP — total ?; splits: test (-1)

Metrics

• weighted F1 score (primary) — range: [0, 1]
  • Averages per-class F1 scores weighted by label frequency to account for class imbalance.

Input / output format

Input: Audio, facial, and text modalities of utterances from the IEMOCAP dataset, paired with categorical emotion annotations and/or VAD scores.

Output: Predicted categorical emotion label for each utterance.

Scoring recipe

def weighted_f1(predictions, gold, classes):
    class_counts = Counter(gold)
    total = sum(class_counts.values())
    f1_scores = []
    for c in classes:
        tp = sum(1 for p, g in zip(predictions, gold) if p == c and g == c)
        fp = sum(1 for p, g in zip(predictions, gold) if p == c and g != c)
        fn = sum(1 for p, g in zip(predictions, gold) if p != c and g == c)
        prec = tp / (tp + fp) if (tp + fp) > 0 else 0
        rec = tp / (tp + fn) if (tp + fn) > 0 else 0
        f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
        f1_scores.append(f1)
    weights = [class_counts[c] / total for c in classes]
    return sum(f * w for f, w in zip(f1_scores, weights))

Common pitfalls

• Treating categorical emotion labels as accurate ground truth despite only ~20% full annotator agreement.
• Assuming that filtering by VAD score dispersion will improve model performance, as it actually decreases or shows no improvement.
• Expecting high agreement across text, audio, and facial modalities, when full agreement occurs in only 4.18% of utterances.

Evidence (verbatim from paper)

Model performance was evaluated using the weighted F1 score, which averages per-class F1 scores weighted by label frequency to account for class imbalance.

Citation

@misc{larsen2026modelling,
  title={Modelling Emotions is an Elusive Pursuit in Affective Computing},
  author={Larsen et al. (2026)},
  year={2026},
  note={arXiv:2603.23017}
}

• arXiv: 2603.23017