# Erc Eval

> Evaluates a model's ability to recognize and classify emotional states from conversational dialogue context. It probes multi-turn emotional understanding, speaker-aware reasoning, and generalization across diverse domain settings and speaker demographics. Use when the user wants to benchmark on IEMOCAP, MELD, EmoryNLP, or asks about evaluating this task. Reports Accuracy.

- Skill: `qhjqhj00/erc-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/erc-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/erc-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/erc-eval

---


# erc-eval

> SELF-EMO: Emotional Self-Evolution from Recognition to Consistent Expression — Shaowei Zhang et al. (arXiv:2604.18003, 2026)

## What this evaluates

Evaluates a model's ability to recognize and classify emotional states from conversational dialogue context. It probes multi-turn emotional understanding, speaker-aware reasoning, and generalization across diverse domain settings and speaker demographics.

## Datasets

- **IEMOCAP** — total 7433; splits: train (5163), dev (647), test (1623)
- **MELD** — total 13708; splits: train (9989), dev (1109), test (2610)
- **EmoryNLP** — total 9489; splits: train (7551), dev (954), test (984)

## Metrics

- `Accuracy` **(primary)** — range: percent
  - Standard classification accuracy: the proportion of correctly predicted emotion labels out of the total number of test instances.
- `Weighted F1 (W-F1)` — range: percent
  - Weighted F1 score: the harmonic mean of precision and recall per class, averaged across all classes weighted by their support (number of true instances per class) to account for label imbalance.

## Input / output format

**Input**: Multi-turn dialogue context including previous utterances and speaker identifiers, with the model tasked to predict the emotion of the current target utterance.

**Output**: A single predicted emotion label corresponding to the target utterance (e.g., neutral, happy, sad, angry, or dataset-specific categories).

## Scoring recipe

```python
def compute_metrics(preds, golds):
    acc = sum(p == g for p, g in zip(preds, golds)) / len(golds) * 100
    classes = set(golds)
    w_f1 = 0.0
    total = len(golds)
    for cls in classes:
        tp = sum(1 for p, g in zip(preds, golds) if p == cls and g == cls)
        fp = sum(1 for p, g in zip(preds, golds) if p == cls and g != cls)
        fn = sum(1 for p, g in zip(preds, golds) if p != cls and g == cls)
        support = sum(1 for g in golds if g == cls)
        prec = tp / (tp + fp) if (tp + fp) > 0 else 0
        rec = tp / (tp + fn) if (tp + fn) > 0 else 0
        f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
        w_f1 += f1 * (support / total)
    return acc, w_f1 * 100
```

## Common pitfalls

- Evaluating on the development split instead of the official test split, which violates the paper's strict evaluation protocol.
- Training and testing on each dataset separately (dataset-specific setting) rather than the unified multi-dataset setting reported in the main results, leading to unfair or non-comparable baselines.
- Ignoring class imbalance when reporting metrics, as ERC datasets are heavily skewed; unweighted accuracy can be misleading without Weighted F1.

## Evidence (verbatim from paper)

> All model evaluations are strictly performed on the official test splits of these benchmarks. ... Specifically, our model realizes an accuracy increase of +6.33% on the Qwen3-4B backbone and +8.54% on the Qwen3-8B backbone, with consistent improvements observed across all benchmarks.

## Citation

```bibtex
@misc{zhang2026selfemo,
  title={SELF-EMO: Emotional Self-Evolution from Recognition to Consistent Expression},
  author={Shaowei Zhang et al.},
  year={2026},
  note={arXiv:2604.18003}
}
```

- arXiv: 2604.18003