# Humdial Eibench Eval

> humdial-eibench-eval

- Skill: `qhjqhj00/humdial-eibench-eval` (Agent Skill)
- Install (CLI): `npx skillmds@latest add qhjqhj00/humdial-eibench-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/humdial-eibench-eval/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-21
- Page: https://skillmd.com/skills/qhjqhj00/humdial-eibench-eval

---


# humdial-eibench-eval

> HumDial-EIBench: A Human-Recorded Multi-Turn Emotional Intelligence Benchmark for Audio Language Models — Wang et al. (2026) (arXiv:2604.11594, 2026)

## What this evaluates

Evaluates the emotional intelligence of audio language models across multi-turn dialogues. It probes four core capabilities: tracking emotional trajectories over time, reasoning about implicit emotional causes, generating empathetic responses, and resolving conflicts between acoustic and textual emotional signals.

## Datasets

- **HumDial-EIBench** — total ?; splits: test (-1); repo https://github.com/ASLP-lab/HumDial-EIBench

## Metrics

- `Accuracy (%)` **(primary)** — range: percent
  - Percentage of correctly selected options in multiple-choice tasks (Tasks 1, 2, and 4). Calculated as (number of correct predictions / total samples) * 100.
- `Empathy/Naturalness Score` — range: [1, 5]
  - Average rating on a 1-5 scale for text empathy (judged by LLMs), acoustic empathy, and response naturalness (judged by humans) in open-ended generation tasks (Task 3).

## Input / output format

**Input**: Multi-turn audio-text dialogues. For Tasks 1, 2, and 4, the model receives context audio/text and multiple-choice options. For Task 3, the model receives conversational prompts and must generate a response.

**Output**: For Tasks 1, 2, and 4: a single selected multiple-choice option. For Task 3: a generated audio/text response.

## Scoring recipe

```python
# Tasks 1, 2, 4 (Multiple Choice)
correct = sum(1 for pred, gold in zip(predictions, golds) if pred == gold)
accuracy = (correct / len(golds)) * 100

# Task 3 (Open-ended)
# LLM judges score text empathy (D1) on a scale (e.g., 1-5)
# Human annotators score acoustic empathy (D2) and naturalness (D3) on a scale (e.g., 1-5)
empathy_score = (llm_judge_score + human_acoustic_score + human_naturalness_score) / 3
```

## Common pitfalls

- Text-dominance bias: Models heavily over-rely on literal text semantics when acoustic and textual signals contradict, causing severe accuracy drops on conflict samples.
- LLM-as-a-judge instability: Automated text empathy scoring shows high variance (up to 1.0 point difference between judges), making it unreliable without human calibration.
- Cross-lingual interference: Models may generate responses in the wrong language (e.g., Chinese to English prompts), artificially deflating empathy scores.

## Evidence (verbatim from paper)

> Table 3: Accuracy (%) on Tasks 1 and 2. Bold: best in column; underline: second best.

## Citation

```bibtex
@misc{wang2026humdialeibench,
  title={HumDial-EIBench: A Human-Recorded Multi-Turn Emotional Intelligence Benchmark for Audio Language Models},
  author={Wang et al. (2026)},
  year={2026},
  note={arXiv:2604.11594}
}
```

- arXiv: 2604.11594