# Eicap Bench Eval

> eicap-bench-eval

- Skill: `qhjqhj00/eicap-bench-eval` (Agent Skill)
- Install (CLI): `npx skillmds@latest add qhjqhj00/eicap-bench-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/eicap-bench-eval/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-21
- Page: https://skillmd.com/skills/qhjqhj00/eicap-bench-eval

---


# eicap-bench-eval

> EICAP: Deep Dive in Assessment and Enhancement of Large Language Models in Emotional Intelligence through Multi-Turn Conversations — Nazar et al. (2025) (arXiv:2508.06196, 2025)

## What this evaluates

Evaluates large language models' emotional intelligence (EI) capabilities across a four-layer taxonomy: emotional tracking, cause inference, appraisal, and emotionally appropriate response generation. It probes fine-grained subcategories including cultural sensitivity, valence judgment, and uncertainty calibration using multi-turn conversational contexts.

## Datasets

- **EICap-Bench** — total ?; splits: test (-1)

## Metrics

- `macro-average accuracy` **(primary)** — range: [0, 1]
  - Percentage of correctly answered multiple-choice questions across all EI subcategories, averaged across dimensions. Reported as overall accuracy and delta accuracy (ΔAcc in percentage points) relative to a baseline after fine-tuning.

## Input / output format

**Input**: Multi-turn conversational context or dialogue prompt requiring emotional intelligence reasoning, presented as a multiple-choice question.

**Output**: Model selects the correct option from the provided choices (MCQ format).

## Scoring recipe

```python
def compute_accuracy(predictions, gold):
    correct = sum(1 for p, g in zip(predictions, gold) if p == g)
    return correct / len(gold)
```

## Common pitfalls

- Delta accuracy (ΔAcc) is reported with statistical significance testing (10,000-sample paired bootstrap with Benjamini-Hochberg FDR correction), so raw point differences without confidence intervals or significance markers should not be interpreted as reliable.
- The benchmark evaluates multi-turn dialogues, but the MCQ format abstracts away generation quality; accuracy only measures option selection, not the quality or appropriateness of generated emotional responses.
- Narrow-slice LoRA fine-tuning on general corpora (UltraChat) often degrades foundation-layer EI performance, indicating that domain-aligned, task-annotated data is required for stable improvements.

## Evidence (verbatim from paper)

> Figure[2] summarizes zero-shot performance of three open-source LLM families Gemma-9B, LLaMA-3-8B, and Qwen-2.5-7B in both Base and Instruct variants on EICap-Bench. The evaluation spans fine-grained subcategories across all EI layers defined in the EICap taxonomy. Results show that Qwen-2.5-7B consistently outperforms the other models in macro-average accuracy across multiple EI dimensions, particularly in emotional inference and uncertainty calibration.

## Citation

```bibtex
@misc{nazar2025eicap,
  title={EICAP: Deep Dive in Assessment and Enhancement of Large Language Models in Emotional Intelligence through Multi-Turn Conversations},
  author={Nazar et al. (2025)},
  year={2025},
  note={arXiv:2508.06196}
}
```

- arXiv: 2508.06196