# Klue Dst Eval

> Evaluates a model's ability to track and update dialogue state across turns in Korean conversations, testing multi-turn reasoning and slot filling. Use when the user wants to benchmark on KLUE-DST, or asks about evaluating this task. Reports Joint Accuracy.

- Skill: `qhjqhj00/klue-dst-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/klue-dst-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/klue-dst-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/klue-dst-eval

---


# klue-dst-eval

> KLUE: Korean Language Understanding Evaluation — Sungjoon Park et al. (arXiv:2105.09680, 2021)

## What this evaluates

Evaluates a model's ability to track and update dialogue state across turns in Korean conversations, testing multi-turn reasoning and slot filling.

## Datasets

- **KLUE-DST** — total ?; splits: train (-1), val (-1), test (-1); repo https://github.com/KLUE-benchmark/KLUE

## Metrics

- `Joint Accuracy` **(primary)** — range: [0, 1]
  - Proportion of dialogue turns where all slot values are predicted exactly correctly.

## Input / output format

**Input**: Dialogue history and current user utterance in Korean.

**Output**: Dictionary of slot-value pairs representing the current dialogue state.

## Scoring recipe

```python
def compute_joint_accuracy(pred_states, gold_states):
    correct = sum(1 for p, g in zip(pred_states, gold_states) if p == g)
    return correct / len(gold_states)
```

## Common pitfalls

- Slot value normalization (e.g., numbers, dates) must match gold exactly.
- Error propagation across turns can unfairly penalize early mistakes.

## Evidence (verbatim from paper)

> KLUE introduces a comprehensive, ethically designed benchmark for Korean NLU with 8 tasks (Topic Classification, STS, NLI, NER, RE, DP, MRC, DST) built from scratch using diverse, copyright-respected corpora.

## Citation

```bibtex
@misc{park2021klue,
  title={KLUE: Korean Language Understanding Evaluation},
  author={Sungjoon Park et al.},
  year={2021},
  note={arXiv:2105.09680}
}
```

- arXiv: 2105.09680