Mtbench Eval

mtbench-eval

Constitution or Collapse? Exploring Constitutional AI with Llama 3-8B — Zhang et al. (2025) (arXiv:2504.04918, 2025)

What this evaluates

Evaluates the model's ability to generate harmless and helpful responses in multi-turn conversations, measuring the trade-off between safety alignment and utility.

Datasets

• MTBench — total ?; splits: test (-1)

Metrics

• MTBench harmlessness score (primary) — range: percent
  • Pairwise comparison of model responses against a reference model (typically GPT-4) across multi-turn prompts, yielding a win rate or average score. The paper reports a 40.8% reduction in harmful responses.
• MTBench helpfulness score — range: percent
  • Same pairwise evaluation protocol as harmlessness, measuring the model's ability to provide useful and accurate assistance. The paper reports a 9.8% drop in helpfulness.

Input / output format

Input: Multi-turn conversation prompts from the MTBench dataset.

Output: Model-generated responses to each turn, evaluated via pairwise comparison against a reference model.

Scoring recipe

def score_mtbench(predictions, gold, judge_model):
    wins = 0
    total = 0
    for prompt, pred in predictions:
        ref_response = get_reference_response(prompt)
        judge_verdict = judge_model.compare(ref_response, pred)
        if judge_verdict == 'pred_wins':
            wins += 1
        total += 1
    return (wins / total) * 100

Common pitfalls

• MTBench scores are relative to a specific reference model (usually GPT-4), so absolute scores are not portable across different baselines.
• Pairwise evaluation can be sensitive to the judge model's prompt, temperature, and system instructions.

Evidence (verbatim from paper)

demonstrating a 40.8% reduction in harmful responses on MTBench via self-critique and DPO-based reinforcement learning. However, helpfulness drops by 9.8%, and clear signs of model collapse emerge due to low-quality self-generated critiques, suggesting that self-improvement—like reasoning—emerges only in sufficiently capable models.

Citation

@misc{zhang2025constitution,
  title={Constitution or Collapse? Exploring Constitutional AI with Llama 3-8B},
  author={Zhang et al. (2025)},
  year={2025},
  note={arXiv:2504.04918}
}

• arXiv: 2504.04918