Castle Eval

castle-eval

CASTLE: A Comprehensive Benchmark for Evaluating Student-Tailored Personalized Safety in Large Language Models — Jia et al. (2026) (arXiv:2602.05633, 2026)

What this evaluates

This benchmark evaluates large language models' ability to detect and mitigate educational safety risks in a student-tailored manner. It probes how well models adapt their responses to individual student profiles across 15 risk domains and 14 psychological and educational attributes.

Datasets

• CASTLE — total 92908; splits: test (-1)

Metrics

• safety score (primary) — range: [0, 5]
  • A 5-point scale rating the safety and appropriateness of the model's response to a student query, considering the 15 educational risk domains and student profile context. Higher scores indicate better risk detection and mitigation.

Input / output format

Input: Student query in Chinese or English, optionally accompanied by a student profile (non-personalized, implicit hints, or explicit attributes).

Output: Natural language response addressing the student query.

Scoring recipe

def compute_safety_score(predictions, queries, profiles):
    scores = []
    for pred, q, p in zip(predictions, queries, profiles):
        # Judge evaluates response safety on 1-5 scale
        # across 15 educational risk domains given profile p
        score = evaluate_safety(pred, q, p)
        scores.append(score)
    return sum(scores) / len(scores)

Common pitfalls

• Evaluating models in non-personalized settings yields consistently low scores (1.7–2.2), which may underestimate their true safety capabilities when context is provided.
• Chinese queries consistently score higher than English ones due to higher information density per token, potentially skewing cross-lingual safety comparisons.
• Assuming that adding student profiles will fully eliminate safety risks; the benchmark shows scores improve but never reach perfection across all dimensions.

Evidence (verbatim from paper)

Table 2 reports the safety score (5-point scale) for each model under non-personalized settings, from which we derive the following key observations. ... Specifically, scores increase from an initial range of 1.8-2.5 to above 3.0 in multiple domains, with the largest improvements observed in psychological well-being and academic integrity.

Citation

@misc{jia2026castle,
  title={CASTLE: A Comprehensive Benchmark for Evaluating Student-Tailored Personalized Safety in Large Language Models},
  author={Jia et al. (2026)},
  year={2026},
  note={arXiv:2602.05633}
}

• arXiv: 2602.05633