Mental Health Chatbot Eval

mental-health-chatbot-eval

Domain-Specific Constitutional AI: Enhancing Safety in LLM-Powered Mental Health Chatbots — Lyu et al. (2025) (arXiv:2509.16444, 2025)

What this evaluates

Evaluates the safety, clinical adherence, and crisis response quality of LLM-powered mental health chatbots against expert-defined guidelines. It probes the model's ability to provide evidence-based advice, identify health risks, maintain consistent crisis intervention, provide appropriate resources, and empower users.

Datasets

• Institute for Future Health Mental Health Query Set — total 100; splits: test (100)

Metrics

• TotalScore (primary) — range: [0, 50]
  • Sum of scores across five clinical guidelines. Each guideline is scored on a 1-10 scale by trained evaluators using detailed clinical rubrics. The maximum possible score is 50 per response.

Input / output format

Input: Mental health-related queries covering scenarios such as depression, anxiety, crises, and general mental wellness.

Output: Natural language chatbot responses to the queries.

Scoring recipe

def compute_total_score(predictions, guidelines):
    total = 0
    for pred, guideline in zip(predictions, guidelines):
        score = evaluator_rubric_score(pred, guideline)  # 1-10 scale
        total += score
    return total  # Max 50

Common pitfalls

• Evaluators must be trained and strictly follow the provided clinical rubrics; untrained scoring or automated LLM-as-judge methods often fail to capture clinical nuance and deviate from the 1-10 scale.
• The aggregate TotalScore can mask severe deficiencies in critical guidelines (e.g., Guideline 4 on crisis resource provision), so per-guideline breakdowns must be examined alongside the total.

Evidence (verbatim from paper)

We use the evaluation metric developed by the Institute for Future Health*[[13]]*, which includes five guideline questions with ground truth for mental health chatbot evaluation: Each model was evaluated based on 100 mental health-related queries about common scenarios like depression, anxiety, crises, and general mental wellness. Health experts provided ideal responses as ground truth. Trained evaluators scored responses on a 1-10 scale per guideline using detailed rubrics aligned with clinical best practices. Total scores represented the sum across all five guidelines, with a maximum possible score of 50 per response.

Citation

@misc{lyu2025domain,
  title={Domain-Specific Constitutional AI: Enhancing Safety in LLM-Powered Mental Health Chatbots},
  author={Lyu et al. (2025)},
  year={2025},
  note={arXiv:2509.16444}
}

• arXiv: 2509.16444