Constitution AI Feedback Eval

constitution-ai-feedback-eval

Evaluating the role of `Constitutions' for learning from AI feedback — Redgate et al. (2024) (arXiv:2411.10168, 2024)

What this evaluates

This evaluation probes how different instructional guidelines (constitutions) shape AI-generated medical dialogues across specific socio-communicative dimensions like empathy, information gathering, and decision-making. It measures human preference for dialogue quality under varying constitutional constraints.

Datasets

• Custom AI-generated medical dialogues — total ?; splits: test (-1)

Metrics

• Bradley-Terry preference rate (primary) — range: [0, 1]
  • Estimated from pairwise human preferences using a Bradley-Terry model. Represents the relative likelihood that a dialogue generated under a specific constitution is preferred over others for a given evaluation dimension.

Input / output format

Input: AI-generated medical dialogue transcripts conditioned on different constitutional guidelines.

Output: Human pairwise preference ratings across six communication dimensions (Fostering Relationship, Gathering Information, Providing Information, Decision Making, Responding to Emotions, Enabling treatment behaviour).

Scoring recipe

# predictions: list of human pairwise choices (winner_constitution, loser_constitution) per dimension
# gold: constitution labels
counts = defaultdict(int)
for winner, loser in predictions:
    counts[winner] += 1
    counts[loser] += 1
# Fit Bradley-Terry model to estimate log-odds preference parameters
bt_params = fit_bradley_terry(counts)
# Convert to preference rate for reporting
return {dim: sigmoid(bt_params[dim]) for dim in dimensions}

Common pitfalls

• Error bars represent 95% confidence intervals not adjusted for multiple comparisons.
• Holistic constitution selection showed no clear pattern despite significant dimension-specific preferences.
• Participants often disliked verbose or overly emotive responses even when rating them as more empathetic.

Evidence (verbatim from paper)

In Figure 2, we show the rate at which the conversations generated according to each constitution are preferred to the others for each dimension of evaluation, alongside the estimated parameters for a Bradley-Terry model.

Citation

@misc{redgate2024constitutions,
  title={Evaluating the role of `Constitutions' for learning from AI feedback},
  author={Redgate et al. (2024)},
  year={2024},
  note={arXiv:2411.10168}
}

• arXiv: 2411.10168