Clinical Field Recovery Eval

clinical-field-recovery-eval

Optimal Question Selection from a Large Question Bank for Clinical Field Recovery in Conversational Psychiatric Intake — Panda et al. (2026) (arXiv:2604.22067, 2026)

What this evaluates

Evaluates sequential question-selection strategies for recovering target clinical fields from synthetic patient responses under a fixed interaction budget. It probes how well adaptive versus fixed questioning policies handle varying patient communication styles to maximize information coverage within conversational constraints.

Datasets

• Clinical Psychiatric Intake Vignette Benchmark — total ?; splits: test (300)

Metrics

• accuracy (primary) — range: percent
  • Percentage of target clinical fields correctly recovered by the end of the interview session (max 20 turns or when all fields are recovered).
• precision — range: percent
  • Percentage of recovered fields that are correctly labeled by the judge, measuring post-disclosure interpretation accuracy.
• conversational_cost — range: other
  • Cumulative word count across the transcript, used to assess efficiency relative to accuracy.

Input / output format

Input: Current conversation transcript, in-loop recovery state, patient behavioral condition profile, and a question bank.

Output: A single question selected from the bank to ask the patient next.

Scoring recipe

def compute_metrics(recovered_fields, gold_fields, transcript):
    total_target = len(gold_fields)
    correct = sum(1 for f, v in recovered_fields.items() if f in gold_fields and v == gold_fields[f])
    accuracy = (correct / total_target) * 100 if total_target > 0 else 0
    precision = (correct / len(recovered_fields)) * 100 if len(recovered_fields) > 0 else 0
    word_count = len(transcript.split())
    return {'accuracy': accuracy, 'precision': precision, 'conversational_cost': word_count}

Common pitfalls

• Assuming higher transcript length or word count directly correlates with better performance; the paper shows the best strategy recovers more information per unit of conversational effort, not just longer transcripts.
• Overestimating the role of post-disclosure interpretation; precision is uniformly high (~98-99%), indicating the bottleneck is topic coverage and question selection, not field labeling.
• Ignoring behavioral condition heterogeneity; performance drops sharply for non-adaptive strategies under 'Guarded + Concise' conditions, masking overall strategy efficacy if averaged without stratification.

Evidence (verbatim from paper)

Random reaches 51.7%±19.6% mean accuracy, while form-based reaches 84.8%±15.8% and LLM-guided reaches 95.4%±8.1%. Precision remains uniformly high across all three strategies, ranging from 98.4% for random to 99.4% for LLM-guided.

Citation

@misc{panda2026optimalquestionselection,
  title={Optimal Question Selection from a Large Question Bank for Clinical Field Recovery in Conversational Psychiatric Intake},
  author={Panda et al. (2026)},
  year={2026},
  note={arXiv:2604.22067}
}

• arXiv: 2604.22067