Condmedqa Eval

condmedqa-eval

Condition-Gated Reasoning for Context-Dependent Biomedical Question Answering — Parekh et al. (2026) (arXiv:2602.17911, 2026)

What this evaluates

Evaluates a model's ability to perform conditional multi-hop reasoning in biomedical question answering, specifically how well it modulates clinical answers based on patient-specific constraints like comorbidities, contraindications, and special population factors.

Datasets

• CondMedQA — total 100; splits: test (100)

Metrics

• performance (primary) — range: [0, 1]
  • Standard exact-match accuracy for biomedical QA, calculated as the proportion of questions where the model's predicted answer exactly matches the gold answer.

Input / output format

Input: A clinical question containing a specific patient condition/modifier (e.g., pregnancy, comorbidity, drug interaction), often requiring synthesis of information from two provided knowledge sources.

Output: The correct clinical answer (e.g., drug name, dosage, diagnostic modality) that applies specifically given the stated patient condition.

Scoring recipe

correct = 0
for pred, gold in zip(predictions, gold_answers):
    if normalize_text(pred) == normalize_text(gold):
        correct += 1
return correct / len(predictions)

Common pitfalls

• Models may memorize default/general answers and fail to adjust when patient-specific modifiers are present.
• Failing to synthesize information across two separate knowledge sources required to form the conditional reasoning trace.
• Overlooking subtle causal dependencies where removing the modifier reverts the answer to the general case.

Evidence (verbatim from paper)

CGR outperforms state-of-the-art methods on condition-sensitive queries while matching or exceeding performance on factual benchmarks, demonstrating that explicit modeling of conditionality is critical for robust clinical reasoning.

Citation

@misc{parekh2026condmedqa,
  title={Condition-Gated Reasoning for Context-Dependent Biomedical Question Answering},
  author={Parekh et al. (2026)},
  year={2026},
  note={arXiv:2602.17911}
}

• arXiv: 2602.17911