Dragon Eval

dragon-eval

DRAGON: A Benchmark for Evidence-Grounded Visual Reasoning over Diagrams — Iyengar et al. (2026) (arXiv:2604.25231, 2026)

What this evaluates

Evaluates evidence-grounded visual reasoning in diagrams by requiring models to localize bounding boxes of supporting visual elements (e.g., labels, axes, connectors) rather than just predicting the correct answer. It probes a model's ability to visually ground reasoning steps within complex diagrammatic contexts and assesses how well localization quality correlates with reasoning performance.

Datasets

• DRAGON — total 11664; splits: ChartQA (-1), Circuit-VQA (-1), InfoVQA (-1), MapIQ (-1), MapWise (-1), AI2D (-1)

Metrics

• Max Pairwise IoU (MPIoU) (primary) — range: [0, 1]
  • Computes the maximum Intersection over Union between each predicted bounding box and any ground-truth box, averaged across instances. Values range from 0 to 1.
• Grounding IoU (GIoU) — range: [0, 1]
  • Measures the IoU between the predicted box and the closest ground-truth box, penalizing non-overlapping regions. Values range from 0 to 1.
• F1 — range: [0, 1]
  • Box-level F1 score computed from precision and recall of predicted bounding boxes against ground-truth evidence boxes at a fixed IoU threshold.

Input / output format

Input: Diagram image paired with a question/prompt requiring visual reasoning and evidence localization.

Output: Bounding box coordinates (e.g., [x_min, y_min, x_max, y_max]) for visual elements that support the answer.

Scoring recipe

def compute_metrics(pred_boxes, gt_boxes, iou_thresh=0.5):
    ious = compute_iou_matrix(pred_boxes, gt_boxes)
    mpiou = np.mean([max(ious[i]) if ious[i].size > 0 else 0 for i in range(len(pred_boxes))])
    gious = [max(ious[i]) for i in range(len(pred_boxes))]
    tp = sum(1 for i in range(len(pred_boxes)) if max(ious[i]) >= iou_thresh)
    fp = len(pred_boxes) - tp
    fn = len(gt_boxes) - tp
    prec = tp / (tp + fp) if (tp + fp) > 0 else 0
    rec = tp / (tp + fn) if (tp + fn) > 0 else 0
    f1 = 2 * prec * rec / (prec + rec) if (prec + rec) > 0 else 0
    return mpiou, np.mean(gious), f1

Common pitfalls

• Models may achieve high answer accuracy but fail to localize the correct visual evidence, leading to misleading performance if only answer accuracy is reported.
• Performance is highly sensitive to prompting strategies (EDGE, SAGE, VERGE), so evaluation must specify the exact prompt template used.
• IoU thresholds for precision/recall/F1 must be explicitly stated, as hit rates vary significantly with threshold choice.

Evidence (verbatim from paper)

We evaluate model outputs using complementary metrics that capture both localization quality and evidence coverage. Specifically, we report Max Pairwise IoU Lin et al. ([2014]), Grounding IoU, threshold hit rates derived from these measures, and box-level precision, recall, and F1.

Citation

@misc{iyengar2026dragon,
  title={DRAGON: A Benchmark for Evidence-Grounded Visual Reasoning over Diagrams},
  author={Iyengar et al. (2026)},
  year={2026},
  note={arXiv:2604.25231}
}

• arXiv: 2604.25231