Venusbench Gd Eval

venusbench-gd-eval

VenusBench-GD: A Comprehensive Multi-Platform GUI Benchmark for Diverse Grounding Tasks — Zhou et al. (2025) (arXiv:2512.16501, 2025)

What this evaluates

This benchmark evaluates GUI grounding capabilities across a hierarchical taxonomy of basic (element, visual, spatial) and advanced (functional, reasoning, refusal) tasks. It probes a model's ability to accurately locate UI elements in screenshots and handle complex, domain-specific, or unanswerable instructions across web, mobile, and desktop platforms.

Datasets

• VenusBench-GD — total 6166; splits: test (-1)

Metrics

• accuracy (primary) — range: percent
  • A prediction is correct if the predicted point lies within the ground-truth bounding box. For refusal tasks, correctness requires strict adherence to a prescribed refusal format (e.g., coordinates [-1,-1] or a specific rejection phrase). The final metric is the accuracy averaged over all UI elements in the benchmark.

Input / output format

Input: A GUI screenshot image paired with a natural language instruction or query.

Output: Predicted coordinates (e.g., [x, y]) or a designated refusal phrase/format.

Scoring recipe

correct = 0
total = 0
for inst in dataset:
    pred = model.predict(inst.image, inst.instruction)
    if inst.task_type == 'refusal':
        if pred == inst.refusal_format:
            correct += 1
    else:
        if point_in_box(pred, inst.gt_box):
            correct += 1
    total += 1
return (correct / total) * 100

Common pitfalls

• Refusal tasks require strict format adherence rather than coordinate proximity, which many models fail to follow.
• Basic grounding tasks are near-saturation for modern VLMs, so evaluating only element/visual/spatial accuracy will not differentiate model capabilities.
• Advanced tasks (functional/reasoning) require domain-specific knowledge and robustness to out-of-distribution queries, which specialized models often lack due to overfitting.

Evidence (verbatim from paper)

Consistent with previous works, we define a prediction as correct when the predicted point lies within the ground-truth bounding box and report the accuracy averaged over all UI elements. For refusal grounding tasks, we consider a model’s output correct only when it adheres strictly to the prescribed refusal format, such as returning coordinates $[-1,-1]$ or a designated rejection phrase as specified in the prompt.

Citation

@misc{zhou2025venusbenchgd,
  title={VenusBench-GD: A Comprehensive Multi-Platform GUI Benchmark for Diverse Grounding Tasks},
  author={Zhou et al. (2025)},
  year={2025},
  note={arXiv:2512.16501}
}

• arXiv: 2512.16501