# Groundingme Eval

> groundingme-eval

- Skill: `qhjqhj00/groundingme-eval` (Agent Skill)
- Install (CLI): `npx skillmds@latest add qhjqhj00/groundingme-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/groundingme-eval/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-21
- Page: https://skillmd.com/skills/qhjqhj00/groundingme-eval

---


# groundingme-eval

> GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluation — Li et al. (2025) (arXiv:2512.17495, 2025)

## What this evaluates

Evaluates multimodal large language models' (MLLMs) visual grounding capabilities across four dimensions: discriminative object distinction, spatial relational understanding, handling occlusion/size constraints, and the ability to reject ungroundable queries. It measures how well models can localize objects in images and whether they hallucinate or correctly refuse impossible requests.

## Datasets

- **GroundingME** — total ?; splits: test (-1)

## Metrics

- `Accuracy@0.5` **(primary)** — range: percent
  - The percentage of test samples where the Intersection over Union (IoU) between the predicted bounding box and the ground-truth bounding box exceeds 0.5.

## Input / output format

**Input**: An input image paired with a textual description or query, formatted using a unified prompt template that specifies viewpoint references, allowed output counts, and strict formatting constraints.

**Output**: Bounding box coordinates for the target object(s), strictly following the template's format constraints and output count limits.

## Scoring recipe

```python
def accuracy_at_0_5(predictions, ground_truths):
    correct = 0
    for pred, gt in zip(predictions, ground_truths):
        if calculate_iou(pred, gt) > 0.5:
            correct += 1
    return (correct / len(predictions)) * 100
```

## Common pitfalls

- Models often hallucinate objects for invalid queries instead of correctly rejecting them, severely depressing rejection scores.
- Reporting only the overall average accuracy masks significant performance stratification across subcategories (e.g., high discriminative scores vs. near-zero rejection scores).
- The default evaluation protocol uses greedy decoding (temperature=0), which may not reflect each model's peak performance.

## Evidence (verbatim from paper)

> For the evaluation metric, we adopt the widely-used Accuracy@0.5, which represents the proportion of total samples where the Intersection over Union (IoU) between the ground-truth and predicted bounding box exceeds 0.5.

## Citation

```bibtex
@misc{li2025groundingme,
  title={GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluation},
  author={Li et al. (2025)},
  year={2025},
  note={arXiv:2512.17495}
}
```

- arXiv: 2512.17495