# Robo Refer Spatial Eval

> Evaluates vision-language models' ability to perform single-step and multi-step spatial understanding and referring tasks in robotics contexts. It probes capabilities like 2D/3D relation reasoning, depth perception, and complex compositional spatial constraints in cluttered scenes. Use when the user wants to benchmark on CV-Bench, BLINK, RoboSpatial, RefSpatial-Bench, RefCOCO, or asks about evaluating this task. Reports Top-1 accuracy.

- Skill: `qhjqhj00/robo-refer-spatial-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/robo-refer-spatial-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/robo-refer-spatial-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/robo-refer-spatial-eval

---


# robo-refer-spatial-eval

> RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Robotics — Zhou et al. (2025) (arXiv:2506.04308, 2025)

## What this evaluates

Evaluates vision-language models' ability to perform single-step and multi-step spatial understanding and referring tasks in robotics contexts. It probes capabilities like 2D/3D relation reasoning, depth perception, and complex compositional spatial constraints in cluttered scenes.

## Datasets

- **CV-Bench** — total ?; splits: (unstated)
- **BLINK** — total ?; splits: val (-1)
- **RoboSpatial** — total ?; splits: (unstated)
- **RefSpatial-Bench** — total 200; splits: Location (100), Placement (100)
- **RefCOCO** — total ?; splits: val (-1), testA (-1), testB (-1)

## Metrics

- `Top-1 accuracy` **(primary)** — range: [0, 1]
  - Calculated as the proportion of correctly predicted instances out of the total. For referring tasks, a prediction is correct if the predicted point lies within the ground-truth mask or bounding box.
- `Success rate` — range: percent
  - Average success rate of predicted points within the ground-truth mask or bounding box, reported as a percentage.

## Input / output format

**Input**: RGB or RGB-D image paired with a natural language spatial query or instruction.

**Output**: Predicted 2D point coordinates (x, y) or bounding box, or a multiple-choice label.

## Scoring recipe

```python
def compute_accuracy(predictions, golds):
    correct = 0
    for pred, gold in zip(predictions, golds):
        if isinstance(gold, dict) and 'mask' in gold:
            if point_in_mask(pred, gold['mask']): correct += 1
        elif isinstance(gold, dict) and 'bbox' in gold:
            if point_in_bbox(pred, gold['bbox']): correct += 1
        else:
            if pred == gold: correct += 1
    return correct / len(predictions)
```

## Common pitfalls

- Evaluating point predictions against bounding boxes using IoU instead of point-in-box containment.
- Ignoring the impact of RGB vs. RGB-D inputs, which drastically changes 3D spatial benchmark performance.
- Assuming single-step accuracy generalizes to multi-step reasoning, which requires sequential refinement.

## Evidence (verbatim from paper)

> For metrics, we report the average success rate of predicted points within the mask. ... Since our model predicts a single point, we deem a prediction correct if the point lies within the ground-truth bounding box.

## Citation

```bibtex
@misc{zhou2025roborefer,
  title={RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Robotics},
  author={Zhou et al. (2025)},
  year={2025},
  note={arXiv:2506.04308}
}
```

- arXiv: 2506.04308