# Ssrbench Eval

> Evaluates vision-language models on spatial understanding and general question answering using image-text pairs. It probes capabilities such as object existence, attribute recognition, action identification, counting, positional reasoning, and object identification, specifically testing how well models leverage depth information and spatial reasoning. Use when the user wants to benchmark on SSRBench, or asks about evaluating this task. Reports accuracy.

- Skill: `qhjqhj00/ssrbench-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/ssrbench-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/ssrbench-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/ssrbench-eval

---


# ssrbench-eval

> SSR: Enhancing Depth Perception in Vision-Language Models via Rationale-Guided Spatial Reasoning — Yang Liu et al. (2025) (arXiv:2505.12448, 2025)

## What this evaluates

Evaluates vision-language models on spatial understanding and general question answering using image-text pairs. It probes capabilities such as object existence, attribute recognition, action identification, counting, positional reasoning, and object identification, specifically testing how well models leverage depth information and spatial reasoning.

## Datasets

- **SSRBench** — total ?; splits: test (-1)

## Metrics

- `accuracy` **(primary)** — range: percent
  - Percentage of correctly answered questions. Evaluated using an LLM-assistant (Qwen2.5-14B-Instruct-1M) that compares model-generated answers against ground-truth answers to determine correctness.

## Input / output format

**Input**: Image, corresponding depth map (estimated via Depth Pro), and a natural language question.

**Output**: Natural language answer string.

## Scoring recipe

```python
def compute_accuracy(predictions, gold_answers):
    correct = 0
    for pred, gold in zip(predictions, gold_answers):
        prompt = f'Question: {gold["question"]}\nModel Answer: {pred}\nGround Truth: {gold["answer"]}\nIs the model answer correct? Answer Yes or No.'
        judge_response = llm_assistant.generate(prompt)
        if 'yes' in judge_response.lower():
            correct += 1
    return (correct / len(predictions)) * 100
```

## Common pitfalls

- Uses an LLM-as-a-judge (Qwen2.5-14B) for scoring instead of exact-match or regex-based evaluation, which can introduce scorer bias or inconsistency.
- Depth maps are not provided as ground truth but are estimated externally using Depth Pro, meaning evaluation performance depends on the depth estimator's accuracy.
- Task-specific metrics (e.g., Position, Existence, Counting) are reported in tables, but the main headline results focus on average accuracy across all tasks.

## Evidence (verbatim from paper)

> Performance metrics include accuracy as well as a quantitative score ranging from 0 to 5, both are produced using the LLM-Assistant powered by the Qwen2.5-14B-Instruct-1M *[[38], [93]]*. SSRBench consists of two primary categories, general understanding and spatial understanding, allowing simultaneous evaluation of VLM performance in both general question answering and spatial reasoning tasks.

## Citation

```bibtex
@misc{liu2025ssr,
  title={SSR: Enhancing Depth Perception in Vision-Language Models via Rationale-Guided Spatial Reasoning},
  author={Yang Liu et al. (2025)},
  year={2025},
  note={arXiv:2505.12448}
}
```

- arXiv: 2505.12448