# visor

> Evaluates text-to-image models on spatial relationship accuracy using the VISOR metric, separating object detection from spatial correctness to reveal biases like object priority and merging.

- Skill: `qhjqhj00/visor` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/visor`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/visor/raw
- Safety review: PASS (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML, Model Training & Fine-tuning
- Tags: Benchmark, Evaluation, Object Detection, Spatial Relationships, Text To Image, Visor
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-08-22
- Page: https://skillmd.com/skills/qhjqhj00/visor

---


# visor

> Benchmarking Spatial Relationships in Text-to-Image Generation — Gokhale et al. (2022) (arXiv:2212.10015, 2022)

## What this evaluates

Evaluates whether text-to-image models correctly render spatial relationships between objects mentioned in a prompt. It disentangles object generation accuracy from spatial correctness to reveal model biases like object priority and merging.

## Datasets

- **SR2D** — total ?; splits: (unstated); repo https://github.com/microsoft/VISOR

## Metrics

- `VISOR` **(primary)** — range: [0, 1]
  - Measures spatial relationship accuracy. VISORuncond scores all prompts. VISORcond scores only prompts where both objects are correctly detected. VISORn variants score specific spatial relations.
- `OA` — range: [0, 1]
  - Object Accuracy: percentage of prompts where both mentioned objects are correctly detected in the generated image.
- `Δ_s` — range: other
  - Average difference between metric score for original prompt t and flipped prompt t_flip: E_t[s^t - s^t_flip]. Measures sensitivity to spatial relationships.

## Input / output format

**Input**: Text prompt t specifying two objects and their spatial relationship.

**Output**: Generated image x from the text-to-image model.

## Scoring recipe

```python
def compute_metrics(prompt, image):
    obj_a, obj_b, rel = parse_prompt(prompt)
    detected = detect_objects(image)
    a_present = obj_a in detected
    b_present = obj_b in detected
    rel_detected = detect_spatial_relation(image)
    
    oa = 1.0 if (a_present and b_present) else 0.0
    visor_cond = 1.0 if (a_present and b_present and rel_detected == rel) else 0.0
    visor_uncond = 1.0 if (rel_detected == rel) else 0.0
    return oa, visor_cond, visor_uncond
```

## Common pitfalls

- Existing metrics like CLIPScore and FID ignore text or spatial relations, yielding Δ_s ≈ 0.
- Object Accuracy (OA) penalizes spatial failures if one object is missing, conflating generation and spatial reasoning.
- Human studies show high inter-annotator agreement but reveal models often merge objects (32.46% for DALLE-v2).

## Evidence (verbatim from paper)

> We generate N=4 images for each text prompt from our SR2D dataset, to obtain 126,720 images per model and compare performance in terms of OA, VISOR, VISORcond, and VISOR1/2/3/4.

## Citation

```bibtex
@misc{gokhale2022visor,
  title={Benchmarking Spatial Relationships in Text-to-Image Generation},
  author={Gokhale et al. (2022)},
  year={2022},
  note={arXiv:2212.10015}
}
```

- arXiv: 2212.10015