qhjqhj00 remindviewbench-eval
Reasoning Path and Latent State Analysis for Multi-view Visual Spatial Reasoning: A Cognitive Science Perspective — Xue et al. (2025) (arXiv:2512.02340, 2025)
What this evaluates
Evaluates vision-language models' ability to perform multi-view spatial reasoning, including relative direction, relative distance, cross-view consistency, and perspective-taking. It probes whether models can maintain geometric coherence and integrate information across multiple camera viewpoints to answer questions about procedurally generated indoor scenes.
Datasets
- ReMindView-Bench — total 50000; splits: test (-1); repo https://github.com/pittisl/ReMindView-Bench
Metrics
accuracy(primary) — range: [0, 1]- Standard multiple-choice accuracy: the fraction of questions where the model's predicted option exactly matches the ground-truth answer derived from scene metadata.
Input / output format
Input: One or more rendered images of an indoor scene (single-view or multi-view) paired with a natural language question about spatial relationships, distances, or object locations.
Output: A single selected option from a predefined set of candidate answers (multiple-choice format).
Scoring recipe
def compute_accuracy(predictions, gold_labels):
correct = sum(1 for pred, gold in zip(predictions, gold_labels) if pred == gold)
return correct / len(gold_labels)
Common pitfalls
- Models often succeed on single-view questions but degrade sharply when cross-view integration is required.
- Perspective-taking queries require shifting the reasoning frame to an object-centered viewpoint, which many VLMs fail to maintain consistently.
- Geometric metadata is used for ground truth, but models rely on visual cues that may be ambiguous or occluded across views.
Evidence (verbatim from paper)
ReMindView-Bench consists of $>50,000$ multi-choice VQA pairs derived from 100 procedurally generated indoor scenes, and is constructed through a fully automated and physically grounded pipeline. ... For each query, candidate answers are drawn from the visible objects in the scene, and the correct answer is computed from the metadata.
Citation
@misc{xue2025remindviewbench,
title={Reasoning Path and Latent State Analysis for Multi-view Visual Spatial Reasoning: A Cognitive Science Perspective},
author={Xue et al. (2025)},
year={2025},
note={arXiv:2512.02340}
}
- arXiv: 2512.02340