E3vqa Eval

e3vqa-eval

Towards Comprehensive Scene Understanding: Integrating First and Third-Person Views for LVLMs — Lee et al. (2025) (arXiv:2505.21955, 2025)

What this evaluates

E3VQA evaluates a model's ability to perform multi-view visual question answering using synchronized egocentric and exocentric image pairs. It specifically probes whether models can identify relevant regions across views, filter redundant information, and integrate complementary visual cues to answer multiple-choice questions.

Datasets

• E3VQA — total 4000; splits: test (4000)

Metrics

• accuracy (primary) — range: [0, 1]
  • Standard multiple-choice accuracy: the proportion of questions where the model's selected answer option exactly matches the ground-truth answer.

Input / output format

Input: A synchronized pair of egocentric and exocentric images, accompanied by a multiple-choice question with four candidate options.

Output: A single selected answer option from the provided multiple-choice set.

Scoring recipe

correct = 0
for pred, gold in zip(predictions, gold_answers):
    if pred == gold:
        correct += 1
accuracy = correct / len(gold_answers)

Common pitfalls

• Models may incorrectly rely on only one view (ego or exo) instead of integrating both, which the benchmark's distractors are specifically designed to penalize.
• Questions are explicitly filtered to ensure they cannot be answered without visual input; models that answer based on text priors or memorization will be unfairly penalized if they ignore the images.
• Variations in viewpoint, occlusion, and scaling across the ego-exo pairs make entity alignment difficult, often causing models to fail on spatial and numerical reasoning sub-tasks.

Evidence (verbatim from paper)

M3CoT enables LVLMs to better reason across views by iteratively fusing scene representations, achieving +4.84% (GPT-4o) and +5.94% (Gemini 2.0 Flash) accuracy gains over baseline chain-of-thought methods, with a notable +8.93% boost on numerical reasoning tasks, revealing critical limitations in current LVLMs' spatial and contextual integration capabilities.

Citation

@misc{lee2025e3vqa,
  title={Towards Comprehensive Scene Understanding: Integrating First and Third-Person Views for LVLMs},
  author={Lee et al. (2025)},
  year={2025},
  note={arXiv:2505.21955}
}

• arXiv: 2505.21955