Express Bench Eval

express-bench-eval

Beyond the Destination: A Novel Benchmark for Exploration-Aware Embodied Question Answering — Jiang et al. (2025) (arXiv:2503.11117, 2025)

What this evaluates

Evaluates an agent's ability to actively explore 3D environments to gather visual evidence and answer questions accurately, while measuring exploration efficiency and navigation performance. It specifically probes whether the agent's final answer is grounded in the actual visual observations collected during its exploration path, detecting hallucinations and ungrounded reasoning.

Datasets

• EXPRESS-Bench — total 2044; splits: test (-1); repo https://github.com/HCPLab-SYSU/EXPRESS-Bench

Metrics

• C (primary) — range: [0, 100]
  • Exploration-Answer Consistency (EAC) measures answer reliability by aligning the final response with the actual visual observations collected during the agent's exploration path. The exact formula is defined in Section 3.4 of the paper.
• C* — range: [0, 100]
  • C without answer grounding (i.e., setting δ_i=1). It measures raw answer accuracy regardless of whether the agent actually observed the relevant visual evidence.
• E_path — range: other
  • Exploration efficiency metric. Higher values indicate more efficient information gathering during navigation. Formula provided in Section 3.4.
• d_T — range: other
  • Navigation performance metric measuring the distance from the agent's final position to the target region. Lower values indicate better navigation.

Input / output format

Input: Question (Q), visual observations/frames collected during exploration, and optionally ground-truth navigation paths or exploration strategies (S/G) for baseline simulation.

Output: Final answer (A) and navigation/exploration trajectory.

Scoring recipe

# C (Exploration-Answer Consistency)
# Aligns answer with visual observations collected during exploration
C = (1/N) * sum(grounding_score(answer_i, observations_i)) * 100

# C* is C without answer grounding constraint (delta_i = 1)
C_star = (1/N) * sum(accuracy_score(answer_i, gold_i)) * 100

# E_path: Exploration efficiency (higher is better)
# d_T: Navigation distance to target region (lower is better)
# Note: Exact formulas for C, E_path, d_T are defined in Sec 3.4 / Supplementary

Common pitfalls

• Confusing C (grounded consistency) with C* (ungrounded accuracy); C* can be significantly inflated by model hallucinations if the agent does not actually observe the answer-relevant regions.
• Assuming passive observation baselines (Socratic/Multi-Frame) are sufficient; the benchmark requires active exploration, and passive models perform poorly on C despite decent C* scores.
• Ignoring the supplementary material or Section 3.4 for the exact mathematical definitions of C, E_path, d_T, NPL, ACE, and WCE, as they are not fully detailed in the main text.

Evidence (verbatim from paper)

Besides the metrics in Section [3.4], we also compute C*, which is C without answer grounding (i.e., setting δ_i=1). Since models like GPT-4 cannot perform exploration, we only assess their performance using C*, as shown in Tab.[2].

Citation

@misc{jiang2025expressbench,
  title={Beyond the Destination: A Novel Benchmark for Exploration-Aware Embodied Question Answering},
  author={Jiang et al. (2025)},
  year={2025},
  note={arXiv:2503.11117}
}

• arXiv: 2503.11117