Mirrorbench Eval

mirrorbench-eval

MirrorBench: Evaluating Self-centric Intelligence in MLLMs by Introducing a Mirror — Guo et al. (2026) (arXiv:2604.14785, 2026)

What this evaluates

This benchmark evaluates self-centric intelligence and mirror self-recognition in Multimodal Large Language Models (MLLMs) within an embodied simulation. It probes the model's ability to perform self-referential reasoning and navigate tasks under varying cognitive difficulty levels and body configurations (humanoid vs. robotic).

Datasets

• MirrorBench — total ?; splits: test (-1)

Metrics

• AVG (primary) — range: [0, 1] or normalized score
  • Average of four sub-metrics (TSR, SIR, FCR, PCR) across four cognitive difficulty levels. Scores may be normalized relative to a random policy baseline, as evidenced by negative values in the results table.
• TSR — range: [0, 1]
  • Success/Recognition rate at Level 1 (guided perception).
• SIR — range: [0, 1]
  • Success/Recognition rate at Level 2.
• FCR — range: [0, 1]
  • Success/Recognition rate at Level 3.
• PCR — range: [0, 1]
  • Success/Recognition rate at Level 4 (self-referential recognition).

Input / output format

Input: Per instance, the model receives 1024×1024 visual observations from a simulated environment (Isaac Sim) and a task instruction specifying a target position. The environment uses either an anthropomorphic or robotic body/hand configuration.

Output: The model must output a sequence of actions to navigate the simulated agent. The maximum number of steps is set to the theoretical minimum (Manhattan distance / step size) plus a buffer of 10 steps.

Scoring recipe

def compute_metrics(predictions, gold_target, max_steps):
    success = 0
    for step in predictions:
        if agent_reaches_target(step, gold_target):
            success = 1
            break
    # TSR, SIR, FCR, PCR are success rates at levels 1-4
    TSR = success_rate(level_1)
    SIR = success_rate(level_2)
    FCR = success_rate(level_3)
    PCR = success_rate(level_4)
    AVG = (TSR + SIR + FCR + PCR) / 4
    return AVG

Common pitfalls

• Models often fall into a 'mirror-self confusion' failure mode, persistently targeting the mirrored reflection instead of the physical counterpart.
• Smaller open-source models may show anomalous performance increases across difficulty levels due to unstable inference rather than genuine capability gains.
• The random policy serves as a critical baseline; many open-source models fail to surpass it, indicating a lack of robust self-referential reasoning.

Evidence (verbatim from paper)

MLLMs are ranked according to the average of the 4 metrics and 4 levels. Human agent vastly outperforms all MLLMs, while most open-source models fail to surpass the random policy.

Citation

@misc{guo2026mirrorbench,
  title={MirrorBench: Evaluating Self-centric Intelligence in MLLMs by Introducing a Mirror},
  author={Guo et al. (2026)},
  year={2026},
  note={arXiv:2604.14785}
}

• arXiv: 2604.14785