Reflect3r Eval

reflect3r-eval

Reflect3r: Single-View 3D Stereo Reconstruction Aided by Mirror Reflections — Jing Wu et al. (2025) (arXiv:2509.20607, 2025)

What this evaluates

Evaluates single-view 3D stereo reconstruction quality in scenes containing mirror reflections. It probes a model's ability to leverage virtual views generated from mirror reflections to recover accurate 3D geometry and camera poses, outperforming standard monocular or stereo baselines that typically hallucinate depth or collapse in reflective regions.

Datasets

• Synthetic Dataset (Reflect3r) — total ?; splits: test (-1)
• Real-world Mirror Scenes — total 16; splits: test (16)

Metrics

• F1 score (primary) — range: percent
  • Harmonic mean of accuracy and completeness: 2 * (acc * comp) / (acc + comp).
• Accuracy % — range: percent
  • Percentage of predicted points within a 1 cm threshold of their nearest ground-truth neighbor.
• Completeness % — range: percent
  • Percentage of ground-truth points within a 1 cm threshold of their nearest predicted neighbor.
• Chamfer Distance — range: other
  • Average nearest-neighbour distance computed bidirectionally between predicted and ground-truth point sets.
• Translation Error ($T_{err}$) — range: other
  • Euclidean distance between estimated and ground-truth camera translation vectors.
• Rotation Error ($R_{err}$) — range: other
  • Angular difference (in degrees) between estimated and ground-truth camera rotation matrices.

Input / output format

Input: Single RGB image containing a mirror reflection. (For baselines, input may be duplicated or used directly depending on the model architecture).

Output: 3D point cloud (pointmaps assigning 3D coordinates to each pixel) and confidence maps. For pose evaluation: estimated camera translation and rotation parameters.

Scoring recipe

def compute_recon_metrics(pred_pts, gt_pts, threshold=0.01):
    # Accuracy: fraction of pred points within threshold of nearest gt point
    acc = np.mean([np.min(np.linalg.norm(p - gt_pts, axis=1)) < threshold for p in pred_pts])
    # Completeness: fraction of gt points within threshold of nearest pred point
    comp = np.mean([np.min(np.linalg.norm(g - pred_pts, axis=1)) < threshold for g in gt_pts])
    # F1: harmonic mean
    f1 = 2 * (acc * comp) / (acc + comp + 1e-8)
    # Chamfer Distance: average bidirectional nearest neighbor distance
    cd = (np.mean([np.min(np.linalg.norm(p - gt_pts, axis=1)) for p in pred_pts]) +
          np.mean([np.min(np.linalg.norm(g - pred_pts, axis=1)) for g in gt_pts])) / 2
    return {'accuracy': acc, 'completeness': comp, 'f1': f1, 'chamfer': cd}

Common pitfalls

• Quantitative metrics are only reported on the synthetic dataset; real-world scenes only have qualitative visual comparisons due to lack of ground truth.
• The 1 cm threshold for accuracy/completeness is strict and explicitly defined; using a different threshold will invalidate direct comparison with the reported scores.
• Baselines like DUSt3R and MASt3R require input duplication to handle single images, which can artificially skew results compared to native single-view methods like VGGT or MoGe.

Evidence (verbatim from paper)

We evaluate reconstruction quality using 4 metrics: completeness, accuracy, F1 score, and chamfer distance. Accuracy and completeness measure the percentage of reconstruction-to-ground-truth and ground-truth-to-reconstruction distances below a 1 cm threshold, respectively. The F1 score is computed as the harmonic mean of accuracy and completeness. Chamfer Distance measures the similarity between two point sets by computing the average nearest-neighbour distance from each point in one set to the other, ensuring both sets are close in 3D space.

Citation

@misc{wu2025reflect3r,
  title={Reflect3r: Single-View 3D Stereo Reconstruction Aided by Mirror Reflections},
  author={Jing Wu et al. (2025)},
  year={2025},
  note={arXiv:2509.20607}
}

• arXiv: 2509.20607