Max Reprojection Difference

max-reprojection-difference

Reference Pose Generation for Long-term Visual Localization via Learned Features and View Synthesis — Zhang et al. (2020) (arXiv:2005.05179, 2020)

What this evaluates

Evaluates camera pose estimation accuracy in long-term visual localization by measuring the maximum pixel displacement of projected 3D points between a reference and an estimated pose. This indirect measure avoids the non-trivial task of quantifying 6-DoF pose uncertainties while remaining sensitive to camera-to-scene distance variations.

Datasets

• Aachen DayNight — total ?; splits: test (-1); repo https://github.com/tsattler/visuallocalizationbenchmark

Metrics

• Maximum reprojection difference (primary) — range: other
  • Computes the maximum L2 distance in pixels between the 2D projections of a set of 3D points under the reference pose and the estimated pose: $r_i^\infty = \max_{l} |\pi(\mathbf{p}_l^r, \mathbf{T}_i^r) - \pi(\mathbf{p}_l^r, \hat{\mathbf{T}}_i)|_2$. Overall accuracy is reported as the percentage of images where this value falls below predefined pixel thresholds.

Input / output format

Input: Reference camera pose, estimated camera pose, set of 3D scene points, and camera intrinsics/distortion parameters.

Output: Scalar maximum reprojection error in pixels per image, or a boolean/percentage indicating whether the error falls below a specified threshold.

Scoring recipe

def compute_max_reprojection_error(reference_pose, estimated_pose, points_3d, intrinsics):
    proj_ref = project(points_3d, reference_pose, intrinsics)
    proj_est = project(points_3d, estimated_pose, intrinsics)
    return np.max(np.linalg.norm(proj_ref - proj_est, axis=1))

def compute_accuracy(errors, thresholds):
    return [np.mean(errors < t) * 100 for t in thresholds]

Common pitfalls

• Fixed error thresholds applied uniformly across a dataset ignore per-image uncertainty variations caused by landmark distance and feature matching quality.
• Direct 6-DoF pose error metrics are highly sensitive to inaccuracies in the reference pose itself, making them unreliable when reference uncertainty is comparable to evaluation error.
• Reprojection thresholds in pixels implicitly correspond to different pose-space uncertainties depending on the camera-to-scene distance, which can mislead performance comparisons if not accounted for.

Evidence (verbatim from paper)

To avoid the need to consider the uncertainties in 6 DoF poses (which is non-trivial as seen before), we follow the literature on object pose estimation and measure pose accuracy based on reprojections [147]. More precisely, we measure the difference between the reprojection of a set of 3D points in the reference and estimated poses. Intuitively, perturbations to the camera pose will result in the changes of the projected 2D locations of 3D points. Therefore, we can define certain thresholds around the reprojection of the 3D points as an indirect measure of the pose uncertainty. A key advantage of this approach is that the error thresholds can be defined on the image plane. While we use the same thresholds for all the images, this actually results in per-image uncertainty thresholds in pose space: the same change in reprojection error will typically result in a position error that increases with increasing distance of the camera to the scene. Formally, we define the following metric: Maximum reprojection difference. The maximum distance between the projected points in the reference pose $\mathbf{T}_i^r$ and the estimated pose $\hat{\mathbf{T}}_i$ is used to measure the localiza

Citation

@misc{zhang2020referencepose,
  title={Reference Pose Generation for Long-term Visual Localization via Learned Features and View Synthesis},
  author={Zhang et al. (2020)},
  year={2020},
  note={arXiv:2005.05179}
}

• arXiv: 2005.05179