Clatch Sfm Eval

clatch-sfm-eval

The CUDA LATCH Binary Descriptor: Because Sometimes Faster Means Better — Parker et al. (2016) (arXiv:1609.03986, 2016)

What this evaluates

Evaluates the computational efficiency and 3D reconstruction accuracy of a GPU-accelerated binary feature descriptor (CLATCH) compared to traditional and deep learning-based descriptors within a Structure-from-Motion pipeline.

Datasets

• Photogrammetry Image Sets (8 scenes) — total ?; splits: test (8)

Metrics

• SfM Scene RMSE (pixels) (primary) — range: other
  • Root Mean Square Error of reprojection errors across all matched 3D points in the reconstructed scene, measured in pixels.
• Descriptor Extraction Time (μs) — range: other
  • Mean time in microseconds required to extract a single local feature descriptor.
• Total SfM Pipeline Time (s) — range: other
  • Total time in seconds for descriptor extraction, matching, and incremental SfM reconstruction.

Input / output format

Input: High-resolution photogrammetry image sets (5,616×3,744 or 3,744×5,616 pixels) for each scene.

Output: 3D point cloud and mesh reconstructions, along with per-scene reprojection RMSE and pipeline timing statistics.

Scoring recipe

def compute_sfm_rmse(reconstructed_3d_points, image_2d_points, camera_params):
    errors = []
    for pt3d, pt2d in zip(reconstructed_3d_points, image_2d_points):
        projected_2d = project_3d_to_2d(pt3d, camera_params)
        errors.append(np.linalg.norm(projected_2d - pt2d))
    return np.sqrt(np.mean(np.square(errors)))

Common pitfalls

• Reprojection RMSE values are very small (~0.5 pixels) and inter-method differences are often <0.1 pixels, making statistical significance difficult to assess on only 8 scenes.
• Pipeline timing is dominated by brute-force nearest neighbor matching, so descriptor extraction speed gains do not linearly translate to total pipeline speedups.
• SIFT uses a different feature detector (DOG) than the other methods (CUDA FAST), which may confound direct speed and accuracy comparisons.

Evidence (verbatim from paper)

Table 2 summarizes these results, providing the final scene reprojection RMSE and the total time for descriptor extraction, matching and SfM reconstruction. All these tests were run on our GTX 1080 GPU.

Citation

@misc{parker2016cudalatch,
  title={The CUDA LATCH Binary Descriptor: Because Sometimes Faster Means Better},
  author={Parker et al. (2016)},
  year={2016},
  note={arXiv:1609.03986}
}

• arXiv: 1609.03986