Scannerf Eval

scannerf-eval

ScanNeRF: a Scalable Benchmark for Neural Radiance Fields — De Luigi et al. (2022) (arXiv:2211.13762, 2022)

What this evaluates

Evaluates the rendering quality and novel-view synthesis capability of Neural Radiance Field (NeRF) methods on real-world inward-facing object scans. It probes how well models generalize to unseen camera poses when trained with varying image densities and localized acquisition patterns.

Datasets

• ScanNeRF — total 35; splits: train (-1), test (-1)

Metrics

• PSNR (primary) — range: dB
  • Peak Signal-to-Noise Ratio between rendered images x_hat and ground truth test images x. Computed as -10 log10((x - x_hat)^2) per the paper (standard MSE-based PSNR). Results are averaged over 35 scanned objects.

Input / output format

Input: RGB images with known camera poses for training and testing inward-facing object scans.

Output: Rendered RGB images at the test camera viewpoints.

Scoring recipe

def compute_psnr(rendered, ground_truth):
    mse = np.mean((rendered - ground_truth) ** 2)
    if mse == 0:
        return float('inf')
    return 10 * np.log10(255.0**2 / mse)
# Average PSNR across all test images and 35 objects

Common pitfalls

• DVGO experiments were run at half resolution due to memory constraints, making direct quantitative comparison with full-resolution methods unfair.
• Instant-NGP training steps were reduced from 100K to 10K to save time, which may affect convergence compared to official defaults.
• Densely localized splits stress generalization to unseen hemispherical regions, but performance heavily depends on the specific train/test split combination.

Evidence (verbatim from paper)

Evaluation metrics. To assess the quality of the rendered images, we compute the Peak Signal Noise Ratio (PSNR) between the rendered $(\hat{x})$ and real test $(x)$ images: $$ \operatorname {P S N R} (\hat {x}, x) = - 1 0 \log_ {1 0} (x - \hat {x}) ^ {2}. \tag {7} $$

Citation

@misc{deluigi2022scannerf,
  title={ScanNeRF: a Scalable Benchmark for Neural Radiance Fields},
  author={De Luigi et al. (2022)},
  year={2022},
  note={arXiv:2211.13762}
}

• arXiv: 2211.13762