Nerf View Synthesis Eval

nerf-view-synthesis-eval

NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis — Ben Mildenhall et al. (arXiv:2003.08934, 2020)

What this evaluates

Evaluates the ability of a neural radiance field to synthesize photorealistic novel views of 3D scenes from a sparse set of input images. It probes geometric reconstruction fidelity, appearance modeling (including non-Lambertian materials), and multi-view consistency across synthetic and real-world captures.

Datasets

• Diffuse Synthetic 360° (DeepVoxels) — total 1479; splits: train (479), test (1000)
• Realistic Synthetic 360° — total 300; splits: train (100), test (200)
• Real Forward-Facing — total ?; splits: train (-1), test (-1)

Metrics

• PSNR (primary) — range: dB
  • Peak Signal-to-Noise Ratio measured in decibels (dB). Computed as 10 * log10(255^2 / MSE) between the rendered and ground truth images.
• SSIM — range: [0, 1]
  • Structural Similarity Index measuring perceived structural changes between two images. Ranges from 0 to 1, where 1 indicates identical structure.
• LPIPS — range: [0, 1]
  • Learned Perceptual Image Patch Similarity. Uses a pre-trained network to compare feature activations of image patches. Lower values indicate higher perceptual similarity.

Input / output format

Input: A set of input images with known camera poses (intrinsics and extrinsics) capturing a scene from multiple viewpoints.

Output: A rendered RGB image at a novel camera viewpoint, generated via differentiable volume rendering.

Scoring recipe

def evaluate(rendered_img, gt_img):
    mse = np.mean((rendered_img - gt_img) ** 2)
    psnr = 10 * np.log10(255**2 / mse)
    ssim = compute_ssim(rendered_img, gt_img)
    lpips = compute_lpips(rendered_img, gt_img)
    return {'PSNR': psnr, 'SSIM': ssim, 'LPIPS': lpips}

Common pitfalls

• LPIPS is lower-is-better, while PSNR and SSIM are higher-is-better; confusing the direction leads to incorrect performance claims.
• The evaluation requires training a separate network for each individual scene, so metrics are averaged over independently trained models rather than a single generalizable network.
• Camera pose accuracy is critical; errors in input poses directly degrade all metrics on test views, making pose estimation quality a hidden confounder.

Evidence (verbatim from paper)

Table 1: Our method quantitatively outperforms prior work on datasets of both synthetic and real images. We report PSNR/SSIM (higher is better) and LPIPS [50] (lower is better).

Citation

@misc{mildenhall2020nerf,
  title={NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis},
  author={Ben Mildenhall et al.},
  year={2020},
  note={arXiv:2003.08934}
}

• arXiv: 2003.08934