Novel View Extrapolation Eval

novel-view-extrapolation-eval

Ray Priors through Reprojection: Improving Neural Radiance Fields for Novel View Extrapolation — Zhang et al. (2022) (CVPR 2022, 2022)

What this evaluates

Evaluates a neural radiance field's ability to synthesize high-quality, artifact-free images of solid objects from viewpoints significantly outside the training camera distribution (novel view extrapolation).

Datasets

• Synthetic-NeRF* — total 300; splits: train (100), test (200)
• MobileObject — total ?; splits: train (100), test (-1)

Metrics

• PSNR (primary) — range: other (dB)
  • Peak Signal-to-Noise Ratio computed in decibels (dB) between the rendered and ground truth images.
• SSIM — range: [0, 1]
  • Structural Similarity Index measuring perceived structural similarity between the rendered and ground truth images.
• LPIPS — range: [0, 1]
  • Learned Perceptual Image Patch Similarity measuring perceptual difference using a pre-trained network.

Input / output format

Input: A set of training images with corresponding camera poses (intrinsics and extrinsics) for a single object.

Output: Rendered RGB images for each test camera pose.

Scoring recipe

def evaluate(rendered_imgs, gt_imgs):
    psnr_scores = [compute_psnr(r, g) for r, g in zip(rendered_imgs, gt_imgs)]
    ssim_scores = [compute_ssim(r, g) for r, g in zip(rendered_imgs, gt_imgs)]
    lpips_scores = [compute_lpips(r, g) for r, g in zip(rendered_imgs, gt_imgs)]
    return {
        'PSNR': np.mean(psnr_scores),
        'SSIM': np.mean(ssim_scores),
        'LPIPS': np.mean(lpips_scores)
    }

Common pitfalls

• Novel view extrapolation requires testing on viewpoints far from the training distribution, not just interpolation.
• Camera pose errors significantly degrade performance for some baselines (e.g., NSVF), so accurate COLMAP reconstruction is critical.
• Metrics should be averaged per object and then across objects, as dataset sizes vary.

Evidence (verbatim from paper)

We use PSNR, SSIM [[43]], and LPIPS [[54]] to measure the rendering quality. As reported in Table 1, RapNeRF obtains the best performance on all metrics for novel view extrapolation.

Citation

@misc{zhang2022raypriors,
  title={Ray Priors through Reprojection: Improving Neural Radiance Fields for Novel View Extrapolation},
  author={Zhang et al. (2022)},
  year={2022},
  note={CVPR 2022}
}

• arXiv: 2205.05922