Neural Rendering Eval

neural-rendering-eval

Anisotropic Neural Representation Learning for High-Quality Neural Rendering — Wang et al. (2023) (arXiv:2311.18311, 2023)

What this evaluates

Evaluates novel view synthesis quality in neural rendering by measuring how well a model reconstructs unseen viewpoints from a set of training images. It probes the model's ability to capture view-dependent appearance, geometric consistency, and texture fidelity under challenging materials and real-world lighting.

Datasets

• Blender — total ?; splits: train (-1), test (-1)
• Shiny Blender — total ?; splits: train (-1), test (-1)
• Mip-360 — total ?; splits: train (-1), test (-1)

Metrics

• PSNR (primary) — range: [0, 1] | dB | other
  • Peak signal-to-noise ratio computed as 10 * log10(MAX^2 / MSE) between rendered and ground truth images.
• SSIM — range: [0, 1]
  • Structural similarity index measuring luminance, contrast, and structure between images.
• LPIPS — range: [0, 1]
  • Learning perceptual image patch similarity using deep network features to measure perceptual distance.
• Avg. — range: other
  • Arithmetic mean summarizing the PSNR, SSIM, and LPIPS values.

Input / output format

Input: Set of training images with corresponding camera poses for a static scene.

Output: Rendered RGB images at novel camera viewpoints.

Scoring recipe

def compute_metrics(rendered, ground_truth):
    mse = mean_squared_error(rendered, ground_truth)
    psnr = 10 * math.log10(255**2 / mse)
    ssim = structural_similarity(rendered, ground_truth)
    lpips = perceptual_similarity(rendered, ground_truth)
    avg = (psnr + ssim + lpips) / 3
    return psnr, ssim, lpips, avg

Common pitfalls

• The 'Avg.' metric is a direct arithmetic mean of PSNR (dB), SSIM, and LPIPS, which have different scales, making cross-method comparisons on Avg. potentially misleading without normalization.
• Baseline implementations vary: official code is used for Nerfacc, K-Planes, and Tri-mipRF, but an open-source version is used for Zip-NeRF, which may introduce implementation discrepancies.
• Optimal hyperparameters differ by dataset: SH degree L=3 is best for Blender, while L=4 is best for Mip-360, and regularization weight lambda=1e-4 is used globally despite dataset-specific variations.

Evidence (verbatim from paper)

We follow previous NeRF methods and report our quantitative results in terms of peak signal-to-noise ratio (PSNR), structural similarity index (SSIM) [[71]], learning perceptual image patch similarity (LPIPS) [[82]] and an average error (Avg.) [[4]] which summarizes three above metrics.

Citation

@misc{wang2023anisotropic,
  title={Anisotropic Neural Representation Learning for High-Quality Neural Rendering},
  author={Wang et al. (2023)},
  year={2023},
  note={arXiv:2311.18311}
}

• arXiv: 2311.18311