Gen Nerf Eval

gen-nerf-eval

Gen-NeRF: Efficient and Generalizable Neural Radiance Fields via Algorithm-Hardware Co-Design — Yonggan Fu et al. (2023) (arXiv:2304.11842, 2023)

What this evaluates

Evaluates the rendering quality and computational efficiency of a generalizable Neural Radiance Field (NeRF) model for novel view synthesis. It measures how accurately the model reconstructs unseen scenes from a few source views, balancing image fidelity against computational cost and hardware throughput.

Datasets

• NeRF Synthetic — total ?; splits: test (-1)
• LLFF — total ?; splits: test (-1)
• DeepVoxels — total ?; splits: test (-1)

Metrics

• PSNR (primary) — range: other
  • Peak Signal-to-Noise Ratio, calculated as 10 * log10(255^2 / MSE) between rendered and ground-truth images. Higher values indicate better reconstruction quality.
• LPIPS — range: [0, 1]
  • Learned Perceptual Image Patch Similarity, a deep learning-based metric that measures perceptual difference between images. Lower values indicate higher perceptual similarity.
• MFLOPs/pixel — range: other
  • Million Floating Point Operations per pixel, measuring the computational cost of rendering a single pixel. Lower values indicate higher efficiency.
• FPS — range: other
  • Frames Per Second, measuring the rendering throughput of the hardware accelerator. Higher values indicate faster real-time performance.

Input / output format

Input: A set of source RGB images (typically 4-10 views) and their corresponding camera poses for a target scene.

Output: A single rendered RGB image of a novel viewpoint for the target scene.

Scoring recipe

def compute_metrics(rendered_img, gt_img):
    mse = np.mean((rendered_img - gt_img) ** 2)
    psnr = 10 * np.log10(255.0**2 / mse)
    lpips_val = lpips_model(rendered_img, gt_img)
    return psnr, lpips_val

# Efficiency metrics are computed separately:
# MFLOPs/pixel = total_model_flops / (image_width * image_height)
# FPS = 1 / (latency_per_frame_in_seconds)

Common pitfalls

• Hardware performance metrics (FPS, area, power) are derived from cycle-accurate simulation and gate-level synthesis, not physical silicon measurements.
• MFLOPs/pixel values are highly dependent on the coarse-then-focus sampling strategy and the number of source views used; comparing across different source view counts without normalization is misleading.
• The evaluation focuses on zero-shot generalizable NeRF; per-scene finetuning results are reported separately and should not be conflated with the base generalizable model.

Evidence (verbatim from paper)

As shown in rows 2-4 of Tab.[2], we can observe that (1) removing the ray transformer leads to a significant PSNR drop due to erroneous density estimation, and (2) integrating Ray-Mixer results in considerably better reconstruction accuracy over that w/o Ray-Mixer, e.g., a 3.37 PSNR improvement on average across four scenes from LLFF.

Citation

@misc{fu2023gennerf,
  title={Gen-NeRF: Efficient and Generalizable Neural Radiance Fields via Algorithm-Hardware Co-Design},
  author={Yonggan Fu et al. (2023)},
  year={2023},
  note={arXiv:2304.11842}
}

• arXiv: 2304.11842