Neusample Eval

neusample-eval

NeuSample: Neural Sample Field for Efficient View Synthesis — Fang et al. (2021) (arXiv:2111.15552, 2021)

What this evaluates

Evaluates the efficiency and rendering quality of a neural sample field for novel view synthesis. It probes how well a model can learn ray sampling distributions to reduce computation cost while maintaining high-fidelity image reconstruction compared to baseline NeRF methods.

Datasets

• Realistic Synthetic 360° — total 8; splits: train (100), test (200)
• Real Forward-Facing — total 8; splits: train (-1), test (-1)

Metrics

• PSNR (primary) — range: other
  • Peak Signal-to-Noise Ratio computed in dB between rendered and ground-truth images. Higher is better.
• SSIM — range: [0, 1]
  • Structural Similarity Index measuring perceived quality of reconstructed images. Ranges from 0 to 1, higher is better.
• LPIPS — range: [0, 1]
  • Learned Perceptual Image Patch Similarity using deep features to measure perceptual distance. Lower is better.

Input / output format

Input: Ray origins and directions (with 10-frequency positional encoding), camera poses, and training images at specified resolutions (800x800 or 1008x756).

Output: Rendered RGB images (pixel colors) for each test camera pose.

Scoring recipe

def compute_metrics(rendered_img, gt_img):
    psnr = 10 * log10(1.0 / mse(rendered_img, gt_img))
    ssim = structural_similarity(rendered_img, gt_img, data_range=1.0)
    lpips = perceptual_loss(rendered_img, gt_img)
    return psnr, ssim, lpips
# Average per scene, then across scenes.

Common pitfalls

• NeRF-ID uses a massive batch size (66k on 16 TPUs) that artificially inflates PSNR; direct comparison without noting this is unfair.
• Resolution settings vary (1008x756 vs 504x378); results are not directly comparable across resolutions without normalization.
• Sample count (192, 128, 64, 32) drastically changes computation cost; evaluating only at one sample count misses the quality-speed trade-off.

Evidence (verbatim from paper)

We use two datasets, Realistic Synthetic 360° and Real Forward-Facing, to evaluate our method... The Realistic Synthetic 360° dataset consists of 8 scenes and each one includes 100 views for training and 200 views for testing. We take all views with an 800×800 resolution. The Real Forward-Facing dataset contains 8 complex real-world scenes... Each scene includes 20 - 62 images. Following [17], we hold out 1/8 images for testing and the rest are for training. All images are at 1008×756 pixels for experiments if unspecified. We show main PSNR results in Tab.1... Table 4: Depth boost effectiveness study on the “fern” scene of the Real Forward-Facing dataset. | #Sample | Depth Boost | PSNR↑ | SSIM↑ | LPIPS↓ |

Citation

@misc{fang2021neusample,
  title={NeuSample: Neural Sample Field for Efficient View Synthesis},
  author={Fang et al. (2021)},
  year={2021},
  note={arXiv:2111.15552}
}

• arXiv: 2111.15552