360roam Eval

360roam-eval

360Roam: Real-Time Indoor Roaming Using Geometry-Aware 360$^\circ$ Radiance Fields — Huajian Huang et al. (2022) (arXiv:2208.02705, 2022)

What this evaluates

Evaluates the capability of neural radiance field models to perform real-time, high-fidelity novel view synthesis on large-scale indoor scenes using 360° panoramic imagery. It probes the trade-off between rendering quality, computational efficiency, and geometric awareness in complex, unbounded indoor environments.

Datasets

• 360Roam Dataset — total ?; splits: train (-1), test (-1)

Metrics

• PSNR (primary) — range: dB
  • Peak Signal-to-Noise Ratio in decibels (dB), calculated as 10 * log10(MAX^2 / MSE) between the rendered and ground-truth images.
• SSIM — range: [0, 1]
  • Structural Similarity Index Measure, evaluating luminance, contrast, and structure similarity between images.
• LPIPS — range: [0, 1]
  • Learned Perceptual Image Patch Similarity, measuring perceptual difference using deep network features (lower is better).
• Inference Speed — range: seconds
  • Wall-clock time in seconds to render a single panoramic view at 1520×760 resolution.

Input / output format

Input: 360° panoramic images (6080×3040 resolution) with known camera poses.

Output: Rendered novel-view panoramic images (1520×760 resolution).

Scoring recipe

def compute_metrics(pred, gt):
    psnr = peak_signal_noise_ratio(gt, pred)
    ssim = structural_similarity(gt, pred)
    lpips = learned_perceptual_image_patch_similarity(gt, pred)
    return psnr, ssim, lpips

def compute_speed(render_fn):
    start = time.time()
    render_fn((1520, 760))
    return time.time() - start

Common pitfalls

• Evaluation images are resized to 1520×760, not the original 6080×3040 resolution.
• Inference speed is measured on a single RTX2080 Ti GPU without averaging over multiple runs or accounting for warm-up time.

Evidence (verbatim from paper)

Quantitative measurements in terms of appearance similarity include Peak Signal-to-noise Ratio (PSNR), Structural Similarity Index Measure (SSIM) and Learned Perceptual Image Patch Similarity (LPIPS) [45]. We also evaluated the inference speed of a panoramic view in 1520 × 760 on a single RTX2080 Ti GPU.

Citation

@misc{huang2022360roam,
  title={360Roam: Real-Time Indoor Roaming Using Geometry-Aware 360$^\circ$ Radiance Fields},
  author={Huajian Huang et al. (2022)},
  year={2022},
  note={arXiv:2208.02705}
}

• arXiv: 2208.02705