Wildrayzer Nvs Eval

wildrayzer-nvs-eval

WildRayZer: Self-supervised Large View Synthesis in Dynamic Environments — Chen et al. (2026) (arXiv:2601.10716, 2026)

What this evaluates

Evaluates novel view synthesis and motion mask estimation in dynamic environments where both camera and objects move. It probes a model's ability to remove transient objects, complete occluded backgrounds, and preserve scene geometry from sparse input views without 3D supervision or ground-truth poses.

Datasets

• D-RE10K-Mask — total 74; splits: test (-1)
• D-RE10K-iPhone — total 50; splits: test (-1)

Metrics

• PSNR (primary) — range: dB
  • Peak Signal-to-Noise Ratio, computed as 10 * log10(MAX^2 / MSE) between predicted and ground truth images.
• SSIM — range: [0, 1]
  • Structural Similarity Index, measuring luminance, contrast, and structure similarity between images.
• LPIPS — range: [0, 1]
  • Learned Perceptual Image Patch Similarity, using deep features to measure perceptual distance.
• mIoU — range: [0, 1]
  • Mean Intersection over Union for motion mask segmentation against human-verified annotations.
• Recall — range: [0, 1]
  • True positive rate for motion mask pixels against ground truth annotations.

Input / output format

Input: 2–4 input images (sparse views) for scene and pose estimation; 6 target views for evaluation.

Output: Synthesized novel view images and/or binary motion masks for transient regions.

Scoring recipe

def compute_nvs_metrics(pred, gt):
    mse = np.mean((pred - gt) ** 2)
    psnr = 10 * np.log10(255**2 / mse)
    ssim = compute_ssim(pred, gt)
    lpips = compute_lpips(pred, gt)
    return psnr, ssim, lpips

def compute_mask_metrics(pred_mask, gt_mask):
    intersection = np.logical_and(pred_mask, gt_mask).sum()
    union = np.logical_or(pred_mask, gt_mask).sum()
    miou = intersection / union if union > 0 else 0.0
    recall = intersection / gt_mask.sum() if gt_mask.sum() > 0 else 0.0
    return miou, recall

Common pitfalls

• Full-image PSNR can be misleading because transient objects typically cover only small image areas, masking background reconstruction quality.
• Off-the-shelf motion estimators are tuned on the D-RE10K validation set but must be evaluated without modification on the iPhone benchmark to test cross-dataset generalization.
• Sparse-view settings (2–4 inputs) cause over-masking or diffuse boundaries in baseline methods, requiring learned cross-view completion rather than simple token dropping.

Evidence (verbatim from paper)

We adopt a sparse-view NVS protocol: two, three or four input images are used for pose and scene estimation, and six target views are used for evaluation. We report static-region metrics for D-RE10K-Mask and report full-image metrics for D-RE10K-iPhone. Image quality is evaluated using PSNR, SSIM, and LPIPS. Furthermore, when evaluating motion mask, motion-mask accuracy is measured using mIoU and Recall against human verified annotations.

Citation

@misc{chen2026wildrayzer,
  title={WildRayZer: Self-supervised Large View Synthesis in Dynamic Environments},
  author={Chen et al. (2026)},
  year={2026},
  note={arXiv:2601.10716}
}

• arXiv: 2601.10716