Df3dv 1k Eval

df3dv-1k-eval

DF3DV-1K: A Large-Scale Dataset and Benchmark for Distractor-Free Novel View Synthesis — Lu et al. (2026) (arXiv:2604.13416, 2026)

What this evaluates

Evaluates the robustness of distractor-free novel view synthesis methods against large-scale, diverse distractor scenarios. It measures how well radiance field and 3D Gaussian Splatting models can reconstruct clean 3D scenes from cluttered or dynamically changing inputs without degrading static background quality.

Datasets

• DF3DV-1K — total 1048; splits: test (-1)
• DF3DV-41 — total 41; splits: test (-1)

Metrics

• PSNR (primary) — range: dB
  • Peak Signal-to-Noise Ratio in decibels, calculated as 10 * log10(MAX^2 / MSE), where MAX is the maximum possible pixel value (typically 1.0 for normalized images) and MSE is the mean squared error between predicted and ground truth images.
• SSIM — range: [0, 1]
  • Structural Similarity Index, measuring perceived changes in structural information, luminance, and contrast between two images. Values range from -1 to 1, with 1 indicating identical images.
• LPIPS — range: [0, 1]
  • Learned Perceptual Image Patch Similarity, a perceptual metric based on deep network features that correlates better with human judgment than pixel-wise metrics. Lower values indicate higher perceptual similarity.

Input / output format

Input: Cluttered/distorted image sequences containing distractors (e.g., dynamic objects, fluid phenomena, nighttime conditions) used to train or evaluate radiance field reconstruction.

Output: Rendered novel-view RGB images from the reconstructed 3D scene or enhanced by the 2D diffusion model.

Scoring recipe

def compute_metrics(pred, gold):
    mse = np.mean((pred - gold) ** 2)
    psnr = 10 * np.log10(1.0 / mse)
    ssim = compute_ssim(pred, gold)
    lpips = compute_lpips(pred, gold)
    return {'PSNR': psnr, 'SSIM': ssim, 'LPIPS': lpips}

Common pitfalls

• Methods relying on semantic features or fixed thresholds fail on semantically similar distractors or low-light scenes.
• Qualitative 'zoom-in' comparisons can mask performance gaps; the benchmark requires direct, unzoomed visual comparison.
• Fine-tuning enhancers on smaller prior benchmarks (e.g., RobustNeRF) severely degrades generalization on DF3DV-1K/41.

Evidence (verbatim from paper)

DI2FIX improves the average rendering quality of 3DGS*[kerbl20233d]* and distractor-free radiance-field methods, particularly in PSNR and LPIPS, achieving a 0.96 dB PSNR gain and a 0.057 reduction in LPIPS.

Citation

@misc{lu2026df3dv1k,
  title={DF3DV-1K: A Large-Scale Dataset and Benchmark for Distractor-Free Novel View Synthesis},
  author={Lu et al. (2026)},
  year={2026},
  note={arXiv:2604.13416}
}

• arXiv: 2604.13416