Realx3d Smoke Eval

realx3d-smoke-eval

3D Smoke Scene Reconstruction Guided by Vision Priors from Multimodal Large Language Models — Zheng et al. (2026) (arXiv:2604.05687, 2026)

What this evaluates

Evaluates the ability of 3D reconstruction models to synthesize novel views of smoke-degraded scenes with high photometric fidelity and structural preservation. It probes view-dependent medium modeling and multi-view consistency under severe scattering conditions.

Datasets

• RealX3D (NTIRE 2026 Track 2 Smoke Subset) — total ?; splits: validation (1), development (4), test (3)

Metrics

• PSNR (primary) — range: other
  • Peak Signal-to-Noise Ratio measures pixel-level reconstruction error and is sensitive to absolute intensity differences between rendered and ground-truth images.
• SSIM — range: other
  • Structural Similarity Index evaluates image similarity based on luminance, contrast, and structure, aligning more closely with human visual perception.

Input / output format

Input: Multi-view RGB images and corresponding camera parameters for training scenes; novel view synthesis (NVS) target camera poses for evaluation scenes.

Output: Rendered novel view synthesis (NVS) images corresponding to the target poses.

Scoring recipe

def compute_metrics(predictions, ground_truth):
    # predictions and ground_truth are aligned RGB images
    mse = np.mean((predictions - ground_truth) ** 2)
    psnr = 10 * np.log10(255**2 / mse)
    ssim = compute_ssim(predictions, ground_truth)
    return psnr, ssim

Common pitfalls

• Test set lacks reference images; evaluation requires submitting rendered images to the NTIRE 2026 challenge server for online scoring.
• Models are trained independently per scene rather than jointly, which may overfit to specific scene characteristics and limit generalization claims.
• Metrics are computed on undisclosed ground-truth images by the server, so local development set scores may not perfectly correlate with final test rankings.

Evidence (verbatim from paper)

We adopt Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) as the evaluation metrics for photometric fidelity. PSNR measures pixel-level reconstruction error and is sensitive to absolute intensity differences. SSIM evaluates image similarity based on luminance, contrast, and structure, aligning more closely with human visual perception. For the development and test sets, we submit the rendered novel NVS images to the challenge server for online evaluation. The server computes the metrics using undisclosed ground-truth reference images and releases the results.

Citation

@misc{zheng2026smokegs,
  title={3D Smoke Scene Reconstruction Guided by Vision Priors from Multimodal Large Language Models},
  author={Zheng et al. (2026)},
  year={2026},
  note={arXiv:2604.05687}
}

• arXiv: 2604.05687