Clara Vid Uavid Eval

clara_vid_uavid_eval

ClaraVid: A Holistic Scene Reconstruction Benchmark From Aerial Perspective With Delentropy-Based Complexity Profiling — Beche et al. (2025) (arXiv:2503.17856, 2025)

What this evaluates

Evaluates neural scene reconstruction and semantic segmentation capabilities on aerial UAV imagery. Probes the model's ability to generate high-fidelity 3D reconstructions, depth maps, and class-aware segmentation masks from multi-view inputs under varying scene complexities and viewpoint distributions.

Datasets

• ClaraVid — total ?; splits: train (-1), val (-1), test (-1)
• UAVid — total ?; splits: train (-1), test (-1)

Metrics

• reconstruction results (primary) — range: [0, 1] | percent
  • Quantitative comparison of rendered half-resolution images against ground truth, typically measured via standard image fidelity metrics (e.g., PSNR, SSIM, LPIPS) as implied by the benchmark context.
• segmentation results — range: [0, 1] | percent
  • Accuracy or mIoU of predicted semantic masks against ground truth labels, evaluated on a reduced palette adapted to the real dataset.
• depth error — range: other
  • Discrepancy between rendered depth maps and ground truth depth, computed without additional learned heads or parameters.

Input / output format

Input: Multi-view aerial imagery (native resolution for ClaraVid, downsampled by factor of 2 for UAVid), scene-level point clouds (for Gaussian Splatting initialization), and semantic class labels for training/evaluation.

Output: Half-resolution rendered images, semantic segmentation maps, and depth maps. Semantic outputs use a reduced palette adapted to the real dataset.

Scoring recipe

def evaluate(predictions, ground_truth):
    # Render at half resolution as specified in protocol
    recon_pred = predictions['images']
    recon_gt = ground_truth['images']
    recon_metric = compute_fidelity(recon_pred, recon_gt) # e.g., PSNR/SSIM
    
    # Segmentation on reduced palette
    seg_pred = predictions['seg_maps']
    seg_gt = ground_truth['seg_maps']
    seg_metric = compute_accuracy(seg_pred, seg_gt, palette='reduced')
    
    # Depth estimation
    depth_pred = predictions['depth_maps']
    depth_gt = ground_truth['depth_maps']
    depth_metric = compute_error(depth_pred, depth_gt)
    
    return recon_metric, seg_metric, depth_metric

Common pitfalls

• Excluding stationary frames with minimal motion due to poor registration, which can skew evaluation if not filtered.
• Using a reduced semantic palette for real-world dataset evaluation instead of the full native palette.
• Applying a strict train/test split for UAVid (train: every 4th frame, test: every 2nd frame with index ≡ 2 mod 4) rather than random or chronological splits.
• Limiting Gaussian Splatting to 3 million points to balance memory and accuracy, which may cap reconstruction quality.
• Reporting results at half-resolution rather than native scale, requiring careful metric normalization.

Evidence (verbatim from paper)

The reported reconstruction and segmentation results correspond to a half-resolution output relative to the original image dimensions, maintaining a trade-off between computational feasibility and fidelity in large-scale aerial scene reconstruction.

Citation

@misc{beche2025claravid,
  title={ClaraVid: A Holistic Scene Reconstruction Benchmark From Aerial Perspective With Delentropy-Based Complexity Profiling},
  author={Beche et al. (2025)},
  year={2025},
  note={arXiv:2503.17856}
}

• arXiv: 2503.17856