Cabuar Burned Area Delineation Eval

cabuar-burned-area-delineation-eval

CaBuAr: California Burned Areas dataset for delineation — Rege Cambrin et al. (2024) (arXiv:2401.11519, 2024)

What this evaluates

This benchmark evaluates a model's ability to accurately delineate wildfire-affected regions from satellite imagery. It probes pixel-level binary segmentation and change detection capabilities using pre- and post-fire Sentinel-2 multispectral data.

Datasets

• CaBuAr — total ?; splits: (unstated); repo https://github.com/DarthReca/CaBuAr

Metrics

• pixel-level accuracy (primary) — range: [0, 1]
  • Standard binary classification metric calculated as the ratio of correctly classified pixels (burned and undamaged) to the total number of pixels.
• separability index (SI) — range: other
  • Ratio of between-class variance to within-class variance for a given spectral index, measuring how well the index separates burned from unburned pixels.

Input / output format

Input: Multispectral Sentinel-2 L2A raster images (pre-fire, post-fire, or both).

Output: Binary image mask where each pixel is labeled as either burned or undamaged.

Scoring recipe

def evaluate_segmentation(pred_mask, gt_mask):
    correct = (pred_mask == gt_mask).sum()
    total = gt_mask.size
    accuracy = correct / total
    return {'accuracy': accuracy}

# For spectral index assessment (Task iv):
# SI = between_class_variance / within_class_variance
# Compute per index using ground truth labels as class indicators.

Common pitfalls

• Models may overfit to specific fire events or geographic regions without cross-validation across different years.
• Spectral index thresholds are highly sensitive to phenological conditions and atmospheric corrections, requiring consistent preprocessing.
• Confusing pre-fire and post-fire inputs can lead to incorrect change detection or segmentation results.

Evidence (verbatim from paper)

The proposed dataset can be used as a benchmark for different tasks in supervised and unsupervised scenarios: (i) binary segmentation based on post-fire acquisitions only, (ii) binary segmentation based on pre-fire and post-fire acquisitions, (iii) binary change detection on pre-fire and post-fire images, and (iv) performance assessment of spectral indexes. ... researchers may use the dataset to evaluate the quality of the index by computing the separability index (SI).

Citation

@misc{regecambrin2024cabuar,
  title={CaBuAr: California Burned Areas dataset for delineation},
  author={Rege Cambrin et al. (2024)},
  year={2024},
  note={arXiv:2401.11519}
}

• arXiv: 2401.11519