Land Cover Segmentation Eval

land-cover-segmentation-eval

Land Cover Segmentation with Sparse Annotations from Sentinel-2 Imagery — Galatola et al. (2023) (arXiv:2306.16252, 2023)

What this evaluates

Evaluates land cover segmentation models on Sentinel-2 imagery using sparse annotations to predict fuel maps. It tests the model's ability to generalize across European regions affected by wildfires and compare against dense ground truth datasets (LUCAS, Urban Atlas).

Datasets

• Sentinel-2 Land Cover Dataset — total ?; splits: train (20398), val (5100), test (394); repo https://github.com/links-ads/spada

Metrics

• F1 score (primary) — range: percent
  • Harmonic mean of precision and recall across all classes. Computed on the LUCAS ground truth dataset.
• IoU — range: percent
  • Intersection over Union of predicted and ground truth masks, averaged across classes (mIoU). Computed on the Urban Atlas ground truth dataset.

Input / output format

Input: Sentinel-2 cloudless mosaic tiles (512x512 pixels) covering European regions.

Output: Per-pixel land cover segmentation mask (fuel map classes).

Scoring recipe

def compute_f1(pred, gold):
    tp = np.sum((pred == 1) & (gold == 1))
    fp = np.sum((pred == 1) & (gold == 0))
    fn = np.sum((pred == 0) & (gold == 1))
    prec = tp / (tp + fp + 1e-8)
    rec = tp / (tp + fn + 1e-8)
    return 2 * prec * rec / (prec + rec + 1e-8)

def compute_iou(pred, gold, num_classes):
    ious = []
    for c in range(num_classes):
        pred_c = (pred == c)
        gold_c = (gold == c)
        inter = np.sum(pred_c & gold_c)
        union = np.sum(pred_c | gold_c)
        ious.append(inter / (union + 1e-8))
    return np.mean(ious)

Common pitfalls

• Test evaluation is performed on full 2048x2048 sections that are tiled at runtime, not on pre-tiled 512x512 chips.
• F1 and IoU are computed on separate ground truth datasets (LUCAS and Urban Atlas respectively), not on a single unified test set.
• Baselines like S2GLC and CLC are remapped to match the considered fuel classes before evaluation.

Evidence (verbatim from paper)

Given the lack of dense annotations, we assess the performance of our solution on test areas using the two most reliable ground truths: LUCAS, using F1 score, and Urban Atlas, by means of the Intersection over Union (IoU) metric.

Citation

@misc{galatola2023spada,
  title={Land Cover Segmentation with Sparse Annotations from Sentinel-2 Imagery},
  author={Galatola et al. (2023)},
  year={2023},
  note={arXiv:2306.16252}
}

• arXiv: 2306.16252