Wildfire Spread Forecasting Eval

wildfire-spread-forecasting-eval

Wildfire spread forecasting with Deep Learning — Anastasiou et al. (2025) (arXiv:2505.17556, 2025)

What this evaluates

This benchmark evaluates a model's ability to forecast the final spatial extent of a wildfire using multi-day spatio-temporal environmental and dynamic features. It probes the model's capacity to capture complex temporal dependencies and spatial patterns in binary segmentation tasks under significant class imbalance.

Datasets

• Mediterranean Wildfire Dataset (2006-2022) — total 9561; splits: train (7619), val (841), test (1101); repo https://github.com/Orion-AI-Lab/WildFireSpread

Metrics

• Dice Score (primary) — range: [0, 1]
  • 2 * |A ∩ B| / (|A| + |B|), where A and B are the predicted and ground truth burned area masks. Mathematically equivalent to F1-Score in binary segmentation.
• IoU — range: [0, 1]
  • |A ∩ B| / |A ∪ B|, measuring the overlap between predicted and ground truth masks. More conservative than Dice as it penalizes false positives and negatives more heavily.
• Precision — range: [0, 1]
  • TP / (TP + FP)
• Recall — range: [0, 1]
  • TP / (TP + FN)
• F1-Score — range: [0, 1]
  • 2 * (Precision * Recall) / (Precision + Recall)

Input / output format

Input: 152-channel tensor (14 dynamic variables × 10 time steps + 12 static variables) of spatial dimensions 64×64 pixels. Covers a 10-day window (4 days pre-ignition to 5 days post-ignition). All values normalized to [0, 1].

Output: Binary segmentation mask of dimensions 64×64 pixels, where burned pixels are labeled 1 and unburned pixels are labeled 0.

Scoring recipe

def compute_dice(pred_mask, gold_mask):
    pred = pred_mask.flatten()
    gold = gold_mask.flatten()
    intersection = np.sum(pred * gold)
    return (2.0 * intersection) / (np.sum(pred) + np.sum(gold))

Common pitfalls

• Significant class imbalance: only ~100 burned pixels out of 4096 per patch, requiring specialized loss functions like BCEDice Loss.
• Temporal split strategy: data is split strictly by year (2006-2020 train, 2021 val, 2022 test), which tests year-to-year generalization but may not reflect spatial generalization across the region.
• F1-Score and Dice Score are mathematically identical for binary segmentation, so reporting both is redundant and the paper only includes Dice in results tables.

Evidence (verbatim from paper)

Model performance was evaluated using standard evaluation metrics commonly applied in binary segmentation tasks: Precision, Recall, Dice Score, F1-Score, and Intersection over Union (IoU). Notably, in the context of binary 2D segmentation, the F1-score is mathematically equivalent to the Dice Score. As such, only the Dice Score is included in the results tables for clarity. Among these scores, the Dice Score was selected as the primary criterion for hyperparameter tuning and model selection during training, due to its effectiveness in quantifying the spatial overlap between predicted and reference burned areas.

Citation

@misc{anastasiou2025wildfire,
  title={Wildfire spread forecasting with Deep Learning},
  author={Anastasiou et al. (2025)},
  year={2025},
  note={arXiv:2505.17556}
}

• arXiv: 2505.17556