Wildfire Prediction Morocco Eval

wildfire-prediction-morocco-eval

Advanced Wildfire Prediction in Morocco: Developing a Deep Learning Dataset from Multisource Observations — Jadouli et al. (2024) (arXiv:2411.06202, 2024)

What this evaluates

Evaluates machine learning and deep learning models' ability to predict wildfire occurrences in Morocco using integrated spatio-temporal environmental and meteorological features. It specifically tests temporal generalization by training on pre-2022 data and validating on post-2022 data.

Datasets

• Morocco Wildfire Dataset — total ?; splits: train (-1), test (-1); repo https://github.com/AyoubJadouli/WildfireForecastingDataPrep

Metrics

• accuracy (primary) — range: [0, 1]
  • Proportion of correctly classified instances (true positives + true negatives) out of the total number of instances.
• AUC — range: [0, 1]
  • Area Under the Receiver Operating Characteristic Curve, measuring the model's ability to distinguish between classes across all classification thresholds.
• average precision — range: [0, 1]
  • Average precision score, calculated as the area under the precision-recall curve.
• average recall — range: [0, 1]
  • Average recall score, representing the true positive rate averaged across thresholds or classes.
• AUC-PR — range: [0, 1]
  • Area Under the Precision-Recall Curve, particularly useful for imbalanced datasets to evaluate the trade-off between precision and recall.

Input / output format

Input: Tabular spatio-temporal features including latitude, longitude, NDVI, soil moisture, distance from sea, and monthly/annual meteorological variables (temperature, precipitation, wind).

Output: Binary label indicating wildfire occurrence (1 for occurrence, 0 for non-occurrence).

Scoring recipe

def compute_metrics(y_true, y_pred_proba, threshold=0.5):
    y_pred = (y_pred_proba >= threshold).astype(int)
    accuracy = np.mean(y_true == y_pred)
    tp = np.sum((y_pred == 1) & (y_true == 1))
    fp = np.sum((y_pred == 1) & (y_true == 0))
    fn = np.sum((y_pred == 0) & (y_true == 1))
    precision = tp / (tp + fp + 1e-8)
    recall = tp / (tp + fn + 1e-8)
    auc_roc = roc_auc_score(y_true, y_pred_proba)
    auc_pr = average_precision_score(y_true, y_pred_proba)
    return {'accuracy': accuracy, 'AUC': auc_roc, 'average precision': precision, 'average recall': recall, 'AUC-PR': auc_pr}

Common pitfalls

• Temporal data leakage: Random train/test splits are invalid; the protocol strictly requires training on data before 2022-01-01 and testing on data after 2022-01-01.
• Class imbalance: Wildfire occurrences are rare, making AUC-PR a more reliable metric than standard accuracy or AUC-ROC for evaluating model performance.

Evidence (verbatim from paper)

We evaluate the models using metrics such as accuracy, Area Under the Curve (AUC), average precision, and average recall. ... It should also be noted that the results are based on validation datasets from a different time period (after 2022-01-01) than the data used in training (before 2022-01-01), indicating that the study is highly effective for predicting real wildfire cases.

Citation

@misc{jadouli2024wildfire,
  title={Advanced Wildfire Prediction in Morocco: Developing a Deep Learning Dataset from Multisource Observations},
  author={Jadouli et al. (2024)},
  year={2024},
  note={arXiv:2411.06202}
}

• arXiv: 2411.06202