Map Based Fdi Eval

map-based-fdi-eval

Not a fragment, but the whole: Map-based evaluation of data-driven Fire Danger Index models — Alvi et al. (2026) (arXiv:2603.25469, 2026)

What this evaluates

This evaluation probes the operational utility of data-driven Fire Danger Index (FDI) models for wildfire forecasting. It assesses both point-level classification accuracy and full-map spatial inference performance, explicitly quantifying detection rates and false positive distributions under realistic deployment conditions.

Datasets

• FireCube — total ?; splits: test (-1), validation (-1)

Metrics

• Recall — range: [0, 1]
  • Percentage of fires correctly recognized (where predicted FDI > 0.5) out of total actual fires on a given day.
• Precision — range: [0, 1]
  • Ratio of correctly predicted fire locations to all locations predicted as fire.
• F1-score — range: [0, 1]
  • Harmonic mean of Precision and Recall: 2 * (Precision * Recall) / (Precision + Recall).
• Map-based Recall Percentiles (primary) — range: [0, 1]
  • Daily recall values are computed for each day with a fire, then aggregated into Q-th percentiles (e.g., 50% percentile indicates the recall threshold exceeded on 50% of fire days).
• False Positive Rate (Skewness) — range: other
  • Assessed by computing the skewness of the FDI value distribution on randomly selected no-fire days; positive skew indicates fewer false positives.

Input / output format

Input: Daily environmental and weather features for fire-susceptible grid cells/locations, filtered by land cover type.

Output: A daily Fire Danger Index (FDI) map assigning a continuous risk score to each fire-susceptible location.

Scoring recipe

def compute_standard_metrics(preds, golds):
    tp = sum(p > 0.5 and g > 0.5 for p, g in zip(preds, golds))
    fp = sum(p > 0.5 and g == 0 for p, g in zip(preds, golds))
    fn = sum(p == 0 and g > 0.5 for p, g in zip(preds, golds))
    recall = tp / (tp + fn)
    precision = tp / (tp + fp)
    f1 = 2 * precision * recall / (precision + recall)
    return recall, precision, f1

def compute_map_recall_percentiles(daily_preds, daily_golds):
    daily_recalls = []
    for day in fire_days:
        day_tp = sum(p > 0.5 and g > 0.5 for p, g in zip(daily_preds[day], daily_golds[day]))
        day_total = sum(g > 0.5 for g in daily_golds[day])
        daily_recalls.append(day_tp / day_total)
    return np.percentile(daily_recalls, [40, 50, 60, 70, 80, 90])

Common pitfalls

• Test set samples are balanced/curated, which may overestimate performance compared to full-map operational inference where class distribution is highly imbalanced.
• FDI values for non-fire-susceptible areas (urban, water) are excluded from evaluation, so models must correctly mask or ignore these regions.
• Map-based recall percentiles aggregate daily performance, potentially masking days with extremely poor detection that are averaged out by high-recall days.

Evidence (verbatim from paper)

We compute daily full-map inference for the fire season (JAS) in 2020 and 2021 (149 days in total). Recall - the percentage of fires correctly recognized (having FDI > 0.5) - is computed from each day when there was a fire. The model performance is expressed in the 40%, 50%, 60%, 70%, 80%, and 90% percentiles. The Q-th percentile is the upper limit of recall on Q% of the days.

Citation

@misc{alvi2026mapbasedfdi,
  title={Not a fragment, but the whole: Map-based evaluation of data-driven Fire Danger Index models},
  author={Alvi et al. (2026)},
  year={2026},
  note={arXiv:2603.25469}
}

• arXiv: 2603.25469