Tornado Prediction Eval

tornado-prediction-eval

Predicting Tornadoes days ahead with Machine Learning — Coccomini and Zara (2022) (arXiv:2208.05855, 2022)

What this evaluates

Evaluates machine learning classifiers' ability to predict tornado occurrences up to five days in advance using historical meteorological grid data. It measures detection capability and false alarm rates under a strict temporal train-test split simulating real-world forecasting.

Datasets

• Custom Tornado Forecasting Dataset — total 5103; splits: train (4993), test (200)

Metrics

• POD (primary) — range: [0, 1]
  • POD = tp / (tp + fn)
• FAR — range: [0, 1]
  • FAR = fp / (tp + fp)

Input / output format

Input: Five daily meteorological grids of size 19×19 cells per event instance, containing temperature, wind, precipitation, and cloud metrics.

Output: Binary classification label (tornadic vs. non-tornadic) or prediction probability.

Scoring recipe

def compute_metrics(tp, fp, fn):
    pod = tp / (tp + fn) if (tp + fn) > 0 else 0.0
    far = fp / (tp + fp) if (tp + fp) > 0 else 0.0
    return {'POD': pod, 'FAR': far}

Common pitfalls

• Temporal split is strict: train on 1990–2016, test only on 2017 to simulate real-time forecasting.
• Non-tornadic instances are defined by a 10-day gap rule from any tornadic event, not random negative sampling.
• Input is pre-aggregated 19×19 grids, not raw point measurements or continuous time series.

Evidence (verbatim from paper)

Classifiers, evaluated using two quality metrics commonly used in this field, Probability Of Detection, $POD = \frac{tp}{(tp + fn)}$ and False Alarm Rate, $FAR = \frac{fp}{(tp + fp)}$ are listed in Table 2.

Citation

@misc{coccomini2022predictingtornadoes,
  title={Predicting Tornadoes days ahead with Machine Learning},
  author={Coccomini and Zara (2022)},
  year={2022},
  note={arXiv:2208.05855}
}

• arXiv: 2208.05855