Fgn Weather Forecasting Eval

fgn-weather-forecasting-eval

Skillful joint probabilistic weather forecasting from marginals — Alet et al. (2025) (arXiv:2506.10772, 2025)

What this evaluates

Evaluates a functional generative network for medium-range probabilistic weather forecasting against operational ground truth (HRES-fc0) and a diffusion-based baseline (GenCast). Probes the model's ability to capture joint spatial structures and predict tropical cyclone tracks using deterministic and probabilistic scoring rules.

Datasets

• HRES-fc0 — total ?; splits: val (-1), test (-1)
• ERA5 — total ?; splits: pre-training (-1)

Metrics

• probabilistic metrics (primary) — range: other
  • Not specified in provided section; typically encompasses ensemble scoring rules such as CRPS, log-likelihood, or reliability diagrams for probabilistic forecasts.

Input / output format

Input: Gridded atmospheric state variables at 0.25° resolution from HRES-fc0 or ERA5, used as conditioning inputs for the generative model.

Output: Ensemble of 15-day forecast trajectories with 6-hour timesteps, representing predicted atmospheric fields.

Scoring recipe

def compute_metric(predictions, gold):
    # Exact formula not detailed in provided section
    # predictions: list of ensemble forecast trajectories
    # gold: ground truth HRES-fc0 analysis fields
    score = aggregate_probabilistic_scores(predictions, gold)
    return score

Common pitfalls

• Specific metric definitions and scoring thresholds are not detailed in this section; refer to the full paper for exact formulas.
• Evaluation uses strict temporal splits (2022 validation, 2023 test) rather than random shuffling, making temporal leakage a critical concern.

Evidence (verbatim from paper)

achieving state-of-the-art performance in deterministic and probabilistic metrics, superior joint spatial structure, and significantly improved tropical cyclone track predictions compared to GenCast and ENS.

Citation

@misc{alet2025skillful,
  title={Skillful joint probabilistic weather forecasting from marginals},
  author={Alet et al. (2025)},
  year={2025},
  note={arXiv:2506.10772}
}

• arXiv: 2506.10772