Chickenpox Hungary Forecasting Eval

chickenpox-hungary-forecasting-eval

Chickenpox Cases in Hungary: a Benchmark Dataset for Spatiotemporal Signal Processing with Graph Neural Networks — Rozemberczki et al. (2021) (arXiv:2102.08100, 2021)

What this evaluates

Evaluates the ability of recurrent graph neural networks to forecast spatiotemporal epidemiological time series. It probes how well models capture spatial dependencies between adjacent regions and temporal dynamics like seasonality and zero-inflation over multiple forecasting horizons.

Datasets

• Chickenpox Cases in Hungary — total ?; splits: train (-1), test (-1); repo https://github.com/benedekrozemberczki/spatiotemporal_datasets

Metrics

• mean squared error (primary) — range: other
  • Average of squared differences between predicted and actual weekly chickenpox case counts across all counties and time steps. Calculated over 10 independent experimental runs and reported with standard deviation.

Input / output format

Input: Graph structure representing undirected direct adjacency of 20 Hungarian counties, combined with 8 temporal lags of standardized weekly chickenpox case counts as node features.

Output: Predicted weekly chickenpox case counts for each county over a fixed forecasting horizon (10, 20, or 40 weeks).

Scoring recipe

def compute_mse(predictions, targets):
    squared_errors = (predictions - targets) ** 2
    mse = squared_errors.mean()
    return mse.item()
# Evaluated over 10 runs, report mean ± std of MSE across runs

Common pitfalls

• The dataset uses standardized (normalized) time series for training, so raw MSE values on the original scale will differ significantly from reported results.
• Performance varies non-monotonically with forecasting horizon; some models perform worse than random at 40 weeks, requiring careful horizon-specific evaluation.
• Graph adjacency is based strictly on direct county borders, not travel or population flow, which limits spatial dependency modeling.

Evidence (verbatim from paper)

We tested the predictive performance of recurrent graph neural networks on county level chickenpox time series forecasting. Using the PyTorch Geometric Temporal [5, 16, 18] implementation of the models we trained on the standardized chickenpox time series and predicted it for a fixed number of weeks ahead. ... In Table 4 we present average mean squared error values for various forecasting horizons calculated from 10 experimental runs.

Citation

@misc{rozemberczki2021chickenpox,
  title={Chickenpox Cases in Hungary: a Benchmark Dataset for Spatiotemporal Signal Processing with Graph Neural Networks},
  author={Rozemberczki et al. (2021)},
  year={2021},
  note={arXiv:2102.08100}
}

• arXiv: 2102.08100