# Epidemiological Benchmark Eval

> Evaluates spatio-temporal graph models on forecasting, stability, and denoising tasks using synthetic epidemiological data generated from PDEs. Probes the model's ability to predict future states, resist noise/dropout, and recover clean signals from corrupted graph time-series. Use when the user wants to benchmark on Epidemiological Synthetic Dataset, or asks about evaluating this task. Reports RMSE.

- Skill: `qhjqhj00/epidemiological-benchmark-eval` (Agent Skill)
- Install (CLI): `npx skillmds add qhjqhj00/epidemiological-benchmark-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/qhjqhj00/epidemiological-benchmark-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: qhjqhj00 (https://skillmd.com/u/qhjqhj00)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/qhjqhj00/epidemiological-benchmark-eval

---


# epidemiological-benchmark-eval

> Synthetic Datasets for Machine Learning on Spatio-Temporal Graphs using PDEs — Arndt et al. (2025) (arXiv:2502.04140, 2025)

## What this evaluates

Evaluates spatio-temporal graph models on forecasting, stability, and denoising tasks using synthetic epidemiological data generated from PDEs. Probes the model's ability to predict future states, resist noise/dropout, and recover clean signals from corrupted graph time-series.

## Datasets

- **Epidemiological Synthetic Dataset** — total ?; splits: train (-1), val (-1); repo https://github.com/Jostarndt/Synthetic_Datasets_for_Temporal_Graphs

## Metrics

- `RMSE` **(primary)** — range: other
  - Root Mean Squared Error, calculated as the square root of the average of squared differences between predicted and true values across all samples and time-steps.

## Input / output format

**Input**: Historical spatio-temporal graph node features over time, optionally corrupted with additive Gaussian noise or dropout for stability/denoising tasks.

**Output**: Predicted node features for subsequent time-steps.

## Scoring recipe

```python
def compute_rmse(predictions, targets):
    # predictions, targets: arrays of shape (num_samples, num_time_steps)
    squared_diff = (predictions - targets) ** 2
    mean_squared_error = np.mean(squared_diff)
    return np.sqrt(mean_squared_error)
```

## Common pitfalls

- RMSE is reported per time-step across the validation set rather than as a single aggregate scalar, requiring careful per-step plotting or aggregation.
- Stability and denoising experiments require applying specific noise distributions (Gaussian or Dropout) to the training data, which must exactly match the evaluation setup.
- Models are trained on synthetic data and evaluated on synthetic validation data for stability/denoising, not real-world data, which limits direct real-world performance claims.

## Evidence (verbatim from paper)

> A visual comparison of RMSEs for each time-step in the forecasting experiment for different models can be seen in Fig[10]... Figure 11: Plots of the RMSE of the presented models during the stability experiments for each predicted time-step on the validation dataset. The models were trained on the synthetic dataset, and evaluated on a synthetic dataset with: a) additive Gaussian noise, b) dropout noise.

## Citation

```bibtex
@misc{arndt2025synthetic,
  title={Synthetic Datasets for Machine Learning on Spatio-Temporal Graphs using PDEs},
  author={Arndt et al. (2025)},
  year={2025},
  note={arXiv:2502.04140}
}
```

- arXiv: 2502.04140