Timeseries Forecasting Eval

timeseries-forecasting-eval

WaveMoE: A Wavelet-Enhanced Mixture-of-Experts Foundation Model for Time Series Forecasting — Wu et al. (2026) (arXiv:2604.10544, 2026)

What this evaluates

Evaluates the forecasting accuracy of a time series foundation model across diverse real-world and synthetic datasets. It probes the model's ability to capture temporal dynamics, periodicity, and multi-scale patterns over a fixed context window to predict future values.

Datasets

• ETT1 — total ?; splits: (unstated)
• ETT2 — total ?; splits: (unstated)
• Exchange Rate — total ?; splits: (unstated)
• M1 Monthly — total ?; splits: (unstated)
• M1 Quarterly — total ?; splits: (unstated)
• M1 Yearly — total ?; splits: (unstated)
• M5 — total ?; splits: (unstated)
• Monash M3 — total ?; splits: (unstated)
• NN5 — total ?; splits: (unstated)
• Traffic — total ?; splits: (unstated)
• Weather — total ?; splits: (unstated)
• M4 Monthly — total ?; splits: (unstated)
• Entsoe — total ?; splits: (unstated)
• Solar with Weather — total ?; splits: (unstated)
• UK Covid — total ?; splits: (unstated)
• Sensor Data — total ?; splits: (unstated)

Metrics

• MSE (primary) — range: [0, ∞)
  • Mean Squared Error: the average of the squared differences between predicted and actual values. Formula: $\frac{1}{N}\sum_{i=1}^{N}(y_i - \hat{y}_i)^2$.
• MAE — range: [0, ∞)
  • Mean Absolute Error: the average of the absolute differences between predicted and actual values. Formula: $\frac{1}{N}\sum_{i=1}^{N}|y_i - \hat{y}_i|$.

Input / output format

Input: A sequence of 512 time steps (context window) from a time series dataset.

Output: A sequence of 96 time steps (prediction horizon) representing the forecasted future values.

Scoring recipe

def compute_metrics(y_true, y_pred):
    mse = np.mean((y_true - y_pred) ** 2)
    mae = np.mean(np.abs(y_true - y_pred))
    return {'MSE': mse, 'MAE': mae}

Common pitfalls

• Context length and prediction horizon are fixed at 512 and 96 respectively across all datasets, which may not match dataset-specific optimal settings.
• Metrics are reported as raw numerical values without normalization, making cross-dataset comparison of absolute scores difficult.
• The paper does not specify train/val/test splits for the 16 benchmarks, relying on standard community splits.

Evidence (verbatim from paper)

In all experiments, the context length is fixed at 512 time steps, and the prediction horizon is set to 96. The evaluation metrics are Mean Squared Error (MSE) and Mean Absolute Error (MAE).

Citation

@misc{wu2026wavemoe,
  title={WaveMoE: A Wavelet-Enhanced Mixture-of-Experts Foundation Model for Time Series Forecasting},
  author={Wu et al. (2026)},
  year={2026},
  note={arXiv:2604.10544}
}

• arXiv: 2604.10544