blast-forecasting-eval
BLAST: Balanced Sampling Time Series Corpus for Universal Forecasting Models — Zezhi Shao et al. (arXiv:2505.17871, 2025)
What this evaluates
Evaluates the zero-shot forecasting capability of universal time series models across multiple domains and prediction horizons. It probes how well pre-trained models generalize to unseen datasets and measures predictive accuracy using standard error metrics.
Datasets
- ETTh1, ETTh2, ETTm1, ETTm2, Weather, GlobalTemp — total ?; splits: test (-1)
- GIFT-Eval — total 43; splits: test (43)
Metrics
MSE (primary) — range: other
- Normalized Mean Squared Error between predicted and actual values.
MAE (primary) — range: other
- Mean Absolute Error between predicted and actual values.
MASE — range: other
- Mean Absolute Scaled Error relative to a naive baseline forecast, following the GIFT-Eval protocol.
Input / output format
Input: Historical time series sequences provided in a zero-shot setting, without task-specific fine-tuning.
Output: Predicted future values for specified forecasting horizons (96, 192, 336, or 720 time steps).
Scoring recipe
def compute_metrics(preds, targets, horizon):
mse = np.mean((preds - targets) ** 2)
mae = np.mean(np.abs(preds - targets))
naive_pred = targets[:-horizon]
mase = np.mean(np.abs(preds - targets)) / np.mean(np.abs(targets[horizon:] - naive_pred))
return {'MSE': mse, 'MAE': mae, 'MASE': mase}
Common pitfalls
- Data leakage: must filter out any time series segments already present in the pre-training corpus (Time-300B, LOTSA, BLAST) before evaluation.
- Horizon-specific reporting: metrics must be evaluated and reported separately for each prediction length (96, 192, 336, 720).
- Zero-shot constraint: models are evaluated without task-specific fine-tuning; results should not reflect adapter or prompt-tuning gains unless explicitly stated.
Evidence (verbatim from paper)
We report the normalized Mean Squared Error (MSE) and Mean Absolute Error (MAE). For the GIFT-Eval benchmark*(Aksu et al., [2024])*, we filtered out data already included in Time-300B (TimeMoE pre-training data), LOTSA (MOIRAI pre-training data), and BLAST, and strictly followed its evaluation pipeline. We report the Mean Absolute Scaled Error (MASE).
Citation
@misc{shao2025blast,
title={BLAST: Balanced Sampling Time Series Corpus for Universal Forecasting Models},
author={Zezhi Shao et al.},
year={2025},
note={arXiv:2505.17871}
}
1---2name: blast-forecasting-eval3description: Evaluates the zero-shot forecasting capability of universal time series models across multiple domains and prediction horizons. It probes how well pre-trained models generalize to unseen datasets and measures predictive accuracy using standard error metrics. Use when the user wants to benchmark on ETTh1, ETTh2, ETTm1, ETTm2, Weather, GlobalTemp, GIFT-Eval, or asks about evaluating this task. Reports MSE, MAE.4---56# blast-forecasting-eval78> BLAST: Balanced Sampling Time Series Corpus for Universal Forecasting Models — Zezhi Shao et al. (arXiv:2505.17871, 2025)910## What this evaluates1112Evaluates the zero-shot forecasting capability of universal time series models across multiple domains and prediction horizons. It probes how well pre-trained models generalize to unseen datasets and measures predictive accuracy using standard error metrics.1314## Datasets1516- **ETTh1, ETTh2, ETTm1, ETTm2, Weather, GlobalTemp** — total ?; splits: test (-1)17- **GIFT-Eval** — total 43; splits: test (43)1819## Metrics2021- `MSE` **(primary)** — range: other22 - Normalized Mean Squared Error between predicted and actual values.23- `MAE` **(primary)** — range: other24 - Mean Absolute Error between predicted and actual values.25- `MASE` — range: other26 - Mean Absolute Scaled Error relative to a naive baseline forecast, following the GIFT-Eval protocol.2728## Input / output format2930**Input**: Historical time series sequences provided in a zero-shot setting, without task-specific fine-tuning.3132**Output**: Predicted future values for specified forecasting horizons (96, 192, 336, or 720 time steps).3334## Scoring recipe3536```python37def compute_metrics(preds, targets, horizon):38 mse = np.mean((preds - targets) ** 2)39 mae = np.mean(np.abs(preds - targets))40 naive_pred = targets[:-horizon]41 mase = np.mean(np.abs(preds - targets)) / np.mean(np.abs(targets[horizon:] - naive_pred))42 return {'MSE': mse, 'MAE': mae, 'MASE': mase}43```4445## Common pitfalls4647- Data leakage: must filter out any time series segments already present in the pre-training corpus (Time-300B, LOTSA, BLAST) before evaluation.48- Horizon-specific reporting: metrics must be evaluated and reported separately for each prediction length (96, 192, 336, 720).49- Zero-shot constraint: models are evaluated without task-specific fine-tuning; results should not reflect adapter or prompt-tuning gains unless explicitly stated.5051## Evidence (verbatim from paper)5253> We report the normalized Mean Squared Error (MSE) and Mean Absolute Error (MAE). For the GIFT-Eval benchmark*(Aksu et al., [2024])*, we filtered out data already included in Time-300B (TimeMoE pre-training data), LOTSA (MOIRAI pre-training data), and BLAST, and strictly followed its evaluation pipeline. We report the Mean Absolute Scaled Error (MASE).5455## Citation5657```bibtex58@misc{shao2025blast,59 title={BLAST: Balanced Sampling Time Series Corpus for Universal Forecasting Models},60 author={Zezhi Shao et al.},61 year={2025},62 note={arXiv:2505.17871}63}64```6566- arXiv: 2505.17871
qhjqhj00