qhjqhj00 window-wise-complexity
Accuracy Law for the Future of Deep Time Series Forecasting — Wang et al. (2025) (arXiv:2510.02729, 2025)
What this evaluates
Quantifies the intrinsic predictability of time series data by measuring window-wise pattern complexity in the frequency domain. It establishes a data-driven performance lower bound for forecasting models and identifies whether standard benchmarks have reached saturation.
Datasets
- ETT — total ?; splits: test (-1)
- Electricity — total ?; splits: test (-1)
- Weather — total ?; splits: test (-1)
- Exchange-Rate — total ?; splits: test (-1)
- Traffic — total ?; splits: test (-1)
Metrics
window-wise complexity(primary) — range: other- Measures pattern complexity in the frequency domain over sliding windows of length (P+F), combining both past observation and forecasting horizon windows. Computed as the average amplitude spectrum difference across windows.
Pearson coefficient— range: [-1, 1]- Standard linear correlation coefficient between the computed window-wise complexity and the best achieved forecasting error/performance on each series.
Input / output format
Input: Univariate time series segments from benchmarks, partitioned into a past observation window (P) and a forecasting horizon (F).
Output: Scalar complexity score per series; best forecasting error achieved by deep models compared against the estimated lower bound.
Scoring recipe
def compute_window_wise_complexity(series, P, F):
windows = [series[i:i+P+F] for i in range(0, len(series)-P-F+1)]
amplitudes = [abs(fft(w)) for w in windows]
return mean([norm(a) for a in amplitudes])
def check_saturation(series, model_error, complexity):
lower_bound = estimate_lower_bound(complexity)
return model_error < lower_bound
Common pitfalls
- Relying on series-wise metrics (ADF, ForeCA, ACF half-life) instead of window-wise complexity yields weak correlation with actual forecasting performance.
- Assuming benchmarks are still challenging when performance gains are marginal; the metric reveals many are already saturated.
- Evaluating large models on in-domain data fails to expose pretraining-complexity imbalances and generalization limits.
Evidence (verbatim from paper)
Despite widespread adoption, we find that these series-wise predictability metrics do not produce a significant correlation with final forecasting performance in terms of the Pearson coefficient... the frequency domain measure significantly outperforms the time-domain DTW... nearly all points in ETT, Electricity, Weather, and Exchange-Rate benchmarks fall below the estimated performance lower bound, indicating that these datasets have likely reached their predictability limits.
Citation
@misc{wang2025accuracylaw,
title={Accuracy Law for the Future of Deep Time Series Forecasting},
author={Wang et al. (2025)},
year={2025},
note={arXiv:2510.02729}
}
- arXiv: 2510.02729