Tsregression Eval

tsregression-eval

Monash University, UEA, UCR Time Series Extrinsic Regression Archive — Tan et al. (2020) (arXiv:2006.10996, 2020)

What this evaluates

Evaluates models on Time Series Extrinsic Regression (TSER), where the goal is to predict a single continuous scalar value from multivariate time series inputs of varying lengths and dimensions. It probes the model's ability to handle irregular time series, missing values, and diverse domain-specific patterns without imputation.

Datasets

• Monash TSER Archive — total ?; splits: train (-1), test (-1); repo https://github.com/ChangWeiTan/TSRegression

Metrics

• R2 (primary) — range: [0, 1]
  • Coefficient of determination, measuring the proportion of variance in the target variable explained by the model. Formula: 1 - (SS_res / SS_tot), where SS_res is the sum of squared residuals and SS_tot is the total sum of squares.

Input / output format

Input: Multivariate time series sequences of varying lengths (24 to 4000 time steps) and dimensions (1 to 9+ features). Missing values are preserved as '?' symbols per the .ts convention and are not imputed.

Output: A single continuous scalar value representing the regression target (e.g., energy consumption in kWh, pollutant concentration, moisture content).

Scoring recipe

def compute_r2(y_true, y_pred):
    ss_res = np.sum((y_true - y_pred) ** 2)
    ss_tot = np.sum((y_true - np.mean(y_true)) ** 2)
    return 1 - (ss_res / ss_tot)

Common pitfalls

• Missing values are explicitly NOT imputed and must be handled natively or via the '?' placeholder.
• Datasets use fixed, predefined train/test splits rather than cross-validation, so results are not directly comparable across papers using different splits.
• Time series lengths and dimensions vary significantly across datasets, requiring models to be length-agnostic or dynamically sized.

Evidence (verbatim from paper)

For fair comparison of regression algorithms, we split the datasets in the archive into predefined train and test sets which will be outlined in the following sections. ... Although the results showed that GBM performed the best, it was only able to explain 57% of the variance (R2) in the test set.

Citation

@misc{tan2020monash,
  title={Monash University, UEA, UCR Time Series Extrinsic Regression Archive},
  author={Tan et al. (2020)},
  year={2020},
  note={arXiv:2006.10996}
}

• arXiv: 2006.10996