Few Shot TS Gen Eval

few-shot-ts-gen-eval

Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach — Gonen et al. (2025) (arXiv:2505.20446, 2025)

What this evaluates

Evaluates the ability of a generative model to produce high-fidelity time series data under extreme data scarcity (few-shot fine-tuning). It probes cross-domain generalization and robustness to varying sequence lengths and channel dimensions by comparing generated samples against real test data.

Datasets

• ECG200 — total ?; splits: test (-1)
• ETTh2 — total ?; splits: test (-1)
• ETTm1 — total ?; splits: test (-1)
• ETTm2 — total ?; splits: test (-1)
• ILI — total ?; splits: test (-1)
• Weather — total ?; splits: test (-1)
• Synthetic sine wave — total ?; splits: test (-1)

Metrics

• Discriminative Score (Disc.) — range: other (lower is better)
  • Measures how well a discriminator distinguishes real time series from generated ones. Lower values indicate generated samples are more realistic.
• Predictive Score (Pred.) — range: other (lower is better)
  • Evaluates the predictive accuracy of a surrogate model trained on real data when applied to generated data. Lower values indicate better predictive fidelity.
• contextFID (c-FID) (primary) — range: other (lower is better)
  • Fréchet Inception Distance adapted for time series, measuring the distance between feature distributions of real and generated sequences. Lower values indicate higher sample quality and diversity.

Input / output format

Input: Time series sequences with varying lengths and channel dimensions, provided as a few-shot fine-tuning subset (e.g., 5%, 10%, 15%, or fixed counts like #10, #25, #50).

Output: Generated time series sequences matching the input length and channel dimensions.

Scoring recipe

# For each dataset and few-shot subset size:
# 1. Fine-tune pre-trained model on the subset.
# 2. Generate N samples from the fine-tuned model.
# 3. Compute Discriminative Score (Disc.) between generated and real test set.
# 4. Compute Predictive Score (Pred.) between generated and real test set.
# 5. Compute contextFID (c-FID) between generated and real test set.
# Return average of Disc., Pred., c-FID across datasets/subsets. Lower is better.

Common pitfalls

• All three metrics (Disc., Pred., c-FID) are lower-is-better, contrary to typical accuracy metrics.
• The benchmark uses both percentage-based (5%, 10%, 15%) and absolute count-based (#10, #25, #50) few-shot subset sizes, which must be tracked separately.
• Pre-training without dataset token conditioning leads to domain ambiguity and poor performance if fine-tuning is skipped.

Evidence (verbatim from paper)

We report the averaged Discriminative Score (Disc.), Predictive Score (Pred.), and contextFID (c-FID) in Tab. 1. Across all subset sizes and evaluation metrics, our method consistently outperforms the baselines, demonstrating strong performance in both percentage-based and count-based few-shot settings.

Citation

@misc{gonen2025timeseriesgeneration,
  title={Time Series Generation Under Data Scarcity: A Unified Generative Modeling Approach},
  author={Gonen et al. (2025)},
  year={2025},
  note={arXiv:2505.20446}
}

• arXiv: 2505.20446