Forecasting Data Prep

Forecasting Data Prep

Use this skill before any forecasting-model skill and to prepare non-classification ordered time series for routing. The output should be a clean data contract, diagnostics, temporal splits, explicit leakage notes, and problem-summary.txt. Do not start downstream work until the readiness checklist passes or the remaining risks are accepted.

Default Workflow

1. Define the problem and inspect schema

   • State the task type, decision or expected output, dataset source, constraints, and success criteria before choosing a library.
   • Identify the timestamp column, target column, optional series ID columns, and candidate exogenous variables.
   • Prefer explicit user-provided column names. If inferring, state confidence and ambiguous alternatives.
   • Confirm the business meaning of one row: event, transaction, snapshot, aggregate period, or already-resampled observation.

2. Normalize time

   • Parse timestamps with an explicit format when known.
   • Use one canonical timezone. Prefer UTC for storage and convert only for reporting/calendar features.
   • Sort by series_id columns and timestamp. Require monotonic time within each series.
   • Decide whether timestamps represent period start, period end, or instant observations.

3. Validate grain and frequency

   • Infer frequency per series where possible; compare with the required modeling frequency.
   • Check for mixed granularities, irregular intervals, daylight-saving artifacts, and partial periods.
   • If resampling is needed, define aggregation rules per column: target sum/mean/last as appropriate, known-future flags by max/any, prices by last/mean, static fields by first after consistency checks.

4. Audit data quality

   • Detect duplicate (series_id, timestamp) keys and resolve them before splitting.
   • Quantify missing timestamps, missing target values, and missing exogenous values separately.
   • Treat temporal gaps differently from null targets: a missing timestamp means the row is absent; a null target means the row exists but target is unknown.
   • Flag outliers with robust rules, then decide whether to keep, cap, correct, or annotate. Do not remove outliers automatically.

5. Handle panels and exogenous variables

   • For panel data, validate each series independently: start/end dates, frequency, length, missingness, and enough history for the requested horizon.
   • Classify exogenous variables as:
     • known_future: calendar, holidays, scheduled promotions, planned prices, contractual capacity.
     • observed_past: weather observations, realized demand drivers, lagged operations, sensor values.
     • static: store, item, geography, category, segment.
     • unknown: requires user confirmation before modeling.
   • Never use a variable at prediction time unless it will truly be available for the full forecast horizon.

6. Split before fitting transformations

   • Create train/validation/test by timestamp cutoffs, not random split.
   • Fit imputers, scalers, encoders, target transforms, anomaly thresholds, feature selection, and PCA only on train.
   • Apply fitted transforms forward to validation/test. Refit only inside a documented rolling-origin or expanding-window backtest fold.

7. Create features without leakage

   • Create lag and rolling features with shift(1) or a horizon-aware offset before rolling/expanding calculations.
   • For direct multi-horizon models, ensure each feature uses only data available at the forecast creation timestamp.
   • Do not compute global statistics, encodings, interpolation, or calendar/event joins using validation/test targets.

8. Set validation and metrics

   • Define forecast_horizon, optional gap, step size, number of folds, and minimum training window.
   • Use holdout test only once for final evaluation. Use validation/backtesting for model selection.
   • Recommended metrics: MAE/RMSE for scale-dependent error, MASE/RMSSE for cross-series comparison, sMAPE/WAPE when appropriate, pinball loss or CRPS for probabilistic forecasts, and bias/mean error for systematic over/under-forecasting.

9. Produce minimum diagnostics

   • Plot target over time for representative series and aggregate total if panel data.
   • Plot missingness/gaps by time and by series.
   • Plot train/validation/test cutoffs on the target chart.
   • Plot residual-like anomalies only after defining a leakage-safe baseline or robust threshold from train.

10. Write the routing artifact

• Create problem-summary.txt in the requested output directory, or in the current workspace when none is specified.
• Use the exact field names below, one field per line. Write unknown instead of guessing.
• Finalize the file after the audit so it reflects validation blockers and leakage risks, not only the initial request.

Deterministic Audit Script

For tabular files, run the bundled audit before manual cleanup. It requires pandas in the active Python environment:

python forecasting-data-prep/scripts/forecasting_data_audit.py data.csv \
  --time-col ds \
  --target-col y \
  --id-cols unique_id \
  --freq D \
  --horizon 14

The script emits JSON diagnostics for inferred columns, duplicates, gaps, missingness, frequency, panel length, exogenous candidates, outlier counts, and split readiness. Treat it as a first pass; domain decisions still require human or task context.

Problem Summary Artifact

Write this UTF-8 text contract without embedding raw data rows or secrets:

PROBLEM SUMMARY
task_type: forecasting | pattern_discovery | feature_aggregation | changepoint | anomaly | unknown
problem: <problem to solve>
decision_or_output: <required result and success criteria>
dataset: <path or source, format, size if known>
observation_unit: <meaning of one row or timestamp>
time: <timestamp field, frequency, timezone, semantics>
target_or_label: <target, signal, or unknown>
series_or_sample_id: <ID fields or none>
features: <observed variables and roles>
exogenous_variables: <known-future, observed-past, static, excluded, or none>
data_structure: <univariate, multivariate, panel, regular or irregular>
horizon_or_window: <forecast horizon, analysis window, or unknown>
validation_plan: <temporal split, folds, metrics>
leakage_risks: <identified risks or none found>
constraints: <runtime, library, interpretability, deployment, or none>
readiness: ready | blocked
open_questions: <unresolved facts or none>

References

• Read references/data-contract.md when column roles, grain, panel structure, or resampling rules are unclear.
• Read references/validation-and-leakage.md when designing temporal splits, rolling-origin backtests, feature windows, or transformation pipelines.
• Read references/library-selection.md when choosing preparation libraries and dependencies.

Ready for Modeling Checklist

• Timestamp and target columns are explicit and parsed correctly.
• Data is sorted and unique at (series_id, timestamp) or (timestamp) for single-series data.
• Timezone, timestamp semantics, frequency, and period boundaries are documented.
• Gaps, duplicates, missing targets, and outliers are quantified and handled or accepted.
• Each panel series has enough history for the horizon, validation plan, and required lags.
• Exogenous variables are classified as known future, observed past, static, or excluded.
• Train/validation/test or backtest folds use temporal cutoffs with any required gap.
• All transformations are fit on train only and applied forward.
• Lag, rolling, expanding, interpolation, and encodings use only past information.
• Metrics match the business objective and support comparison across series when needed.
• problem-summary.txt exists and matches the validated data contract.