Feature Engineering

Feature Engineering

References

See references/rolling-stats-reference.md for the complete RollingFeatures constructor, all 9 available statistics, feature name generation formula, window behavior, and kwargs_stats usage.

When to Use This Skill

• Forecast accuracy has plateaued and you suspect better features would help
• User asks about "exogenous variables", "external regressors", or "feature creation"
• Time series has calendar patterns (hourly, weekly, seasonal) not yet captured
• Raw datetime index is used directly instead of engineered features
• User mentions feature_engine, RollingFeatures, or skforecast preprocessing
• Energy/transport/outdoor domain where sunlight hours may be predictive

When NOT to Use

• Tabular ML (non-time-series): Use a general feature engineering skill instead
• Deep learning forecasters (RNNs, Transformers): These learn features internally; manual engineering adds less value
• Feature selection/importance: This skill covers creation, not selection — use model-based selection after creating features
• Data cleaning/imputation: Handle missing values and outliers before feature engineering

Overview

Calendar Features with feature_engine

Manual extraction (pandas)

import pandas as pd

# Data must have a DatetimeIndex with frequency set
data = data.asfreq('h')

data['year'] = data.index.year
data['month'] = data.index.month
data['day_of_week'] = data.index.dayofweek
data['hour'] = data.index.hour

Automated extraction (DatetimeFeatures)

from feature_engine.datetime import DatetimeFeatures

features_to_extract = ['month', 'week', 'day_of_week', 'hour']
calendar_transformer = DatetimeFeatures(
    variables           = 'index',
    features_to_extract = features_to_extract,
    drop_original       = True,
)

calendar_features = calendar_transformer.fit_transform(data)

DatetimeFeatures is sklearn-compatible and can be passed directly as transformer_exog in skforecast forecasters.

Cyclical Encoding

Cyclical features (hour, day_of_week, month) should NOT be treated as linear integers — hour 23 is only 1 hour from hour 0. Use sin/cos encoding to preserve the cyclical relationship.

from feature_engine.datetime import DatetimeFeatures
from feature_engine.creation import CyclicalFeatures

# Step 1: Extract calendar features
features_to_extract = ['month', 'week', 'day_of_week', 'hour']
calendar_transformer = DatetimeFeatures(
    variables           = 'index',
    features_to_extract = features_to_extract,
    drop_original       = True,
)
calendar_features = calendar_transformer.fit_transform(data)

# Step 2: Encode as cyclical (sin/cos)
features_to_encode = ['month', 'week', 'day_of_week', 'hour']
max_values = {
    'month': 12,
    'week': 52,
    'day_of_week': 7,
    'hour': 24,
}
cyclical_encoder = CyclicalFeatures(
    variables     = features_to_encode,
    max_values    = max_values,
    drop_original = True,
)
exog_calendar = cyclical_encoder.fit_transform(calendar_features)
# Produces columns: month_sin, month_cos, week_sin, week_cos, ...

Sunlight Features

Sunrise/sunset times can be powerful features for energy, transport, or activity-related series.

from astral.sun import sun
from astral import LocationInfo

location = LocationInfo('Washington, D.C.', 'USA')
sunrise_hour = [sun(location.observer, date=date)['sunrise'] for date in data.index]
sunset_hour = [sun(location.observer, date=date)['sunset'] for date in data.index]

# Round to the nearest hour
sunrise_hour = pd.Series(sunrise_hour, index=data.index).dt.round('h').dt.hour
sunset_hour = pd.Series(sunset_hour, index=data.index).dt.round('h').dt.hour

sun_light_features = pd.DataFrame({
    'sunrise_hour': sunrise_hour,
    'sunset_hour': sunset_hour,
})
sun_light_features['daylight_hours'] = (
    sun_light_features['sunset_hour'] - sun_light_features['sunrise_hour']
)

Rolling Features (Window Statistics)

from skforecast.preprocessing import RollingFeatures
from skforecast.recursive import ForecasterRecursive
from lightgbm import LGBMRegressor

# Single window size for all stats
rolling = RollingFeatures(
    stats=['mean', 'std', 'min', 'max'],
    window_sizes=7,  # int applies same window to all stats
)

# Different window sizes per statistic
rolling = RollingFeatures(
    stats=['mean', 'std', 'min', 'max'],
    window_sizes=[7, 7, 14, 14],  # Must match length of stats
)

# Multiple RollingFeatures objects
rolling_short = RollingFeatures(stats=['mean', 'std'], window_sizes=7)
rolling_long = RollingFeatures(stats=['mean', 'std'], window_sizes=30)

forecaster = ForecasterRecursive(
    estimator=LGBMRegressor(),
    lags=24,
    window_features=[rolling_short, rolling_long],  # List of RollingFeatures
)

Available Rolling Statistics

Standard: 'mean', 'std', 'min', 'max', 'sum', 'median', 'ratio_min_max', 'coef_variation'

Exponential weighted: 'ewm' — requires kwargs_stats:

rolling = RollingFeatures(
    stats=['ewm'],
    window_sizes=7,
    kwargs_stats={'ewm': {'alpha': 0.3}},
)

Differencing (Non-Stationary Series)

# Built-in — forecaster handles differencing and inverse transform automatically
forecaster = ForecasterRecursive(
    estimator=LGBMRegressor(),
    lags=24,
    differentiation=1,  # First-order differencing (removes linear trend)
    # differentiation=2,  # Second-order (removes quadratic trend)
)
forecaster.fit(y=y_train)
predictions = forecaster.predict(steps=10)  # Auto inverse-transformed

Data Transformers (Scaling)

from sklearn.preprocessing import StandardScaler, MinMaxScaler

# Scale target variable — transformer applied automatically during fit/predict
forecaster = ForecasterRecursive(
    estimator=LGBMRegressor(),
    lags=24,
    transformer_y=StandardScaler(),
    transformer_exog=StandardScaler(),
)

# For multi-series, different transformers per series
from skforecast.recursive import ForecasterRecursiveMultiSeries

forecaster = ForecasterRecursiveMultiSeries(
    estimator=LGBMRegressor(),
    lags=24,
    transformer_series={
        'series_1': StandardScaler(),
        'series_2': MinMaxScaler(),
    },
)

Combining Features — Full Example

import pandas as pd
from feature_engine.datetime import DatetimeFeatures
from feature_engine.creation import CyclicalFeatures
from skforecast.preprocessing import RollingFeatures
from skforecast.recursive import ForecasterRecursive
from sklearn.preprocessing import StandardScaler
from lightgbm import LGBMRegressor

# 1. Calendar features with cyclical encoding
calendar_transformer = DatetimeFeatures(
    variables='index',
    features_to_extract=['month', 'day_of_week', 'hour'],
    drop_original=True,
)
cyclical_encoder = CyclicalFeatures(
    variables=['month', 'day_of_week', 'hour'],
    max_values={'month': 12, 'day_of_week': 7, 'hour': 24},
    drop_original=True,
)
exog_calendar = cyclical_encoder.fit_transform(
    calendar_transformer.fit_transform(data)
)

# 2. Combine with other exogenous variables
exog = pd.concat([exog_external, exog_calendar], axis=1)

# 3. Rolling features + lags + differencing
rolling = RollingFeatures(stats=['mean', 'std'], window_sizes=[7, 14])

forecaster = ForecasterRecursive(
    estimator=LGBMRegressor(),
    lags=[1, 2, 3, 7, 14, 24],
    window_features=rolling,
    transformer_y=StandardScaler(),
    differentiation=1,
)
forecaster.fit(y=y_train, exog=exog.loc[y_train.index])
predictions = forecaster.predict(steps=10, exog=exog.loc[forecast_index])

Common Mistakes

1. Not encoding cyclical features: Using raw integers for hour/month/day_of_week loses the cyclical relationship (hour 23 appears far from hour 0). Always use sin/cos encoding.
2. Forgetting frequency on index: Calendar transformers require DatetimeIndex with frequency set (data.asfreq('h')).
3. Not covering forecast horizon with exog: Calendar features for predict() must include future dates covering the entire forecast horizon.
4. Over-engineering features: Start with lags only, then add rolling features and calendar features incrementally. Validate each addition with backtesting.