Model Improve

Commodity Model Improvement Workflow

Complete model development pipeline for the Quantamental commodity trading system. Produces production-ready logistic regression models on remodeled futures contracts with full validation.

8 phases. Execute in order. Do not skip phases.

Before You Start

Read these files to understand the current state:

1. production/{commodity}_production_std.py — current model
2. PRODUCTION_MANIFEST.json — baseline metrics
3. production/shared_constants.py — lot sizes, TC, tickers
4. production/data_loader.py — available data sources and PUBLICATION_LAGS

Identify the commodity's economic identity before engineering features. Every commodity has a unique supply/demand structure:

• Precious metals (gold, silver, platinum): real rates, USD, safe haven, jewelry, industrial
• Base metals (copper, aluminium): China demand, industrial cycle, housing
• Energy (crude, natgas, heating oil): OPEC, storage, weather, refining margins
• Agriculture (corn, wheat, soybeans): weather, planting, stocks-to-use, ethanol
• Softs (coffee, sugar, cocoa): EM supply, weather, currency
• Livestock (hogs, cattle): herd cycle, feed costs, slaughter

Phase 1: Audit Current Model

cd production && python {commodity}_production_std.py
python ../scripts/preflight.py {commodity}

Document: current features, performance (CV/OOS/min Sharpe), coverage gaps, data sources.

Phase 2: Data Source Audit

Non-negotiable. Every feature must use live, verified data.

Approved sources (0 lag, real-time or weekly)

Conditional sources (use only if lag acceptable)

Banned: IndexMundi (web scrape, no API, frequently months stale)

Verification: For each feature, confirm last data date is < 4 weeks old and publication lag is in PUBLICATION_LAGS.

COT timing: Test lag sensitivity (0w vs 1w). If min Sharpe drops >50% with 1-week lag, document the timing risk.

Phase 3: Exhaustive Feature Engineering

Build ~200+ candidates from ALL approved sources. Use pd.concat(dict, axis=1) to avoid DataFrame fragmentation.

Data loading

yf = load_yfinance()
fr = load_fred()
ct = load_cot()
commodity_price = yf[commodity].dropna()
idx = commodity_price.index
# Align all series with proper lags via align_with_lag()

Feature categories (cover ALL of these)

1. Own transforms: momentum (4/8/13/26w), z-score (26/52), risk-adjusted mom, acceleration, percentile rank
2. Related commodities: ratios z-scored, cross-commodity momentum
3. COT positioning: MM/PM z-scores, binary extremes, spec concentration (MM/OI), cross-commodity divergence (e.g., z(silver_mm) - z(copper_mm)), COT breadth
4. Energy/industrial: crack spreads (HO/crude, gas/crude), sector ETFs (XLE, XLI, XLB), solar (TAN)
5. Macro/rates: TIPS (level, diff, z, binary), yield curve (inverted, steepening, acceleration), breakevens, Fed funds (continuous diff + binary), CPI
6. Risk/credit: VIX (z26/z52, high/low, acceleration), HY/IG/BBB/TED spreads, MOVE, GVZ, OVX
7. FX/USD: DXY (z, momentum, acceleration), USD broad, USD/EM, EUR/USD, USD/JPY, AUD/USD
8. Liquidity: M2 growth (13w/26w), Fed balance sheet growth
9. Equity: SPY/QQQ/IWM momentum and z-scores
10. Multi-timeframe composites: trend strength count, macro tailwind count, industrial health count, metals breadth
11. Seasonal: commodity-specific demand/supply seasons
12. Binary regime indicators: real_rate_easing, inflation_elevated, m2_growing
13. Interactions (4 categories — the creative edge):
    • Curve x macro regime (e.g., YC inverted x BEI rising)
    • Crack spread x inventory/flow (e.g., HO crack high x OI rising)
    • Risk x growth/demand (e.g., VIX elevated x copper recovering)
    • Supply stress x USD/liquidity (e.g., producers covering x DXY weak)

Phase 4: Two-Stage Feature Selection

Stage 1: IC Screen

• Spearman IC with remodeled futures labels
• Require: |IC| >= 0.015, p < 0.15, sign stable (same sign in first/second half of sample)
• Take top 60 by |IC|

Stage 2: Greedy Forward Selection

• Walk-forward on remodeled futures contract using build_futures_model_target() + compute_futures_pnl()
• Add one feature at a time, keep only if min(CV_Sharpe, OOS_Sharpe) improves by >= 0.005
• Stop at max 15 features
• Use fillna(0) for features with shorter history

Phase 5: Optuna Optimization (300 trials)

# Search space
C: log_uniform(0.05, 3.0)
train_weeks: int(40, 130, step=13)
retrain_weeks: int(4, 30)
long_threshold: uniform(0.51, 0.60)
short_threshold: uniform(0.35, 0.49)
# + 4 binary short rule inclusion + conditional thresholds

# Objective: maximize min(CV_Sharpe, OOS_Sharpe)

After Optuna: Compare best trial against default parameter grid. If greedy selection defaults scored higher, discard Optuna params. Prefer round numbers (C=0.20 not C=0.2137). The parameter grid is the robustness proof, not Optuna's single best trial.

Phase 6: Validation Suite (ALL must pass)

Phase 7: Final Data Source Verification

After all validation, audit EVERY selected feature one more time:

1. Source is live and updating (< 4 weeks stale)
2. Publication lag correctly set in PUBLICATION_LAGS
3. Data available at pipeline time (Friday 6 PM ET)

If any feature fails: DROP IT, re-run Phases 4-6.

Phase 8: Production Script

Follow gold_production_std.py pattern with futures adapter:

from futures_model_adapter import (
    build_futures_model_target, build_futures_pnl_config, build_futures_results_frame)
from futures_pnl import compute_futures_pnl

Structure: load_data() → build_features() → run_backtest() → get_live_signal() → main()

Pre-commit:

python scripts/validate_outputs.py {commodity}
python scripts/preflight.py {commodity}

Deploy to production outputs:

export ALLOW_OFF_CYCLE_SCHEDULED_WRITE=1 SIGNAL_RUN_TYPE=scheduled
python production/{commodity}_production_std.py

Key Principles

1. Live data only. No stale sources, no missing publication lags.
2. Features from cache, targets from futures. Features are built from cached data (yfinance/FRED/COT parquets via load_commodity_data). Labels and PnL come from Bloomberg roll-adjusted futures data via build_futures_model_target() + compute_futures_pnl(). Never legacy compute_dollar_pnl.
3. Economic rationale first. Every feature needs a credible explanation. But don't force features that degrade walk-forward.
4. Parameter robustness over point optimization. The grid proves the plateau; Optuna finds a peak that may be noise.
5. fillna(0) for shorter history. The engine does this anyway. Features are neutral until their data starts.
6. Round numbers. C=0.20, retrain=13, threshold=0.53. Precise = overfit.
7. Test everything twice. Model params (125 grid) AND feature params (256+ grid).

Position Sizing Rules (NON-NEGOTIABLE)

All production scripts MUST use futures_pnl.py for position sizing and PnL. No inline sizing logic.

lots = floor(var_limit / (raw_price × lot_size × price_mult × weekly_vol × z_conf))
lots = max(lots, 1)  # minimum 1 lot when signal is active

Why no vol clip: The weekly rebalance is too slow to adapt during vol spikes. If caught on the wrong side at max lots during a vol explosion, the model cannot reduce exposure fast enough. Letting vol scale naturally ensures smaller positions precisely when being wrong is most expensive.