# Pybroker Model Trainer

> Register, train, wire, and debug machine learning models for PyBroker backtests using the bundled PyBroker wiki references generated from the local docs. Use when an agent needs to register a model with pybroker.model, write train_fn/predict_fn code for scikit-learn, XGBoost, LightGBM, CatBoost, PyTorch, Keras, statsmodels (ARIMA/SARIMAX), or arch models, build ensembles or regime models, run walkforward analysis, build time-series or lagged-feature models, train pooled multi-symbol models, load pretrained models, cache data and trained models, or prevent lookahead leakage in model-driven backtests.

- Skill: `edtechre/pybroker-model-trainer` (Agent Skill, multi-file: 13 files)
- Install (CLI): `npx skillmds@latest add edtechre/pybroker-model-trainer`
- Raw SKILL.md: https://api.skillmd.com/api/skills/edtechre/pybroker-model-trainer/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: edtechre (https://skillmd.com/u/edtechre)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/edtechre/pybroker-model-trainer

---


# PyBroker Model Trainer

## Overview

Wire machine learning models into PyBroker backtests by registering training and prediction functions with `pybroker.model`, feeding them indicator features, and evaluating them with walkforward analysis while keeping the train/test flow free of lookahead leakage. Covers per-symbol, pooled multi-symbol, per-bar time-series, lagged-feature, and pretrained models across common libraries such as scikit-learn, XGBoost, and arch.

## Workflow

1. Extract the modeling spec: library, prediction target and horizon, features (indicators, lagged columns, custom columns), per-symbol vs pooled training, vectorized vs per-bar prediction, walkforward windows and lookahead, caching, and whether the model is trainable or pretrained.
2. Ask only for missing blockers. If details are absent but noncritical, make conservative assumptions and state them in the final answer or code comments.
3. Read `references/wiki-index.md` to choose the smallest relevant wiki page. For nontrivial model work, also read `references/model-training-patterns.md`.
4. Build a complete runnable model surface:
   - define feature indicators with `pybroker.indicator` or built-ins, and register any non-OHLCV data columns with `pybroker.register_columns`
   - write a `train_fn` that builds the target from train data only and returns the model, or `(model, input_cols)` to pin prediction columns
   - register the model with `pybroker.model(name, train_fn, ...)`, choosing `indicators`, `lags`/`lag_cols`, `per_bar`, `pooled`, `pretrained`, `input_data_fn`, and `predict_fn` as needed
   - consume predictions in an execution function via `ctx.preds(name)` and pass the model source to `Strategy.add_execution(models=...)`
   - run `strategy.walkforward(windows, train_size, lookahead)` for evaluation, or `backtest(train_size=...)` for a single train/test pass
5. Validate the produced code as far as the environment allows. At minimum, run syntax checks for created Python files. Run a small local-data walkforward when the repo and data make that practical.

## Implementation Rules

- Treat PyBroker as a backtesting framework, not a source of financial advice. Make modeling assumptions explicit and avoid performance claims that are not supported by the produced backtest, including model fit metrics.
- Use completed historical bar data only. Do not use future prices, future indicator values, or shuffled time series outside the supported train-split shuffle. An indicator value at bar `i` may depend only on inputs at index `i` and earlier: no backward shifts such as `shift(-1)` outside the sanctioned `train_fn` target, and no negative indexing into full-length arrays inside indicator functions (a negative index silently wraps to the end of the series — the future).
- Set `lookahead` to the number of bars ahead of the prediction target (default `1` for next-bar targets). Walkforward holds out `lookahead` bars between each train and test split, so an understated value leaks train-adjacent bars into testing.
- When registering with `lags`, the current bar's value is the first feature of each lag block, so the training target must be the next bar's value (for example `fit(lag_train[:-1], target[1:])`). The training `fn` must accept `lag_train` and `lag_test` keyword arguments.
- Keep feature data out-of-band: never widen or mutate the input DataFrame. Work on a `.copy()` inside `train_fn` when adding a target column.
- An `input_data_fn` must return exactly one row per bar. A vectorized `predict_fn` must return one prediction per input row; for classifiers, slice a single `predict_proba` column. With `per_bar=True`, `predict_fn` is required, receives rows up to and including the current bar, must return a scalar, and cannot be combined with `pooled=True`.
- A pooled `train_fn` receives a sorted `symbols` tuple and combined frames with a `symbol` column. Build targets with per-symbol operations such as `groupby("symbol")[col].shift(-1)` so labels never cross a symbol boundary, and return `(model, input_cols)` to keep `symbol` out of model input.
- To train a model on a longer time interval, bind it with `model_source.intervals("weekly")` when passing it to `add_execution(models=...)`; binding is exhaustive, so the base-timeframe model is trained only when `"base"` is included (e.g. `model_source.intervals("base", "weekly")`), and the bound interval is available through `ctx.interval` without declaring it in `intervals=` (which provides bars only). For interval-bound models, `lookahead` is measured in that interval's compressed bars, and predictions are read with `ctx.interval("...").preds(name)`. `timeframe=` is then required on `backtest`/`walkforward`.
- `strategy.optimize` supports pretrained models only. Tune trainable models inside `train_fn` with a search over the train window, or compare registrations across walkforward runs; `pybroker.hyperparam` is for strategy-level parameters.
- Fit scalers, encoders, and any early-stopping validation splits on train data only.
- Never use pandas to implement indicator or execution logic: write indicators as vectorized NumPy over `BarData` arrays (Numba `@njit` for explicit loops) and read `ctx.*` NumPy arrays in execution functions — no `pd.Series`/`pd.DataFrame` construction and no `.rolling`/`.ewm`/`.shift`/`.apply` in either. Pandas belongs only at the `train_fn`/`input_data_fn` boundary where PyBroker hands you DataFrames; building the target there with `shift(-1)` on a `.copy()` stays sanctioned.
- Enable caching while iterating: `pybroker.enable_data_source_cache(name)` to skip refetching data, or `pybroker.enable_caches(name)` to also cache indicators and trained models.
- Call `pybroker.disable_progress_bar()` in agent-run scripts; progress bar output floods AI token context.
- Orders fill at `PriceType.MIDDLE` — the midpoint of the low and high of the *execution* bar, which under the default `buy_delay`/`sell_delay` of `1` is the bar after the prediction, so `PriceType.CLOSE` means the next bar's close. Override with `ctx.buy_fill_price` / `ctx.sell_fill_price`, which take a `PriceType` (`OPEN`, `HIGH`, `LOW`, `CLOSE`, `MIDDLE`, `AVERAGE`), a number, or a `(symbol, bar_data)` callable, and read back as `None` rather than `MIDDLE` until set. A limit price only *gates* the fill: the order still fills at the fill price, never at the limit.
- `StrategyConfig.exit_on_last_bar` defaults to `False`, which leaves any position still open when the data ends out of `trade_count`, `win_rate`, `total_pnl` and every other trade-level metric, with its P&L stranded in `unrealized_pnl`. Set `exit_on_last_bar=True` whenever trade statistics are reported; in `walkforward` the liquidation fires only on each symbol's true final bar, never at window boundaries.
- `calc_bootstrap` is a `walkforward`/`backtest` parameter defaulting to `False`, not a `StrategyConfig` field, and it is the natural companion to walkforward analysis: it puts confidence intervals around a model's out-of-sample edge instead of a single point estimate. Pass `calc_bootstrap=True` to populate `result.bootstrap` with `conf_intervals` (BCa — bias corrected and accelerated — intervals for profit factor and Sharpe; 6x2, MultiIndexed on `name` then `conf`, columns `lower`/`upper`) and `drawdown_conf` (percentile bounds on max drawdown; 4x2, indexed on `conf`, columns `amount`/`percent`). A profit factor interval whose `lower` sits below `1` means the edge is not distinguishable from noise. It leaves `metrics_df` unchanged, stays `None` under `train_only=True`, costs roughly `bars x StrategyConfig.bootstrap_samples` (default `10_000`) once for the whole run rather than per window, and needs `StrategyConfig.bars_per_year` or the Sharpe intervals are per-bar rather than annualized.
- Report `result.metrics_df` as the human-readable summary. When structured output is needed (agent parsing, saved report files, downstream tools), use `result.to_json()` / `result.to_json_str()`: the default payload serializes metrics, trades, orders, and bootstrap capped at `max_rows=100` rows per table, `symbols=` filters to specific tickers, and `include=` opts into `portfolio`/`positions`/`metrics_df`/`signals`/`stops` (`signals` carries model predictions when `StrategyConfig(return_signals=True)`). Do not replace the `metrics_df` print outright: the default JSON payload (trades plus orders) is usually larger than the metrics table.
- On a Numba compilation or typing error in an `@njit` indicator, re-run once with the environment variable `NUMBA_DISABLE_JIT=1` to get a readable Python traceback, fix the error, then re-run with JIT enabled. Never leave JIT disabled in the final script.
- Self-test novel indicator logic for lookahead with the bump-last-bar check: recompute after changing only the final input bar and assert every earlier output is unchanged.
- Guard lookbacks with `ctx.bars` or `warmup`, and set at most one order side per symbol per bar.
- Use `ctx.calc_target_shares(target_size)` for allocation-based sizing. Use fixed `ctx.buy_shares` or `ctx.sell_shares` only when the user asks for fixed share sizing.
- Check `ctx.long_pos()` or `ctx.short_pos()` before entering or exiting positions. Use `ctx.sell_all_shares()` and `ctx.cover_all_shares()` for full exits.
- Set entry-time stops on the same bar as the entry order: `hold_bars`, `stop_loss_pct`, `stop_profit_pct`, or `stop_trailing_pct`.
- Rank by model score with `ctx.long_score` / `ctx.short_score` and cap positions with `strategy.set_max_long_positions(n)` / `set_max_short_positions(n)`; the `StrategyConfig` fields of the same names are deprecated. For score-driven rotation, `strategy.enable_rotation(worst_rank_held=...)` makes scores drive all trading and ignores order fields set in execution functions.
- Use `strategy.set_before_exec` or `strategy.set_after_exec` for cross-symbol portfolio logic instead of hiding global state inside a per-symbol execution function.
- Optional packages are not PyBroker dependencies: name the required `pip install` for every model or data-source library the script imports (for example `pip install yfinance scikit-learn` for the template) and never assume one is importable. When the network or a data-source package is unavailable, validate with a tiny local DataFrame passed to `Strategy` instead.
- If exact API names, constructor parameters, or methods matter, read `references/api-public-surface.md`.
- For exact type signatures — `pybroker.model()` and `train_fn`/`predict_fn` parameter types in `references/pybroker_model.pyi`, `ExecContext` prediction access and its writable order/stop attributes in `references/pybroker_context.pyi` — read the matching `references/pybroker_*.pyi` stub.
- If the user wants a standalone file, copy and adapt `assets/model_training_template.py`.

## Common Deliverables

- Standalone `.py` walkforward backtest script with a trained model.
- `train_fn`/`predict_fn` pairs for a user's chosen library.
- Conversion of an existing single-symbol model to pooled multi-symbol training.
- Debugging notes and patches for leaking targets, misaligned predictions, or invalid `pybroker.model` registrations.
- Notebook-ready PyBroker model training cells.

## Resources

- `references/wiki-index.md`: start here for topic routing across the bundled references.
- `references/wiki-06-training-a-model.md`: model registration, train/backtest flow, model caching, and walkforward analysis.
- `references/wiki-16-time-series-models.md`: GARCH with `per_bar=True` and Random Forest on lagged returns with `lags`/`lag_cols`.
- `references/wiki-17-multi-symbol-models.md`: pooled multi-symbol training with `pooled=True`.
- `references/model-training-patterns.md`: load when writing nontrivial train/predict code; library recipes, session hygiene, and the leakage checklist.
- `references/api-public-surface.md`: generated public API signatures and first docstring sentences from local source.
- `references/pybroker_model.pyi`: generated type stubs for `model()`, `indicator()`, vector helpers, data sources, and top-level module functions.
- `references/pybroker_context.pyi`: generated type stubs for `ExecContext` (including its writable order/stop attributes), `IntervalContext`, `RotationContext`, `ExecResult`, and the slippage models.
- `references/pybroker_strategy.pyi`: generated type stubs for `Strategy`, `StrategyConfig`, `TestResult`, and the optimization types.
- `references/pybroker_types.pyi`: generated type stubs for enums, `BarData`, `Portfolio`, order/trade/position records, and evaluation result types.
- `assets/model_training_template.py`: copy and adapt when creating a new standalone model training script.