# Backtesting

> The directional-specific half of backtesting — window sizing, the metric threshold table, parameter sweeps, and the out-of-sample go/no-go. The tool contract and the family-agnostic rules live in the shared `backtest_flow` skill.

- Skill: `hummingbot/backtesting` (Agent Skill, multi-file: 5 files)
- Install (CLI): `npx skillmds@latest add hummingbot/backtesting`
- Raw SKILL.md: https://api.skillmd.com/api/skills/hummingbot/backtesting/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: hummingbot (https://skillmd.com/u/hummingbot)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/hummingbot/backtesting

---


# Backtesting

Take an uploaded config → a named winner with confirmed metrics, ready to deploy.
Read this hub, then fetch the companion for the step you are actually on:

```
manage_skill(action="read_file", name="backtesting", file="interpret_metrics.md")
```

## Which companion to read

| You are…                                                          | Read                       |
|-------------------------------------------------------------------|----------------------------|
| Running anything for the first time this session                   | shared `backtest_flow`     |
| Sizing the window for a controller interval                        | `windows_and_costs.md`     |
| Reading results — thresholds, red flags, how to report them        | `interpret_metrics.md`     |
| Designing or ranking a parameter sweep; checking for overfitting   | `parameter_sweep.md`       |
| Validating out-of-sample and deciding deploy / don't deploy        | `go_no_go.md`              |

A full pass reads `backtest_flow` once, then all four in that order. A one-off
"what does this Sharpe mean?" needs only `interpret_metrics.md`.

## The tool contract — read it once, elsewhere

How to run a backtest is **not** directional knowledge, so it is not duplicated
here. The shared `backtest_flow` skill holds the single copy: the `backtest_chart`
routine, blocking vs `run_async`/`get_instance`, the `table_data` columns,
`chart=False` for sweeps, `task_id` persistence and retention, plus the rules that
hold for every strategy family (trade cost, resolution fidelity, the trade-count
gate, one-parameter sweeps, mandatory out-of-sample).

```
manage_skill(action="read", name="backtest_flow")
```

Read it before your first run of a session. What follows in *this* playbook is the
part that is calibrated to directional trading and does not transfer to other
strategy families.

## The loop

1. **Baseline** — one run over a meaningful window (`windows_and_costs.md`).
2. **Read it** — apply the threshold table; stop early if the signal is noise
   (`interpret_metrics.md`).
3. **Sweep** — one parameter at a time, look for a plateau (`parameter_sweep.md`).
4. **Validate** — held-out window, then decide (`go_no_go.md`).
5. **Document the winner** and hand it to the `deploy_and_monitor` playbook.

## Non-negotiables

The universal ones — one parameter at a time, plateau over peak, mandatory
out-of-sample, the trade-count gate, report the bad numbers too — live in
`backtest_flow` and apply here unchanged. On top of them, for directional:

- **Sharpe < 0 means the signal is actively harmful.** Go back to `research`; do
  not sweep a negative-edge signal until it looks positive.
- **Never deploy a config that has not passed `go_no_go.md`** — a good baseline is
  not a decision.
- **Judge the exit, not just the entry.** An average trade duration far off the
  controller interval means `time_limit` or the stop is driving exits, not the
  signal — a "profitable" config of that shape is not the strategy you think you
  are deploying.