Contract
- Input: trading strategy definition (signal, universe, sizing, costs).
- Output: backtest audit report with strategy spec, data audit, statistics, stress tests, and verdict.
- Side effects: none.
- Dependencies: data source access.
- Stop condition: audit report complete with verdict.
- Risk: low.
- Boundary: produces audit report; does not execute live trading.
Quant Backtest Audit
Take a strategy and produce a backtest result you can defend. A backtest that can't survive audit is decoration.
When to use
- The user has a strategy and wants its historical performance.
- A claimed alpha or Sharpe needs independent verification.
- A live-trading decision will be made from the result.
Process
1. Spec the strategy
Capture in writing:
- Signal(s) — input → score → decision.
- Universe — assets and any filtering rules.
- Schedule — rebalance dates, holding period.
- Sizing — equal weight, vol-targeted, Kelly-fraction.
- Costs — commissions, spread, borrow, impact.
No ambiguity. If a parameter is unspecified, ask.
Completion criterion: the strategy could be re-implemented from the spec alone.
2. Audit the data
- Survivorship bias: use point-in-time dataset; keep delisted names.
- Look-ahead: features computed only on data available at the decision timestamp.
- Adjustments: splits, dividends, mergers consistent with provider.
- Corporate actions: delistings reflected in returns (delisting return ≠ 0).
Completion criterion: each bias named and the dataset policy is documented.
3. Run the backtest
- Transaction-cost model: not zero. Commissions + spread + impact estimate.
- Out-of-sample split: train / test / walk-forward, not just one backtest window.
- Benchmark: explicit (e.g. cap-weighted universe, risk-free, SPY).
- Report: Sharpe, Sortino, max drawdown, Calmar, annualised return, annualised vol, win rate.
Completion criterion: backtest run; out-of-sample period honoured; all statistics present.
4. Stress tests
- Subperiods: does it survive across decades / regimes (2008, 2020, 2022)?
- Sectors / geographies: is the effect localised to one slice?
- Turnover: annualised; high turnover often eats alpha.
- Crowding: is the strategy correlated with known crowded trades?
Completion criterion: each stress test reported; weakest point named.
5. Coverage validation
Does VaR 95% cover ~95% of out-of-sample days? If not, recalibrate.
Completion criterion: coverage check completed; discrepancy addressed.
6. Verdict
Deliver the verdict:
- Robust: survives stress tests, stable across subperiods, reasonable turnover, Sharpe > 1 after costs.
- Weak: passes some tests but fragile under regime change or high turnover.
- Fails: does not survive out-of-sample or stress tests.
Completion criterion: verdict stated with evidence; recommendation (deploy / do not deploy / revise).
7. Deliver
Markdown artifact: strategy spec, data audit, backtest statistics, stress tests, coverage validation, and verdict with explicit conditions.
Completion criterion: report complete; all sections present; verdict honest.
1---2name: quant-backtest-23description: Run a backtest with full audit hygiene — biases, costs, out-of-sample, regime splits — and produce a verdict on whether a strategy is robust.4---56## Contract78- **Input:** trading strategy definition (signal, universe, sizing, costs).9- **Output:** backtest audit report with strategy spec, data audit, statistics, stress tests, and verdict.10- **Side effects:** none.11- **Dependencies:** data source access.12- **Stop condition:** audit report complete with verdict.13- **Risk:** low.14- **Boundary:** produces audit report; does not execute live trading.1516# Quant Backtest Audit1718Take a **strategy** and produce a **backtest result you can defend**. A backtest that can't survive audit is decoration.1920## When to use2122- The user has a strategy and wants its historical performance.23- A claimed alpha or Sharpe needs independent verification.24- A live-trading decision will be made from the result.2526## Process2728### 1. Spec the strategy2930Capture in writing:3132- **Signal(s)** — input → score → decision.33- **Universe** — assets and any filtering rules.34- **Schedule** — rebalance dates, holding period.35- **Sizing** — equal weight, vol-targeted, Kelly-fraction.36- **Costs** — commissions, spread, borrow, impact.3738No ambiguity. If a parameter is unspecified, ask.3940**Completion criterion:** the strategy could be re-implemented from the spec alone.4142### 2. Audit the data4344- **Survivorship bias:** use point-in-time dataset; keep delisted names.45- **Look-ahead:** features computed only on data available at the decision timestamp.46- **Adjustments:** splits, dividends, mergers consistent with provider.47- **Corporate actions:** delistings reflected in returns (delisting return ≠ 0).4849**Completion criterion:** each bias named and the dataset policy is documented.5051### 3. Run the backtest5253- **Transaction-cost model:** not zero. Commissions + spread + impact estimate.54- **Out-of-sample split:** train / test / walk-forward, not just one backtest window.55- **Benchmark:** explicit (e.g. cap-weighted universe, risk-free, SPY).56- Report: Sharpe, Sortino, max drawdown, Calmar, annualised return, annualised vol, win rate.5758**Completion criterion:** backtest run; out-of-sample period honoured; all statistics present.5960### 4. Stress tests6162- **Subperiods:** does it survive across decades / regimes (2008, 2020, 2022)?63- **Sectors / geographies:** is the effect localised to one slice?64- **Turnover:** annualised; high turnover often eats alpha.65- **Crowding:** is the strategy correlated with known crowded trades?6667**Completion criterion:** each stress test reported; weakest point named.6869### 5. Coverage validation7071Does VaR 95% cover ~95% of out-of-sample days? If not, recalibrate.7273**Completion criterion:** coverage check completed; discrepancy addressed.7475### 6. Verdict7677Deliver the verdict:7879- **Robust:** survives stress tests, stable across subperiods, reasonable turnover, Sharpe > 1 after costs.80- **Weak:** passes some tests but fragile under regime change or high turnover.81- **Fails:** does not survive out-of-sample or stress tests.8283**Completion criterion:** verdict stated with evidence; recommendation (deploy / do not deploy / revise).8485### 7. Deliver8687Markdown artifact: strategy spec, data audit, backtest statistics, stress tests, coverage validation, and verdict with explicit conditions.8889**Completion criterion:** report complete; all sections present; verdict honest.
quantumquirkxyz