Multiple Testing Ledger

Apply family-wise error and false-discovery control to a whole research programme, using the trial ledger's own registered count as m. TRIGGER - Bonferroni, Holm, Benjamini-Hochberg, Benjamini-Yekutieli, BHY, FDR, FWER, "adjust my p-values", statsmodels multipletests, "which of my factors survive the correction"; haircut Sharpe ratio, Harvey-Liu-Zhu, "t-stat of 3", factor zoo, "is a t of 2.1 enough"; "I screened 500 signals and 30 came back significant", "which correction should I use", "my strategies are all correlated so Bonferroni is too strict". SKIP for the deflated and probabilistic Sharpe ratios and the ledger file format itself (backtest-validation); for PBO, CSCV and the minimum backtest length (backtest-overfitting); for SPA / StepM / MCS, which correct for the search using the bootstrap instead of p-values (backtest-validation); and for the leakage and cost gates that must pass before a p-value means anything (research-integrity-guards).

howard-lynn-ye Updated 1 repo stars

File contents

howard-lynn-ye/fin-skills commit 604f98b596

Frequently asked questions

npx get-skill add howard-lynn-ye/multiple-testing-ledger