Experiment Arms

Runs many single-hypothesis "arms" as do-not-merge PRs, measures each with paired, interleaved runs against a cost-matched control, keeps only the arms that clear a keep bar set in advance, records every negative, and composes the winners into a stack that is re-measured on real traffic after merge. It opens PRs and burns CI, so it runs only on `/lever arms` or an explicit ask to measure ideas ("which of these ideas actually helps?", "is this change worth it?", "try that tip I saw", "run arms on this") for a CI, perf, eval, benchmark or cost hill-climb where the system can be run on demand.

GitHub (Microsoft) Updated 41.2k repo stars

File contents

github/lever commit 8a37437631

Frequently asked questions

npx get-skill add github/experiment-arms