Experiment Stats

This skill should be used when the user asks 'is this A/B test significant', 'did the variant actually win', 'compute lift and confidence interval', 'score a marketing experiment', 'run the promote-gate', or wants a statistically defensible promote/hold/reject call on an A/B or multivariate marketing experiment. Computes lift, a bootstrap confidence interval, and a significance test (Mann-Whitney U or two-proportion z) using pure Python standard library, without scipy or numpy. Every p-value is computed from the input data, never estimated. Use before promoting any experiment variant to production.

b-open-io Updated 13 repo stars

File contents

b-open-io/product-skills commit a9fc5352db

Frequently asked questions

npx get-skill add b-open-io/experiment-stats