Results for “experiment-analysis”

Design, prioritize, and evaluate product experiments with clear hypotheses and defensible decisions, including A/B testing, sample size estimation, and statistical interpretation.

Analyse experiment results, run validity checks (SRM, exposure parity, data integrity, novelty/primacy), interpret causally, make a ship/iterate/kill decision against the pre-declared rule, and append to cumulative learnings. Forces honest readouts — strips significance claims from underpowered or peek-violating tests; never lets directional results masquerade as causal wins. Load when results exist, or when the user says "read out this experiment", "analyse the test", "did the test win", "interpret the results", "what did we learn", "ship or kill", or when the experimentation orchestrator routes here.

Analyze ML experiment results, compute statistics, generate comparison tables and insights. Use when user says "analyze results", "compare", or needs to interpret experimental data.

Transforms A/B test and product experiment data into actionable readouts with hypothesis, metrics, interpretation, and decision.

Autonomously runs iterative experiment loops to optimize code against a measurable metric. Use when the user wants to improve execution time, memory usage, test pass rate, or any numeric performance goal across repeated experiments — NOT for one-shot bug fixes or simple code review.

Design and evaluate A/B tests with power, sample size, and robust metric interpretation. Use when: (1) planning controlled experiments, (2) reading p-values/effects, (3) sequential testing safeguards. NOT for: dark-pattern optimization.

Analyze A/B test results with statistical significance, sample size validation, confidence intervals, and ship/extend/stop recommendations. Use when evaluating experiment results, checking if a test reached significance, interpreting split test data, or deciding whether to ship a variant.

Challenge system assumptions against accumulated evidence. Triages observations and tensions, detects patterns, generates proposals. The scientific method applied to knowledge systems. Triggers on "/rethink", "review observations", "challenge assumptions", "what have I learned".

Run a single experiment iteration. Edit the target file, evaluate, keep or discard. Use when the user runs /ar:run or asks for one manual autoresearch iteration.

Design experiments and studies before data collection — choosing a design, randomizing, blocking, and laying out treatment combinations so results are interpretable.

Build reproducible ML experiment plans with hypotheses, metrics, and ablations. Use when: (1) planning experiments, (2) comparing variants, (3) defining acceptance thresholds. NOT for: long-running experiment execution.

Run a single experiment iteration. Edit the target file, evaluate, keep or discard. Use when the user runs /ar:run or asks for one manual autoresearch iteration.

Resume a paused experiment. Checkout the experiment branch, read results history, continue iterating.

Resume a paused experiment. Checkout the experiment branch, read results history, continue iterating. Use when the user runs /ar:resume or asks to pick up a previously started autoresearch experiment.

Run a single experiment iteration. Edit the target file, evaluate, keep or discard.

Design product experiments around hypotheses, audience, metrics, guardrails, and decisions.

Set up a new autoresearch experiment interactively. Collects domain, target file, eval command, metric, direction, and evaluator.

Set up a new autoresearch experiment interactively. Collects domain, target file, eval command, metric, direction, and evaluator.

Econometrics skill for Difference-in-Differences (DID) analysis. Activates when the user asks about: "difference in differences", "DID", "DiD", "diff-in-diff", "parallel trends", "treatment group", "control group", "pre-treatment", "post-treatment", "policy evaluation", "natural experiment", "staggered DID", "event study regression", "two-way fixed effects DID", "callaway santanna", "sun and abraham", "双重差分", "倍差法", "平行趋势", "处理组", "对照组", "政策评估", "事件研究", "交错DID", "渐进处理"

Set up a new autoresearch experiment interactively. Collects domain, target file, eval command, metric, direction, and evaluator. Use when the user runs /ar:setup or asks to start optimizing a file with the autoresearch loop.

Evaluate scientific claims and evidence quality by assessing experimental design, identifying biases and confounders, and applying evidence grading frameworks like GRADE and Cochrane Risk of Bias.

Refactoring Analyst