alexe-ev Design Experiment Plan
Purpose
Turn experiment inputs into a complete execution-ready test design.
Skill type
Conceptual orchestration skill
Use this skill when
- The user is ready to run a test
- The hypothesis has already been validated
- Metrics and decision rules are known
- A team-ready experiment brief is needed
Required inputs
- Validated hypothesis
- Primary metric
- Success criteria
Optional inputs
- Secondary metrics
- Guardrails
- Sample size
- Estimated duration
- Target segment
- Eligibility rules
- Randomization logic
- Traffic constraints
- Rollout constraints
Additional resources
Use examples/ when
- you need to see what good input/output looks like
- you need to match the expected output structure
- you need to see how this skill behaves under rich, light, or poor context
Use REFERENCE.md when
- this skill includes one and you need additional decision rules
- you need deeper context-handling guidance
- you need edge cases or caveats not included in the main instructions
Resource priority
- Read this Skill.md first
- Use examples/ for behavior patterns and output structure
- Use REFERENCE.md for deeper rules, caveats, and edge cases
Upstream context
This skill works best when the following already exist:
- a validated hypothesis
- a primary metric
- success criteria
- target segment
- at least rough threshold logic
- sample size and duration, if launch planning needs them
If upstream context is missing
If the hypothesis, primary metric, threshold logic, or stopping logic is missing, do not generate a fake-ready experiment brief. Return a partial planning skeleton and clearly mark which gaps block launch readiness.
Downstream handoff
A complete output from this skill should be usable by:
- experiment launch preparation
- implementation handoff
- later result analysis
- later post-test decision-making
A good handoff includes:
- objective
- variants
- audience
- metrics
- thresholds
- stopping logic
- risks
- launch blockers if any
Context collection rules
Context-rich
Produce an execution-ready plan using the real audience, funnel, metric, constraints, and risks.
Context-light
Produce a partial plan, but clearly mark missing fields as unresolved.
Context-poor
Do not generate a fake-ready experiment brief. Switch to planning skeleton mode and show what must be defined first.
Minimum context to collect before calling the plan execution-ready
- exact change being tested
- target segment
- primary metric
- success threshold
- stopping logic
- basic rollout risk or guardrail logic
Instructions
- Restate the hypothesis clearly.
- Define control and test variants.
- Define target segment and eligibility rules.
- Define primary, secondary, and guardrail metrics.
- Include minimum meaningful effect and success threshold if available.
- Include sample size and expected duration if available.
- Define stopping logic.
- Define analysis plan at a high level.
- Define success, neutral, and failure outcomes.
- Define what action should follow each outcome.
- Flag operational or validity risks.
Output
Provide a structured experiment brief with:
- Objective
- Hypothesis
- Variants
- Target audience
- Metrics
- Thresholds
- Sample size and duration
- Stopping logic
- Launch notes
- Risks
- Decision rules
- Missing fields that block launch readiness
Risks / caveats
- Do not produce a fake-complete brief if core decisions are still missing
- Do not recommend running a test without knowing how the result will be interpreted
- Do not allow flexible stopping without calling it out as a validity risk