alexe-ev Design Channel Experiment
Purpose
Help teams design time-boxed, hypothesis-driven experiments to validate whether a new acquisition channel can deliver users at acceptable cost and quality before scaling.
Skill type
Conceptual skill
Use this skill when
- A new channel is being considered for investment
- An existing channel is underperforming and a different approach needs to be tested
- A channel budget needs to be justified with a structured test
- Multiple channels need to be compared systematically
Do not use this skill when
- The goal is product A/B experiment design (use design-experiment-plan)
- The goal is broad channel strategy (use strategize-channel-campaigns)
Required inputs
- Channel to test
- Hypothesis: why should this channel work for this product and segment?
- Target metric (CAC, conversion rate, user quality proxy)
Optional inputs
- Budget available for the test
- Time horizon
- Existing channel baseline for comparison
- Target segment for the channel
Upstream context
Works best when:
- Growth model is defined
- Acquisition funnel is mapped
- Channel strategy exists
Downstream handoff
Output can feed:
- strategize-channel-campaigns (validated channels go into channel portfolio)
- map-acquisition-funnel (channel experiment data feeds funnel by channel)
Instructions
- State the channel hypothesis clearly: "We believe [channel] will acquire [segment] at [$X CAC / Y% conversion] because [reason]."
- Define success criteria: what would make this channel worth scaling?
- Define failure criteria: what would mean this channel doesn't work for us?
- Define the minimum test budget and timeline to get a meaningful signal.
- Define the measurement plan: what metrics will be tracked and how.
- Identify the test parameters: audience, creative, message, landing experience.
- Define the decision framework: scale / iterate / kill.
Output
Provide:
- Channel hypothesis
- Success and failure criteria
- Test parameters (audience, creative, message)
- Budget and timeline
- Measurement plan
- Decision framework: scale / iterate / kill triggers
- Risks and mitigations
Risks / caveats
- Underfunded channel tests produce inconclusive results — define minimum test budget before starting
- Don't test channel and message simultaneously — isolate variables
- User quality (retention, conversion downstream) matters as much as acquisition volume