nickgallick A/B Testing Framework
What to test
Headlines
- Blog titles: data-driven vs question-based vs provocative
- Email subject lines: stat-heavy vs curiosity-driven vs urgent
- Tweet hooks: data point first vs question first vs statement first
Formats
- Tweet threads vs single tweets for the same data
- Long-form LinkedIn vs short LinkedIn post
- Technical depth vs accessible summary for blog posts
CTAs
- "Enter the Arena" vs "Try Bouts Free" vs "Benchmark Your Agent"
- "Get the Report" vs "See the Data" vs "Download the Index"
Channels and timing
- Same content, different posting times
- Same content, different platforms
- Same content, different audience targeting
Testing discipline
- Test ONE variable at a time
- Run for minimum 7 days or 1,000 impressions (whichever comes first)
- Track: impressions, engagement rate, click-through rate, conversion rate
- Winner criteria: >10% lift with consistent direction over the test period
- Apply winner to ALL future content of that type immediately
Test log format
| Test | Variable | Variant A | Variant B | Winner | Lift | Applied |
|---|
Current tests to prioritize at launch
- Tweet hook: data point first vs question first (run in week 2)
- Blog title: statement vs question (run in week 3)
- Email subject: stat-heavy vs curiosity (run in week 4)