Experimentation Throughput Strategy
Use this skill to increase the rate of product experimentation without turning
the experiment pipeline into an unreliable traffic jam. It focuses on testing
capacity, isolated versus overlapping strategies, interaction effects, and the
process/tooling needed to coordinate experiments at scale.
Source Traceability
Primary source: Next-Level A/B Testing by Leemay Nassery. Guidance is
transformed and paraphrased from Chapter 1 on rate, quality, and cost; Chapter
2 on testing availability, isolated and overlapping strategies, and interaction
effects; and Chapter 9 on balancing rate with quality, cost, and usability.
Related skills:
ab-testing-platform-strategy for platform architecture and ownership.experiment-verification-monitoring for monitoring conflicts and active-test
health.experiment-sensitivity-optimization for reducing traffic needs by improving
metric sensitivity.
Reference Routing
| Need |
Read |
| Throughput concepts and terminology |
references/core/knowledge.md |
| Strategy selection and coordination rules |
references/core/rules.md |
| Scenario examples and tradeoffs |
references/core/examples.md |
| Step-by-step throughput plan |
workflows/improve-experiment-throughput.md |
Workflow
- Map the current experiment pipeline and where tests wait.
- Identify whether the bottleneck is traffic, coordination, tooling, review,
QA, analysis, or decision latency.
- Decide whether isolated testing, overlapping testing, or a hybrid model fits
the product surface and metric precision needs.
- Define interference and interaction-effect safeguards.
- Add visibility: current tests, upcoming tests, capacity, ownership, and
conflict flags.
- Write rollout rules so teams know when to schedule, overlap, defer, or split
experiments.
Output Format
# Experimentation Throughput Plan
## Bottleneck
[What is limiting experiment rate and what evidence shows it.]
## Recommended Strategy
[Isolated | Overlapping | Hybrid | Keep current strategy] because [reason].
## Capacity View
| Surface or Audience | Current Tests | Upcoming Tests | Constraint | Owner |
|---------------------|---------------|----------------|------------|-------|
## Interference Safeguards
- Conflict dimensions:
- Monitoring:
- Escalation:
## Process And Tooling Changes
1. [Change]
2. [Change]
3. [Change]
## Decision Rules
- Overlap when:
- Isolate when:
- Defer when:
- Revisit when:
Quality Bar
- Do not increase experiment count by ignoring validity risks.
- Do not default to isolated testing when capacity is the primary constraint and
experiments can run independently.
- Do not default to overlapping testing when experiments change the same user
journey, metric, or surface in ways that can interact.
- Make the coordination mechanism explicit; "teams will communicate" is not a
scalable throughput strategy.
Source: hashgraph-online/awesome-codex-plugins → plugins/LVTD-LLC/skills/skills/experimentation-throughput-strategy/SKILL.md
1---2name: experimentation-throughput-strategy3description: Plan experiment throughput strategies for mature A/B testing programs. Use when testing capacity is constrained, teams are waiting for experiment slots, roadmap coordination is slowing learning, or a team must choose isolated, overlapping, parallel, or capacity-aware experiment scheduling without sacrificing result quality.4---5
6
7# Experimentation Throughput Strategy
8
9Use this skill to increase the rate of product experimentation without turning
10the experiment pipeline into an unreliable traffic jam. It focuses on testing
11capacity, isolated versus overlapping strategies, interaction effects, and the
12process/tooling needed to coordinate experiments at scale.
13
14## Source Traceability
15
16Primary source: *Next-Level A/B Testing* by Leemay Nassery. Guidance is
17transformed and paraphrased from Chapter 1 on rate, quality, and cost; Chapter
182 on testing availability, isolated and overlapping strategies, and interaction
19effects; and Chapter 9 on balancing rate with quality, cost, and usability.
20
21Related skills:
22
23- `ab-testing-platform-strategy` for platform architecture and ownership.
24- `experiment-verification-monitoring` for monitoring conflicts and active-test
25 health.
26- `experiment-sensitivity-optimization` for reducing traffic needs by improving
27 metric sensitivity.
28
29## Reference Routing
30
31| Need | Read |
32|------|------|
33| Throughput concepts and terminology | `references/core/knowledge.md` |
34| Strategy selection and coordination rules | `references/core/rules.md` |
35| Scenario examples and tradeoffs | `references/core/examples.md` |
36| Step-by-step throughput plan | `workflows/improve-experiment-throughput.md` |
37
38## Workflow
39
401. Map the current experiment pipeline and where tests wait.
412. Identify whether the bottleneck is traffic, coordination, tooling, review,
42 QA, analysis, or decision latency.
433. Decide whether isolated testing, overlapping testing, or a hybrid model fits
44 the product surface and metric precision needs.
454. Define interference and interaction-effect safeguards.
465. Add visibility: current tests, upcoming tests, capacity, ownership, and
47 conflict flags.
486. Write rollout rules so teams know when to schedule, overlap, defer, or split
49 experiments.
50
51## Output Format
52
53```markdown
54# Experimentation Throughput Plan
55
56## Bottleneck
57[What is limiting experiment rate and what evidence shows it.]
58
59## Recommended Strategy
60[Isolated | Overlapping | Hybrid | Keep current strategy] because [reason].
61
62## Capacity View
63| Surface or Audience | Current Tests | Upcoming Tests | Constraint | Owner |
64|---------------------|---------------|----------------|------------|-------|
65
66## Interference Safeguards
67- Conflict dimensions:
68- Monitoring:
69- Escalation:
70
71## Process And Tooling Changes
721. [Change]
732. [Change]
743. [Change]
75
76## Decision Rules
77- Overlap when:
78- Isolate when:
79- Defer when:
80- Revisit when:
81```
82
83## Quality Bar
84
85- Do not increase experiment count by ignoring validity risks.
86- Do not default to isolated testing when capacity is the primary constraint and
87 experiments can run independently.
88- Do not default to overlapping testing when experiments change the same user
89 journey, metric, or surface in ways that can interact.
90- Make the coordination mechanism explicit; "teams will communicate" is not a
91 scalable throughput strategy.
92
93---
94
95**Source:** [`hashgraph-online/awesome-codex-plugins`](https://github.com/hashgraph-online/awesome-codex-plugins) → `plugins/LVTD-LLC/skills/skills/experimentation-throughput-strategy/SKILL.md`
thedixitjain