# Experiment Platform

> Skill — Experiment Platform

- Skill: `sairam0424/experiment-platform` (Agent Skill)
- Install (CLI): `npx skillmds@latest add sairam0424/experiment-platform`
- Raw SKILL.md: https://api.skillmd.com/api/skills/sairam0424/experiment-platform/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: sairam0424 (https://skillmd.com/u/sairam0424)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/sairam0424/experiment-platform

---


# Skill — Experiment Platform

## When this skill activates
This skill activates when building experimentation infrastructure, implementing statistical testing frameworks, or designing A/B testing platforms. Use when organizations need to scale testing velocity while maintaining statistical rigor.

## Mandatory actions when this skill is active

### Before writing any code
1. Define experiment framework components: randomization service, exposure logging, metric computation pipeline, and statistical analysis engine
2. Establish statistical rigor standards: minimum sample size, power (80%+), significance level (5%), minimum detectable effect, and multiple comparison corrections
3. Design guardrail metrics framework: business health (revenue, retention), user experience (latency, errors), and ecosystem health (partner impact)
4. Plan experiment lifecycle states: draft, review, running, paused, completed, archived with transition criteria and approval gates

### During implementation
- Implement consistent randomization using stable hashing (user_id + experiment_id) ensuring users see same variant across sessions
- Build exposure logging capturing: timestamp, user_id, experiment_id, variant, context for accurate sample size and covariate adjustment
- Create metric computation pipeline with: numerator/denominator structure, winsorization for outliers, delta method for ratios, bootstrap for confidence intervals
- Design sequential testing capability for early stopping: alpha spending functions, futility boundaries, and minimum runtime requirements
- Implement stratified analysis for heterogeneous treatment effects: by platform, user segment, geography with interaction effect testing
- Build automated guardrail checks: alert on significant negative movement in critical metrics with experiment auto-pause capability
- Create experiment metadata repository: hypothesis, success criteria, related experiments, learnings, and decision outcome for institutional knowledge

### After implementation
- Generate automated experiment scorecards: primary metric movement, guardrail status, statistical significance, practical significance, recommendation
- Build experiment catalog with search and discovery: hypothesis library, metric glossary, analysis templates, and historical results
- Create experimentation health dashboard: velocity (experiments/week), quality (statistical power distribution), impact (significant wins), and coverage (features tested)
- Document statistical methodology: test selection, variance reduction techniques, multiple comparison approach, and sequential testing procedures

## Self-check before task completion
- [ ] Randomization service ensures stable assignment and balanced allocation across variants
- [ ] Exposure logging captures all necessary context for accurate analysis and debugging
- [ ] Statistical analysis engine implements proper corrections for multiple comparisons and peeking
- [ ] Guardrail metrics monitored automatically with alerting and experiment pause capability
- [ ] Experiment lifecycle enforces minimum runtime and sample size before declaring results