beita6969 Experimental Design Skill
Design rigorous, reproducible experiments across scientific disciplines.
When to Use
- "Design an experiment to test..."
- "How many samples do I need?"
- "What controls should I include?"
- "Help me plan a clinical trial"
- "Is this experimental design valid?"
- Power analysis and sample size calculation
When NOT to Use
- Running the actual experiment (use code-execution)
- Analyzing collected data (use scipy-analysis + statsmodels-stats)
- Writing up results (use paper-writing)
- Literature review (use literature-search)
Design Components
1. Research Question and Hypotheses
- State clear, testable research question
- Formulate H0 and H1 (see hypothesis-gen skill)
- Define primary and secondary outcomes
2. Study Design Selection
| Design | When to Use | Strengths | Weaknesses |
|---|---|---|---|
| RCT | Causal inference needed | Gold standard causality | Expensive, ethical limits |
| Factorial | Multiple factors | Tests interactions | Complex analysis |
| Crossover | Within-subject comparison | Reduced variability | Carryover effects |
| Quasi-experimental | Randomization impossible | Practical feasibility | Weaker causality |
| Observational (cohort) | Long-term outcomes | Natural setting | Confounding |
| Case-control | Rare outcomes | Efficient for rare events | Recall bias |
3. Power Analysis
# Sample size calculation template (using scipy/statsmodels)
from statsmodels.stats.power import TTestIndPower
analysis = TTestIndPower()
n = analysis.solve_power(
effect_size=0.5, # Cohen's d (small=0.2, medium=0.5, large=0.8)
alpha=0.05, # Significance level
power=0.80, # Statistical power (commonly 0.80 or 0.90)
ratio=1.0, # Ratio of group sizes (n2/n1)
alternative='two-sided'
)
print(f"Required sample size per group: {int(n) + 1}")
Key parameters:
- Effect size: Expected magnitude of difference
- Alpha: Type I error rate (usually 0.05)
- Power: 1 - Type II error rate (usually 0.80-0.95)
- Attrition: Add 10-20% for expected dropout
4. Variable Control
- Independent variables: What you manipulate
- Dependent variables: What you measure
- Confounding variables: What could bias results
- Control strategies: Randomization, blocking, matching, blinding
5. Randomization
- Simple randomization (coin flip)
- Block randomization (balanced groups)
- Stratified randomization (balance key covariates)
- Cluster randomization (group-level assignment)
6. Blinding
- Single-blind: Participants unaware of assignment
- Double-blind: Participants and researchers unaware
- Triple-blind: Including data analysts
Reproducibility Checklist
- Protocol pre-registered (OSF, ClinicalTrials.gov, PROSPERO)
- All materials/reagents specified with catalog numbers
- Detailed step-by-step procedure written
- Statistical analysis plan pre-specified
- Data management plan documented
- Raw data sharing plan established
- Code availability ensured
- Sample size justified with power analysis
- Randomization method specified
- Blinding procedures documented
- Inclusion/exclusion criteria defined
- Primary endpoint pre-specified
Output Format
## Experimental Design: [Title]
**Research Question**: [Clear question]
**Design Type**: [RCT/Factorial/etc.]
### Participants/Samples
- Population: [target population]
- Inclusion: [criteria]
- Exclusion: [criteria]
- Sample Size: N=[total] ([n] per group) — Power=[X], alpha=[X], effect=[X]
### Groups
- Experimental: [treatment description]
- Control: [control description]
- Blinding: [single/double/triple/none]
### Variables
- IV: [variables]
- DV: [primary + secondary outcomes]
- Controls: [confounds and how addressed]
### Procedure
1. [Step-by-step protocol]
### Analysis Plan
- Primary: [statistical test]
- Secondary: [additional analyses]
- Multiple comparison correction: [method]
### Timeline
- [Phase 1]: [duration]
- [Phase 2]: [duration]
### Ethics
- IRB/IACUC requirements: [details]
- Consent procedure: [details]