Power = 80% is conventional. Higher power (90%) is better when study is expensive or one-shot.
import numpy as np
from scipy.stats import norm
# Parameters
alpha = 0.05
power = 0.80
mde = 0.25 # Effect size in SD units (realistic for social programs)
p = 0.50 # Treatment allocation
# Base calculation
z_alpha = norm.ppf(1 - alpha/2)
z_power = norm.ppf(power)
n_base = ((z_alpha + z_power)**2) / (p * (1-p) * mde**2)
print(f"COMPREHENSIVE POWER ANALYSIS")
print(f"=" * 60)
print(f"Parameters: α={alpha}, power={power}, MDE={mde} SD")
print(f"\n1. Base sample size: {int(np.ceil(n_base))}")
# Adjustment 1: Clustering
icc = 0.05
cluster_size = 30
design_effect = 1 + (cluster_size - 1) * icc
n_after_clustering = n_base * design_effect
print(f"\n2. Clustering adjustment:")
print(f" ICC={icc}, avg cluster size={cluster_size}")
print(f" Design effect: {design_effect:.2f}")
print(f" After clustering: {int(np.ceil(n_after_clustering))}")
# Adjustment 2: Non-compliance
compliance_rate = 0.70
n_after_compliance = n_after_clustering / (compliance_rate**2)
print(f"\n3. Non-compliance adjustment:")
print(f" Expected compliance: {compliance_rate:.0%}")
print(f" Inflation factor: {1/compliance_rate**2:.2f}")
print(f" After compliance: {int(np.ceil(n_after_compliance))}")
# Adjustment 3: Attrition
attrition_rate = 0.20
n_final = n_after_compliance / (1 - attrition_rate)
print(f"\n4. Attrition adjustment:")
print(f" Expected attrition: {attrition_rate:.0%}")
print(f" Inflation factor: {1/(1-attrition_rate):.2f}")
print(f" FINAL REQUIRED N: {int(np.ceil(n_final))}")
print(f"\n" + "=" * 60)
print(f"Total inflation: {n_final/n_base:.2f}x base sample size")
print(f"Baseline N needed: {int(np.ceil(n_final))}")
print(f"Endline N expected: {int(np.ceil(n_final * (1-attrition_rate)))}")
return int(np.ceil(n_final)),
</code_template>
</implementation_pattern>
<examples>
<example context="cluster_rct_power" difficulty="intermediate">
<description>Calculate power for cluster-randomized trial with design effect</description>
<code>
```python
@app.cell
def cluster_rct_power():
# Cluster RCT power accounting for design effect
# Shows how clustering can reduce power by 2-4x
import numpy as np
from scipy.stats import norm
alpha = 0.05
power = 0.80
mde = 0.30
# Individual randomization (baseline)
z_a = norm.ppf(1 - alpha/2)
z_p = norm.ppf(power)
n_individual = ((z_a + z_p)**2) / (0.25 * mde**2)
print(f"CLUSTER vs. INDIVIDUAL RANDOMIZATION")
print(f"=" * 60)
print(f"Individual randomization: {int(np.ceil(n_individual))} participants")
# Cluster randomization scenarios
scenarios = [
{"icc": 0.05, "cluster_size": 20},
{"icc": 0.05, "cluster_size": 50},
{"icc": 0.10, "cluster_size": 30},
{"icc": 0.20, "cluster_size": 30},
]
for scenario in scenarios:
icc = scenario["icc"]
m = scenario["cluster_size"]
de = 1 + (m - 1) * icc
n_cluster = n_individual * de
n_clusters_needed = int(np.ceil(n_cluster / m))
print(f"\nICC={icc}, cluster size={m}:")
print(f" Design effect: {de:.2f}")
print(f" Total N needed: {int(np.ceil(n_cluster))}")
print(f" Clusters needed: {n_clusters_needed}")
print(f" Inflation: {de:.2f}x individual design")
print("\nKey lesson: Higher ICC or larger clusters → much larger N needed")
return ()
Then multiply by:
- Design effect if clustering: 1+(m-1)×ICC
- 1/compliance² for non-compliance
- 1/(1-attrition_rate) for attrition
Do NOT: proceed with underpowered study and hope for best.
1---2name: rct-power-calculations3description: Calculate statistical power and sample sizes for RCTs. Use when user mentions: power analysis, sample size calculation, minimum detectable effect, MDE, statistical power, effect size, clustering effects, design effect, ICC, intra-cluster correlation.4---56<skill_content>78<overview>9Power analysis determines the sample size required to detect treatment effects of a given magnitude with specified probability. Adequate power prevents Type II errors (failing to detect real effects). Power calculations must account for clustering (design effects), non-compliance (reduces effective sample), attrition (reduces final sample), and multiple testing (inflates α). Underpowered studies waste resources and risk being misinterpreted as evidence of no effect.1011Power = 80% is conventional. Higher power (90%) is better when study is expensive or one-shot.12</overview>1314<mandatory_requirements>1516<requirement priority="critical">17 <name>Conduct Power Analysis Before Data Collection</name>18 <description>Calculate required sample size BEFORE starting enrollment or randomization</description>19 <rationale>Post-hoc power calculations are meaningless. Power must guide design, not rationalize results (Hoenig & Heisey 2001)</rationale>20 <consequence>Underpowered studies waste resources, fail to detect real effects, mislead policy with false nulls</consequence>21</requirement>2223<requirement priority="critical">24 <name>Account for Clustering Design Effects</name>25 <description>When using cluster randomization, multiply base sample size by design effect: DE = 1 + (m-1) × ICC</description>26 <rationale>Clustering reduces effective sample size due to within-cluster correlation. Ignoring this causes severe underpowering (Donner & Klar 2000)</rationale>27 <consequence>Studies underpowered by 2-4x, inability to detect real effects despite large nominal sample</consequence>28</requirement>2930<requirement priority="critical">31 <name>Inflate for Attrition and Non-Compliance</name>32 <description>Adjust sample size for expected attrition and imperfect compliance before finalizing design</description>33 <rationale>Attrition reduces final sample. Non-compliance attenuates ITT effects. Both reduce power substantially</rationale>34 <consequence>Final analysis underpowered even if initial enrollment meets naive power target</consequence>35</requirement>3637<requirement priority="high">38 <name>Use Realistic Effect Sizes</name>39 <description>Base MDE on pilot data, similar interventions, or smallest policy-relevant effect, not on wishful thinking</description>40 <rationale>Overoptimistic effect size assumptions guarantee underpowered study. True effects in social science typically 0.1-0.3 SD (Vivalt 2020)</rationale>41 <consequence>Study designed to detect implausibly large effects, fails to detect realistic effects, wasted resources</consequence>42</requirement>4344<requirement priority="high">45 <name>Adjust for Multiple Comparisons</name>46 <description>When testing multiple outcomes, increase sample size or reduce α to maintain family-wise error rate</description>47 <rationale>Multiple testing inflates Type I error. With 5 tests at α=0.05, false positive rate is 26% not 5%</rationale>48 <consequence>Study powered for single test but not for actual analysis plan, false discoveries</consequence>49</requirement>5051</mandatory_requirements>5253<assumptions>5455<assumption name="Effect Size Estimate">56 <description>Assumed MDE or effect size is realistic based on prior evidence or theory</description>57 <how_to_check>Review similar interventions, conduct pilot, consult domain experts, use smallest policy-relevant effect</how_to_check>58 <if_violated>Study either grossly overpowered (wasteful) or underpowered (fails to detect real effects)</if_violated>59</assumption>6061<assumption name="ICC Estimate for Cluster Designs">62 <description>Assumed intra-cluster correlation reflects true within-cluster similarity</description>63 <how_to_check>Use baseline data from same clusters, similar studies, or conservative upper bound (ICC=0.10-0.20)</how_to_check>64 <if_violated>Design effect wrong → sample size wrong → power incorrect</if_violated>65</assumption>6667<assumption name="Attrition and Compliance Rates">68 <description>Expected attrition and take-up rates match what actually occurs</description>69 <how_to_check>Base on pilot data, similar studies in same context, plan retention strategies</how_to_check>70 <if_violated>Actual power lower than designed power, inability to detect effects</if_violated>71</assumption>7273</assumptions>7475<thinking_process>76When conducting power analysis:771. Specify primary outcome and effect size (MDE or expected effect)782. Choose significance level (α=0.05 typical) and desired power (0.80-0.90)793. Calculate base sample size for simple design804. Apply design effect if cluster randomization (multiply by 1+(m-1)×ICC)815. Inflate for expected non-compliance (divide by compliance²)826. Inflate for expected attrition (divide by (1-attrition_rate))837. Adjust for multiple testing if applicable848. Check feasibility; if infeasible, reduce scope or accept larger MDE85</thinking_process>8687<implementation_pattern>8889<code_template>90```python91@app.cell92def comprehensive_power_calculation():93 # Full power calculation with all real-world adjustments94 # Demonstrates compounding inflation factors for RCT sample size9596 import numpy as np97 from scipy.stats import norm9899 # Parameters100 alpha = 0.05101 power = 0.80102 mde = 0.25 # Effect size in SD units (realistic for social programs)103 p = 0.50 # Treatment allocation104105 # Base calculation106 z_alpha = norm.ppf(1 - alpha/2)107 z_power = norm.ppf(power)108 n_base = ((z_alpha + z_power)**2) / (p * (1-p) * mde**2)109110 print(f"COMPREHENSIVE POWER ANALYSIS")111 print(f"=" * 60)112 print(f"Parameters: α={alpha}, power={power}, MDE={mde} SD")113 print(f"\n1. Base sample size: {int(np.ceil(n_base))}")114115 # Adjustment 1: Clustering116 icc = 0.05117 cluster_size = 30118 design_effect = 1 + (cluster_size - 1) * icc119 n_after_clustering = n_base * design_effect120121 print(f"\n2. Clustering adjustment:")122 print(f" ICC={icc}, avg cluster size={cluster_size}")123 print(f" Design effect: {design_effect:.2f}")124 print(f" After clustering: {int(np.ceil(n_after_clustering))}")125126 # Adjustment 2: Non-compliance127 compliance_rate = 0.70128 n_after_compliance = n_after_clustering / (compliance_rate**2)129130 print(f"\n3. Non-compliance adjustment:")131 print(f" Expected compliance: {compliance_rate:.0%}")132 print(f" Inflation factor: {1/compliance_rate**2:.2f}")133 print(f" After compliance: {int(np.ceil(n_after_compliance))}")134135 # Adjustment 3: Attrition136 attrition_rate = 0.20137 n_final = n_after_compliance / (1 - attrition_rate)138139 print(f"\n4. Attrition adjustment:")140 print(f" Expected attrition: {attrition_rate:.0%}")141 print(f" Inflation factor: {1/(1-attrition_rate):.2f}")142 print(f" FINAL REQUIRED N: {int(np.ceil(n_final))}")143144 print(f"\n" + "=" * 60)145 print(f"Total inflation: {n_final/n_base:.2f}x base sample size")146 print(f"Baseline N needed: {int(np.ceil(n_final))}")147 print(f"Endline N expected: {int(np.ceil(n_final * (1-attrition_rate)))}")148149 return int(np.ceil(n_final)),150```151</code_template>152153</implementation_pattern>154155<examples>156157<example context="cluster_rct_power" difficulty="intermediate">158<description>Calculate power for cluster-randomized trial with design effect</description>159<code>160```python161@app.cell162def cluster_rct_power():163 # Cluster RCT power accounting for design effect164 # Shows how clustering can reduce power by 2-4x165166 import numpy as np167 from scipy.stats import norm168169 alpha = 0.05170 power = 0.80171 mde = 0.30172173 # Individual randomization (baseline)174 z_a = norm.ppf(1 - alpha/2)175 z_p = norm.ppf(power)176 n_individual = ((z_a + z_p)**2) / (0.25 * mde**2)177178 print(f"CLUSTER vs. INDIVIDUAL RANDOMIZATION")179 print(f"=" * 60)180 print(f"Individual randomization: {int(np.ceil(n_individual))} participants")181182 # Cluster randomization scenarios183 scenarios = [184 {"icc": 0.05, "cluster_size": 20},185 {"icc": 0.05, "cluster_size": 50},186 {"icc": 0.10, "cluster_size": 30},187 {"icc": 0.20, "cluster_size": 30},188 ]189190 for scenario in scenarios:191 icc = scenario["icc"]192 m = scenario["cluster_size"]193 de = 1 + (m - 1) * icc194 n_cluster = n_individual * de195 n_clusters_needed = int(np.ceil(n_cluster / m))196197 print(f"\nICC={icc}, cluster size={m}:")198 print(f" Design effect: {de:.2f}")199 print(f" Total N needed: {int(np.ceil(n_cluster))}")200 print(f" Clusters needed: {n_clusters_needed}")201 print(f" Inflation: {de:.2f}x individual design")202203 print("\nKey lesson: Higher ICC or larger clusters → much larger N needed")204205 return ()206```207</code>208<lesson>209Design effect = 1 + (m-1) × ICC. With ICC=0.10 and m=30, DE=3.9, meaning you need 4x the individual-randomized sample size. Cluster randomization trades power for validity when spillovers are a concern.210</lesson>211</example>212213</examples>214215<common_mistakes>216217<mistake severity="critical">218 <what>Conducting power analysis after data collection (post-hoc power)</what>219 <consequence>Meaningless exercise, doesn't inform anything, misinterpreted as evidence quality</consequence>220 <prevention>Power analysis is for design phase only. Never calculate "observed power" after study</prevention>221</mistake>222223<mistake severity="critical">224 <what>Ignoring clustering design effect in sample size calculation</what>225 <consequence>Study underpowered by 2-4x, fails to detect real effects despite large nominal sample</consequence>226 <prevention>ALWAYS multiply by design effect DE = 1+(m-1)×ICC for cluster randomization</prevention>227</mistake>228229<mistake severity="high">230 <what>Using overoptimistic effect size assumptions</what>231 <consequence>Study designed to detect implausibly large effects, fails for realistic effects</consequence>232 <prevention>Use conservative estimates from pilots, literature, or smallest policy-relevant effect</prevention>233</mistake>234235<mistake severity="high">236 <what>Not inflating for attrition and non-compliance</what>237 <consequence>Final analysis underpowered even if enrollment meets naive target</consequence>238 <prevention>Always inflate for expected attrition (÷ (1-attrition_rate)) and compliance (÷ compliance²)</prevention>239</mistake>240241<mistake severity="medium">242 <what>Treating power as binary threshold (80% good, 79% bad)</what>243 <consequence>Arbitrary decisions, missing that power is continuous and context-dependent</consequence>244 <prevention>Power is a continuum. Consider costs, effect importance, and Type I vs. II error tradeoffs</prevention>245</mistake>246247</common_mistakes>248249<interpretation_guide>250251<sample_size_rules_of_thumb>252For 80% power, α=0.05, 50/50 allocation:253- MDE = 0.10 SD: ~3,140 total254- MDE = 0.20 SD: ~786 total255- MDE = 0.30 SD: ~350 total256- MDE = 0.40 SD: ~198 total257- MDE = 0.50 SD: ~128 total258259Then multiply by:260- Design effect if clustering: 1+(m-1)×ICC261- 1/compliance² for non-compliance262- 1/(1-attrition_rate) for attrition263</sample_size_rules_of_thumb>264265<when_study_is_underpowered>266If power calculation shows infeasible N:267- Increase MDE (accept detecting only larger effects)268- Reduce scope (fewer outcomes, simpler design)269- Improve efficiency (stratification, ANCOVA with baseline)270- Pool with other studies (consortium approach)271- Use alternative design (regression discontinuity, DID)272- Accept higher Type II error risk (document explicitly)273274Do NOT: proceed with underpowered study and hope for best.275</when_study_is_underpowered>276277</interpretation_guide>278279<references>280<paper>Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Erlbaum.</paper>281<paper>Duflo, E., Glennerster, R., & Kremer, M. (2007). Using randomization in development economics research. Handbook of Development Economics, 4, 3895-3962.</paper>282<paper>Donner, A., & Klar, N. (2000). Design and Analysis of Cluster Randomization Trials in Health Research. Arnold Publishers.</paper>283<paper>Hoenig, J. M., & Heisey, D. M. (2001). The abuse of power: The pervasive fallacy of power calculations for data analysis. American Statistician, 55(1), 19-24.</paper>284<paper>Vivalt, E. (2020). How much can we generalize from impact evaluations? Journal of the European Economic Association, 18(6), 3045-3089.</paper>285</references>286287</skill_content>