Randomization must be reproducible, documented, and executed exactly once per study.
import numpy as np
import pandas as pd
from datetime import datetime
# Set seed ONCE at beginning (DOCUMENT THIS VALUE)
RANDOM_SEED = 42
np.random.seed(RANDOM_SEED)
# Load eligible sample
df = pd.read_csv("/mnt/data/eligible_units.csv")
n = len(df)
print(f"RANDOMIZATION PROTOCOL")
print(f"Date: {datetime.now()}")
print(f"Random seed: {RANDOM_SEED}")
print(f"Sample size: {n}")
# Stratified permutation randomization
treatment_prop = 0.5
strata_var = 'baseline_risk' # Strong outcome predictor
df['treatment'] = 0
for stratum in df[strata_var].unique():
strata_mask = df[strata_var] == stratum
n_strata = strata_mask.sum()
n_treat = int(n_strata * treatment_prop)
# Permutation within stratum
assignments = np.concatenate([
np.ones(n_treat, dtype=int),
np.zeros(n_strata - n_treat, dtype=int)
])
np.random.shuffle(assignments)
df.loc[strata_mask, 'treatment'] = assignments
# CRITICAL: Save immediately (NEVER re-run randomization)
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
df.to_csv(f"/mnt/data/randomization_assignments_{timestamp}.csv", index=False)
# Create backup
df.to_csv(f"/mnt/data/randomization_assignments_BACKUP.csv", index=False)
print(f"\nAssignments saved to:")
print(f" /mnt/data/randomization_assignments_{timestamp}.csv")
print(f"Treatment: {df['treatment'].sum()} ({df['treatment'].mean():.1%})")
print(f"Control: {(~df['treatment'].astype(bool)).sum()} ({(~df['treatment'].astype(bool)).mean():.1%})")
return df,
</code_template>
</implementation_pattern>
<examples>
<example context="cluster_randomization" difficulty="intermediate">
<description>Cluster randomization to prevent spillovers</description>
<code>
```python
@app.cell
def cluster_randomize(df):
# Cluster randomization when spillovers likely within clusters
# Reduces power due to design effect, but necessary for validity
import numpy as np
import pandas as pd
# CRITICAL: Set seed once
np.random.seed(123)
# Identify clusters
clusters = df['village_id'].unique()
n_clusters = len(clusters)
n_treat_clusters = int(n_clusters * 0.5)
print(f"CLUSTER RANDOMIZATION")
print(f"Total clusters: {n_clusters}")
print(f"Treatment clusters: {n_treat_clusters}")
# Permutation at cluster level
cluster_assignments = np.concatenate([
np.ones(n_treat_clusters, dtype=int),
np.zeros(n_clusters - n_treat_clusters, dtype=int)
])
np.random.shuffle(cluster_assignments)
# Map cluster assignments to individuals
cluster_map = dict(zip(clusters, cluster_assignments))
df['treatment'] = df['village_id'].map(cluster_map)
# Report
print(f"\nIndividuals in treatment: {df['treatment'].sum()}")
print(f"Individuals in control: {(~df['treatment'].astype(bool)).sum()}")
# Calculate design effect
avg_cluster_size = df.groupby('village_id').size().mean()
icc = 0.05 # Assumed intra-cluster correlation
design_effect = 1 + (avg_cluster_size - 1) * icc
print(f"\nAverage cluster size: {avg_cluster_size:.1f}")
print(f"Assumed ICC: {icc:.3f}")
print(f"Design effect: {design_effect:.2f}")
print(f"Effective sample size: {len(df)/design_effect:.0f}")
return df,
Never re-randomize based on balance. If you must, use proper re-randomization protocol with inference adjustment (complex).
1---2name: rct-randomization3description: Implement randomization methods for RCTs. Use when user mentions: random assignment, treatment allocation, stratification, blocking, permutation randomization, cluster randomization, randomization balance, misfit handling, re-randomization.4---56<skill_content>78<overview>9Randomization is the foundation of experimental causal inference. Proper implementation requires setting seeds once, documenting procedures, saving assignments immediately, and never re-running randomization to "improve" balance. Stratification improves precision when strata correlate with outcomes. Cluster randomization prevents spillovers but reduces power through design effects.1011Randomization must be reproducible, documented, and executed exactly once per study.12</overview>1314<mandatory_requirements>1516<requirement priority="critical">17 <name>Set Seed Once and Document</name>18 <description>Set random seed exactly ONCE at the beginning, document seed value, never reset seed within randomization code</description>19 <rationale>Multiple seed settings compromise randomness. Seed documentation enables replication and verification of randomization integrity</rationale>20 <consequence>Non-reproducible randomization, suspicion of manipulation, inability to verify implementation</consequence>21</requirement>2223<requirement priority="critical">24 <name>Save Assignment Immediately, Never Re-Run</name>25 <description>Save randomization assignments to persistent storage immediately after generation, never re-run randomization</description>26 <rationale>Re-running randomization to "improve balance" invalidates inference and creates selection bias</rationale>27 <consequence>Invalid p-values, corrupted randomization, inability to claim true random assignment</consequence>28</requirement>2930<requirement priority="critical">31 <name>Use Permutation (Not Simple) When Sample Size Known</name>32 <description>Use permutation randomization (shuffle fixed number of 1s and 0s) rather than independent Bernoulli draws when N is known</description>33 <rationale>Permutation ensures exact allocation ratios. Simple randomization creates sampling variation in treatment/control split</rationale>34 <consequence>Unbalanced groups, power loss, inability to explain allocation discrepancies</consequence>35</requirement>3637<requirement priority="high">38 <name>Stratify on Strong Predictors When Available</name>39 <description>Use stratified randomization when baseline covariates strongly predict outcomes</description>40 <rationale>Stratification ensures balance on key variables, improves precision, reduces sampling variability (Bruhn & McKenzie 2009)</rationale>41 <consequence>Larger standard errors, power loss, imbalanced groups on important characteristics</consequence>42</requirement>4344<requirement priority="high">45 <name>Account for Clustering in Randomization Level</name>46 <description>Randomize at cluster level when spillovers likely within clusters</description>47 <rationale>Individual randomization with spillovers violates SUTVA and biases estimates. Cluster randomization prevents contamination</rationale>48 <consequence>Biased treatment effect estimates from spillover contamination</consequence>49</requirement>5051</mandatory_requirements>5253<assumptions>5455<assumption name="No Selective Exclusion">56 <description>Sample for randomization represents full eligible population, no post-randomization exclusions</description>57 <how_to_check>Document exclusions before randomization, never drop units after seeing assignment</how_to_check>58 <if_violated>Selection bias, non-random sample, invalid causal inference</if_violated>59</assumption>6061<assumption name="Implementation Fidelity">62 <description>Treatment assignment is actually implemented as randomized (no crossover without documentation)</description>63 <how_to_check>Monitor compliance, track any protocol deviations</how_to_check>64 <if_violated>Document and report deviations, analyze as encouragement design if substantial crossover</if_violated>65</assumption>6667</assumptions>6869<thinking_process>70When implementing randomization:711. Define eligible sample BEFORE randomization722. Choose randomization method (simple/permutation/stratified/cluster)733. Set random seed ONCE, document seed value744. Generate assignments (never re-run)755. Save assignments to persistent storage immediately766. Create backup of assignments777. Check balance (but don't re-randomize if unbalanced)788. Document any implementation deviations79</thinking_process>8081<implementation_pattern>8283<code_template>84```python85@app.cell86def randomize_trial():87 # Permutation randomization with stratification88 # CRITICAL: Set seed ONCE, save immediately, NEVER re-run8990 import numpy as np91 import pandas as pd92 from datetime import datetime9394 # Set seed ONCE at beginning (DOCUMENT THIS VALUE)95 RANDOM_SEED = 4296 np.random.seed(RANDOM_SEED)9798 # Load eligible sample99 df = pd.read_csv("/mnt/data/eligible_units.csv")100 n = len(df)101102 print(f"RANDOMIZATION PROTOCOL")103 print(f"Date: {datetime.now()}")104 print(f"Random seed: {RANDOM_SEED}")105 print(f"Sample size: {n}")106107 # Stratified permutation randomization108 treatment_prop = 0.5109 strata_var = 'baseline_risk' # Strong outcome predictor110111 df['treatment'] = 0112113 for stratum in df[strata_var].unique():114 strata_mask = df[strata_var] == stratum115 n_strata = strata_mask.sum()116 n_treat = int(n_strata * treatment_prop)117118 # Permutation within stratum119 assignments = np.concatenate([120 np.ones(n_treat, dtype=int),121 np.zeros(n_strata - n_treat, dtype=int)122 ])123 np.random.shuffle(assignments)124125 df.loc[strata_mask, 'treatment'] = assignments126127 # CRITICAL: Save immediately (NEVER re-run randomization)128 timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")129 df.to_csv(f"/mnt/data/randomization_assignments_{timestamp}.csv", index=False)130131 # Create backup132 df.to_csv(f"/mnt/data/randomization_assignments_BACKUP.csv", index=False)133134 print(f"\nAssignments saved to:")135 print(f" /mnt/data/randomization_assignments_{timestamp}.csv")136 print(f"Treatment: {df['treatment'].sum()} ({df['treatment'].mean():.1%})")137 print(f"Control: {(~df['treatment'].astype(bool)).sum()} ({(~df['treatment'].astype(bool)).mean():.1%})")138139 return df,140```141</code_template>142143</implementation_pattern>144145<examples>146147<example context="cluster_randomization" difficulty="intermediate">148<description>Cluster randomization to prevent spillovers</description>149<code>150```python151@app.cell152def cluster_randomize(df):153 # Cluster randomization when spillovers likely within clusters154 # Reduces power due to design effect, but necessary for validity155156 import numpy as np157 import pandas as pd158159 # CRITICAL: Set seed once160 np.random.seed(123)161162 # Identify clusters163 clusters = df['village_id'].unique()164 n_clusters = len(clusters)165 n_treat_clusters = int(n_clusters * 0.5)166167 print(f"CLUSTER RANDOMIZATION")168 print(f"Total clusters: {n_clusters}")169 print(f"Treatment clusters: {n_treat_clusters}")170171 # Permutation at cluster level172 cluster_assignments = np.concatenate([173 np.ones(n_treat_clusters, dtype=int),174 np.zeros(n_clusters - n_treat_clusters, dtype=int)175 ])176 np.random.shuffle(cluster_assignments)177178 # Map cluster assignments to individuals179 cluster_map = dict(zip(clusters, cluster_assignments))180 df['treatment'] = df['village_id'].map(cluster_map)181182 # Report183 print(f"\nIndividuals in treatment: {df['treatment'].sum()}")184 print(f"Individuals in control: {(~df['treatment'].astype(bool)).sum()}")185186 # Calculate design effect187 avg_cluster_size = df.groupby('village_id').size().mean()188 icc = 0.05 # Assumed intra-cluster correlation189 design_effect = 1 + (avg_cluster_size - 1) * icc190191 print(f"\nAverage cluster size: {avg_cluster_size:.1f}")192 print(f"Assumed ICC: {icc:.3f}")193 print(f"Design effect: {design_effect:.2f}")194 print(f"Effective sample size: {len(df)/design_effect:.0f}")195196 return df,197```198</code>199<lesson>200Cluster randomization trades power for validity. Design effect = 1 + (m-1) × ICC where m is cluster size and ICC is intra-cluster correlation. Higher ICC or larger clusters → larger design effect → greater power loss. But necessary when spillovers would contaminate individual randomization.201</lesson>202</example>203204</examples>205206<common_mistakes>207208<mistake severity="critical">209 <what>Re-running randomization to get "better" balance</what>210 <consequence>Invalidates randomization inference, creates selection bias, corrupts p-values</consequence>211 <prevention>Run ONCE, save immediately, accept whatever balance you get. If deeply concerned about imbalance, use stratification or re-randomization with proper adjustment (difficult)</prevention>212</mistake>213214<mistake severity="critical">215 <what>Setting random seed multiple times in randomization code</what>216 <consequence>Compromises randomness, makes sequence non-random, creates reproducibility issues</consequence>217 <prevention>Set seed ONCE at very beginning, never call np.random.seed() again in randomization code</prevention>218</mistake>219220<mistake severity="high">221 <what>Using simple randomization when sample size is known</what>222 <consequence>Sampling variation creates unbalanced groups, power loss, difficult to explain discrepancies</consequence>223 <prevention>Use permutation randomization (shuffle fixed number of 1s and 0s) when N is known</prevention>224</mistake>225226<mistake severity="high">227 <what>Not stratifying when strong predictors available</what>228 <consequence>Unnecessary power loss, imbalanced groups on important characteristics</consequence>229 <prevention>Stratify on 1-3 strong outcome predictors to improve precision</prevention>230</mistake>231232<mistake severity="medium">233 <what>Not documenting randomization seed and procedure</what>234 <consequence>Non-reproducible, inability to verify implementation, suspicion of manipulation</consequence>235 <prevention>Document seed, date, time, code version, sample characteristics in protocol</prevention>236</mistake>237238</common_mistakes>239240<interpretation_guide>241242<balance_check_interpretation>243After randomization (but NOT as reason to re-randomize):244- Normalized diff |d| < 0.1: Excellent balance245- 0.1 ≤ |d| < 0.25: Acceptable balance246- |d| ≥ 0.25: Imbalanced on this variable (include as control, but don't re-randomize)247- Joint F-test p > 0.10: Overall balance adequate248- Joint F-test p < 0.05: Some imbalance (expected 5% of time by chance)249250Never re-randomize based on balance. If you must, use proper re-randomization protocol with inference adjustment (complex).251</balance_check_interpretation>252253<when_to_use_each_method>254- **Simple randomization**: Sequential enrollment, N unknown at randomization255- **Permutation**: N known, want exact allocation ratio256- **Stratified**: Strong baseline predictors available, want precision gains257- **Cluster**: Spillovers likely within clusters (social, geographic)258- **Re-randomization**: Advanced method requiring special inference (seek expert help)259</when_to_use_each_method>260261</interpretation_guide>262263<references>264<paper>Bruhn, M., & McKenzie, D. (2009). In pursuit of balance: Randomization in practice in development field experiments. American Economic Journal: Applied Economics, 1(4), 200-232.</paper>265<paper>Athey, S., & Imbens, G. W. (2017). The econometrics of randomized experiments. Handbook of Economic Field Experiments, 1, 73-140.</paper>266<paper>Duflo, E., Glennerster, R., & Kremer, M. (2007). Using randomization in development economics research. Handbook of Development Economics, 4, 3895-3962.</paper>267</references>268269</skill_content>