Exp Lens Randomization Blocking

Randomization & Blocking Experimental Design Lens

Philosophical Mode: Design-Structural Primary Question: "Where does comparability come from?" Focus: Assignment Mechanisms, Blocking Factors, Stratification, Balanced Designs, Replication

When to Use

• Experiment uses randomization or structured assignment
• Need to verify blocking and stratification
• Checking for pseudoreplication
• User invokes /exp-lens-randomization-blocking or /make-experiment-diag randomization

Critical Constraints

NEVER:

• Modify any source code files
• Do not litter the codebase with useless comments, TODO markers, or explanatory annotations — the skill output and diagram speak for themselves
• Assume comparability without tracing its source

ALWAYS:

• Trace the exact mechanism that creates comparability between treatment groups
• Identify every nuisance factor and how it is controlled
• Flag pseudoreplication risks (replicating at the wrong unit)
• Verify that replication is adequate for the claimed inferential precision
• BEFORE creating any diagram, LOAD the /mermaid skill using the Skill tool - this is MANDATORY

Analysis Workflow

Step 1: Launch Parallel Exploration Subagents

Spawn Explore subagents to investigate:

Assignment Mechanism

• Find how units are assigned to conditions
• Look for: random, assign, shuffle, permute, allocate, condition, arm, group

Blocking & Stratification

• Find blocking variables and stratification logic
• Look for: block, stratify, batch, fold, group_by, cluster, site

Replication Structure

• Find how many runs/replicates per condition
• Look for: replicate, repeat, n_runs, n_trials, seed_list, iteration

Order & Timing Effects

• Find whether order of execution matters
• Look for: order, sequence, schedule, first, last, warm_up, cold_start

Exclusion & Attrition

• Find unit exclusion or retry logic
• Look for: exclude, drop, filter, retry, timeout, fail, skip

Step 2: Map the Allocation Flow

Map the complete allocation flow from population of units through assignment to analysis. Identify:

• What is the randomization unit?
• What blocking factors are used?
• Is replication adequate?
• Are there confounds between treatment and nuisance factors (time, hardware, order)?

Step 3: CRITICAL — Analyze Comparability Source

For every treatment contrast:

• Comparability mechanism: What makes the groups comparable — randomization, matching, blocking, or assumption?
• Randomization quality: If randomization — is it truly random or deterministic-with-seed? Could seed choice affect results?
• Block crossing: If blocked — do blocks fully cross with treatments, or are some treatments confounded with blocks?
• Replication level: Is the replication unit the same as the randomization unit? If not, pseudoreplication risk exists.
• Temporal confounds: Could time-of-execution differences (cache state, load, ordering) introduce systematic bias?

Distinguish clearly:

• True randomization: Independent draws, unconfounded with nuisance
• Blocked randomization: Randomization within homogeneous blocks
• Matched pairs: Units matched on confounders before assignment
• Deterministic assignment: Fixed seed or rotation — not truly random

Step 4: Create the Diagram

Use flowchart with:

Direction: TB (population flows down through allocation to analysis)

Subgraphs:

• POPULATION/POOL
• BLOCKING
• RANDOMIZATION
• TREATMENT ARMS
• ANALYSIS

Node Styling:

• cli class: population/pool entry
• phase class: blocking/stratification nodes
• handler class: randomization mechanism
• stateNode class: treatment arm conditions
• output class: analysis endpoints
• gap class: pseudoreplication or confound risks
• detector class: balance checks

Step 5: Write Output

Write the diagram to: temp/exp-lens-randomization-blocking/exp_diag_randomization_blocking_{YYYY-MM-DD_HHMMSS}.md

Output Template

# Randomization & Blocking Diagram: {System Name}

**Lens:** Randomization & Blocking (Design-Structural)
**Question:** Where does comparability come from?
**Date:** {YYYY-MM-DD}
**Scope:** {What was analyzed}

## Design Structure

| Factor | Levels | Type | Crossed/Nested | Notes |
|--------|--------|------|----------------|-------|
| {Factor 1} | {levels} | Treatment | Crossed | {notes} |
| {Factor 2} | {levels} | Block | Nested | {notes} |
| {Factor 3} | {levels} | Nuisance | Controlled | {notes} |

## Randomization & Blocking Diagram

```mermaid
%%{init: {'flowchart': {'nodeSpacing': 50, 'rankSpacing': 60, 'curve': 'basis'}}}%%
flowchart TB
    %% CLASS DEFINITIONS %%
    classDef cli fill:#1a237e,stroke:#7986cb,stroke-width:2px,color:#fff;
    classDef stateNode fill:#004d40,stroke:#4db6ac,stroke-width:2px,color:#fff;
    classDef handler fill:#e65100,stroke:#ffb74d,stroke-width:2px,color:#fff;
    classDef phase fill:#6a1b9a,stroke:#ba68c8,stroke-width:2px,color:#fff;
    classDef newComponent fill:#2e7d32,stroke:#81c784,stroke-width:2px,color:#fff;
    classDef output fill:#00695c,stroke:#4db6ac,stroke-width:2px,color:#fff;
    classDef detector fill:#b71c1c,stroke:#ef5350,stroke-width:2px,color:#fff;
    classDef gap fill:#ff6f00,stroke:#ffa726,stroke-width:2px,color:#000;
    classDef integration fill:#c62828,stroke:#ef9a9a,stroke-width:2px,color:#fff;

    subgraph Pool ["POPULATION/POOL"]
        direction TB
        UNITS["{Unit Type}<br/>━━━━━━━━━━<br/>N={total units}"]
    end

    subgraph Blocking ["BLOCKING"]
        direction TB
        BLK1["{Block Factor 1}<br/>━━━━━━━━━━<br/>{levels}"]
        BLK2["{Block Factor 2}<br/>━━━━━━━━━━<br/>{levels}"]
    end

    subgraph Randomization ["RANDOMIZATION"]
        direction TB
        RAND["{Assignment Mechanism}<br/>━━━━━━━━━━<br/>{random/deterministic/matched}"]
    end

    subgraph Arms ["TREATMENT ARMS"]
        direction TB
        ARM1["{Treatment A}<br/>━━━━━━━━━━<br/>n={replicates}"]
        ARM2["{Treatment B}<br/>━━━━━━━━━━<br/>n={replicates}"]
        CTRL["{Control}<br/>━━━━━━━━━━<br/>n={replicates}"]
    end

    subgraph Analysis ["ANALYSIS"]
        direction TB
        ENDPOINT["{Primary Endpoint}<br/>━━━━━━━━━━<br/>{estimand}"]
        BALANCE["{Balance Check}<br/>━━━━━━━━━━<br/>Verify comparability"]
    end

    PSEUDO["{Pseudoreplication Risk}<br/>━━━━━━━━━━<br/>{unit mismatch description}"]
    CONFOUND["{Confound Risk}<br/>━━━━━━━━━━<br/>{nuisance factor description}"]

    UNITS --> BLK1
    UNITS --> BLK2
    BLK1 --> RAND
    BLK2 --> RAND
    RAND --> ARM1
    RAND --> ARM2
    RAND --> CTRL
    ARM1 --> ENDPOINT
    ARM2 --> ENDPOINT
    CTRL --> ENDPOINT
    ENDPOINT --> BALANCE
    PSEUDO -.->|risk| RAND
    CONFOUND -.->|risk| ARM1

    %% CLASS ASSIGNMENTS %%
    class UNITS cli;
    class BLK1,BLK2 phase;
    class RAND handler;
    class ARM1,ARM2,CTRL stateNode;
    class ENDPOINT,BALANCE output;
    class PSEUDO,CONFOUND gap;
    class BALANCE detector;

Color Legend:

Comparability Assessment

Replication Summary

---

## Pre-Diagram Checklist

Before creating the diagram, verify:

- [ ] LOADED `/mermaid` skill using the Skill tool
- [ ] Using ONLY classDef styles from the mermaid skill (no invented colors)
- [ ] Diagram will include a color legend table

---

## Related Skills

- `/make-experiment-diag` - Parent skill for lens selection
- `/mermaid` - MUST BE LOADED before creating diagram
- `/exp-lens-causal-assumptions` - For causal structure analysis
- `/exp-lens-unit-interference` - For interference and spillover risks