Exp Lens Unit Interference

Unit Interference Experimental Design Lens

Philosophical Mode: Causal-Structural Primary Question: "What is the unit, and can treatments spill over?" Focus: Experimental Unit, Cluster Structure, Shared Resources, Network Effects, SUTVA Violations

When to Use

• Online A/B tests with shared infrastructure
• Distributed systems where units share caches, queues, or services
• Social or network experiments where units are connected
• User invokes /exp-lens-unit-interference or /make-experiment-diag unit

Critical Constraints

NEVER:

• Modify any source code files
• Do not litter the codebase with useless comments, TODO markers, or explanatory annotations — the skill output and diagram speak for themselves

ALWAYS:

• Focus on the unit definition and whether SUTVA is plausible
• Map the full unit-cluster-resource hierarchy before assessing interference
• Identify every shared resource that could transmit treatment effects across groups
• Distinguish direct spillover (shared cache) from indirect spillover (market equilibrium)
• BEFORE creating any diagram, LOAD the /mermaid skill using the Skill tool - this is MANDATORY

Analysis Workflow

Step 1: Launch Parallel Exploration Subagents

Spawn Explore subagents to investigate:

Unit Definition

• Find what constitutes one experimental unit
• Is it a user, request, session, query, item, sample, trial, or instance?
• Look for: user, request, session, query, item, sample, trial, instance

Cluster & Group Structure

• Find groupings of units that might share treatment effects
• Identify natural clustering that predates treatment assignment
• Look for: cluster, group, shard, server, region, batch, household, team

Shared Resources

• Find infrastructure shared across treatment groups
• Identify components both treatment and control units touch
• Look for: cache, queue, pool, database, service, load_balancer, gpu, memory

Network & Social Connections

• Find connections between units that could transmit treatment effects
• Identify paths by which a treated unit could alter a control unit's experience
• Look for: network, graph, friend, neighbor, link, message, recommend, influence

Treatment Assignment Boundary

• Find where the treatment boundary is drawn
• Identify whether the assignment is at the unit level or a coarser level
• Look for: bucket, hash, experiment_id, variant, flag, feature_flag, rollout

Step 2: Map the Unit-Cluster-Resource Hierarchy

For each level of the hierarchy:

• Can treatment at one level affect outcomes at another?
• Identify specific spillover pathways between levels
• Assess whether SUTVA (stable unit treatment value assumption) is plausible at each level

Document:

• Unit Level: The atomic entity receiving treatment
• Cluster Level: Natural groupings of units with shared context
• System Level: Infrastructure shared across all groups

Step 3: Analyze Interference Pathways

CRITICAL — Analyze Interference Pathways: For every shared resource or connection:

• Could treatment group A's behavior change the experience of control group B?
• Is this interference direct (shared cache hit rates) or indirect (market-level equilibrium effects)?
• What is the likely magnitude: negligible, moderate, or invalidating?
• Is there a mitigation strategy (cluster-level randomization, holdout, depletion correction)?

Rate each pathway:

• HIGH: Interference almost certainly contaminates the control group
• MEDIUM: Plausible interference under realistic usage patterns
• LOW: Theoretical but unlikely to affect measured outcomes

Step 4: Create the Diagram

Use flowchart with:

Direction: TB (units nested within clusters nested within the system)

Subgraphs:

• "EXPERIMENTAL UNITS" (the atomic entities being randomized)
• "CLUSTER STRUCTURE" (groupings above the unit level)
• "SHARED RESOURCES" (infrastructure accessible by both groups)
• "INTERFERENCE PATHWAYS" (explicit spillover routes)

Node Styling:

• cli class: Experimental units
• phase class: Cluster / group nodes
• stateNode class: Shared resources
• gap class: Interference pathways
• handler class: Treatment assignment
• detector class: SUTVA boundary

Step 5: Write Output

Write the diagram to: temp/exp-lens-unit-interference/exp_diag_unit_interference_{YYYY-MM-DD_HHMMSS}.md

Output Template

# Unit Interference Diagram: {System / Experiment Name}

**Lens:** Unit Interference (Causal-Structural)
**Question:** What is the unit, and can treatments spill over?
**Date:** {YYYY-MM-DD}
**Scope:** {What was analyzed}

## Unit Hierarchy

| Level | Count | Shared Resources | Interference Risk |
|-------|-------|-----------------|-------------------|
| {unit level} | {N} | {resources} | {LOW / MEDIUM / HIGH} |
| {cluster level} | {N} | {resources} | {LOW / MEDIUM / HIGH} |
| {system level} | {N} | {resources} | {LOW / MEDIUM / HIGH} |

## Unit-Cluster Diagram

```mermaid
%%{init: {'flowchart': {'nodeSpacing': 40, 'rankSpacing': 50, 'curve': 'basis'}}}%%
flowchart TB
    %% CLASS DEFINITIONS %%
    classDef cli fill:#1a237e,stroke:#7986cb,stroke-width:2px,color:#fff;
    classDef stateNode fill:#004d40,stroke:#4db6ac,stroke-width:2px,color:#fff;
    classDef handler fill:#e65100,stroke:#ffb74d,stroke-width:2px,color:#fff;
    classDef phase fill:#6a1b9a,stroke:#ba68c8,stroke-width:2px,color:#fff;
    classDef newComponent fill:#2e7d32,stroke:#81c784,stroke-width:2px,color:#fff;
    classDef output fill:#00695c,stroke:#4db6ac,stroke-width:2px,color:#fff;
    classDef detector fill:#b71c1c,stroke:#ef5350,stroke-width:2px,color:#fff;
    classDef gap fill:#ff6f00,stroke:#ffa726,stroke-width:2px,color:#000;
    classDef integration fill:#c62828,stroke:#ef9a9a,stroke-width:2px,color:#fff;

    subgraph Assignment ["TREATMENT ASSIGNMENT"]
        ASSIGN["Assignment Logic<br/>━━━━━━━━━━<br/>hash / flag / bucket"]
        SUTVA{"SUTVA<br/>Boundary<br/>plausible?"}
    end

    subgraph Units ["EXPERIMENTAL UNITS"]
        direction TB
        TREAT["Treatment Units<br/>━━━━━━━━━━<br/>receive variant"]
        CTRL["Control Units<br/>━━━━━━━━━━<br/>receive baseline"]
    end

    subgraph Clusters ["CLUSTER STRUCTURE"]
        direction TB
        CL_T["Treatment Cluster<br/>━━━━━━━━━━<br/>shared context"]
        CL_C["Control Cluster<br/>━━━━━━━━━━<br/>shared context"]
    end

    subgraph Shared ["SHARED RESOURCES"]
        direction TB
        RES1["Shared Cache<br/>━━━━━━━━━━<br/>cross-group access"]
        RES2["Shared Queue<br/>━━━━━━━━━━<br/>cross-group access"]
    end

    subgraph Interference ["INTERFERENCE PATHWAYS"]
        direction TB
        PATH1["Spillover Path 1<br/>━━━━━━━━━━<br/>mechanism + magnitude"]
        PATH2["Spillover Path 2<br/>━━━━━━━━━━<br/>mechanism + magnitude"]
    end

    %% ASSIGNMENT %%
    ASSIGN --> TREAT
    ASSIGN --> CTRL
    ASSIGN --> SUTVA

    %% CLUSTER NESTING %%
    TREAT --> CL_T
    CTRL --> CL_C

    %% SHARED RESOURCE ACCESS %%
    CL_T --> RES1
    CL_C --> RES1
    CL_T --> RES2
    CL_C --> RES2

    %% INTERFERENCE %%
    RES1 -.->|"spillover"| PATH1
    RES2 -.->|"spillover"| PATH2
    PATH1 -.->|"contaminates"| CTRL
    PATH2 -.->|"contaminates"| CTRL

    %% CLASS ASSIGNMENTS %%
    class TREAT,CTRL cli;
    class CL_T,CL_C phase;
    class RES1,RES2 stateNode;
    class PATH1,PATH2 gap;
    class ASSIGN handler;
    class SUTVA detector;

Color Legend:

Interference Pathway Analysis

SUTVA Assessment

• Independence assumption: {plausible / violated / unknown}
• Identified violations: {list specific pathways or "none detected"}
• Recommended mitigation: {cluster randomization / geo holdout / none needed / further investigation}

---

## Pre-Diagram Checklist

Before creating the diagram, verify:

- [ ] LOADED `/mermaid` skill using the Skill tool
- [ ] Using ONLY classDef styles from the mermaid skill (no invented colors)
- [ ] Diagram will include a color legend table

---

## Related Skills

- `/make-experiment-diag` - Parent skill for experimental lens selection
- `/mermaid` - MUST BE LOADED before creating diagram
- `/exp-lens-causal-assumptions` - For DAG-level causal structure analysis
- `/exp-lens-randomization-blocking` - For randomization strategy and blocking design