Unit Interference Experimental Design Lens
Philosophical Mode: Causal-Structural
Primary Question: "What is the unit, and can treatments spill over?"
Focus: Experimental Unit, Cluster Structure, Shared Resources, Network Effects, SUTVA Violations
Arguments
/autoskillit:exp-lens-unit-interference [context_path] [experiment_plan_path]
- context_path (optional positional arg 1) — Absolute path to a lens context file
containing IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria.
If provided, read this file before beginning analysis to obtain structured context.
If omitted, discover context by exploring the CWD.
- experiment_plan_path (optional positional arg 2) — Absolute path to the full
experiment plan. If provided, read for complete experimental methodology and design.
If omitted, locate the experiment plan by exploring the CWD.
When to Use
- Online A/B tests with shared infrastructure
- Distributed systems where units share caches, queues, or services
- Social or network experiments where units are connected
- User invokes
/autoskillit:exp-lens-unit-interference or /autoskillit:make-experiment-diag unit
Critical Constraints
NEVER:
- Modify any source code files
- Do not litter the codebase with useless comments, TODO markers, or explanatory annotations — the skill output and diagram speak for themselves
- Create files outside
{{AUTOSKILLIT_TEMP}}/exp-lens-unit-interference/
- Run subagents in the background (
run_in_background: true is prohibited)
ALWAYS:
Focus on the unit definition and whether SUTVA is plausible
Map the full unit-cluster-resource hierarchy before assessing interference
Identify every shared resource that could transmit treatment effects across groups
Distinguish direct spillover (shared cache) from indirect spillover (market equilibrium)
BEFORE creating any diagram, LOAD the /autoskillit:mermaid skill using the Skill tool - this is MANDATORY
If the Skill tool cannot be used (disable-model-invocation) or refuses this invocation, do NOT proceed with diagram creation. Abort this step and omit the diagram from output.
Write output to {{AUTOSKILLIT_TEMP}}/exp-lens-unit-interference/exp_diag_unit_interference_{YYYY-MM-DD_HHMMSS}.md
After writing the file, emit the structured output token as literal plain text with no
markdown formatting on the token name (the adjudicator performs a regex match):
diagram_path = /absolute/path/to/{{AUTOSKILLIT_TEMP}}/exp-lens-unit-interference/exp_diag_unit_interference_{...}.md
Analysis Workflow
Step 0: Parse optional arguments
If positional arg 1 (context_path) is provided and the file exists, read it to obtain
IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria. If positional
arg 2 (experiment_plan_path) is provided and exists, read the experiment plan for full
methodology. Use this structured context as the foundation for Steps 1-5; skip the CWD
exploration for these fields if the context file supplies them.
Step 1: Launch Parallel Exploration Subagents
Spawn Explore subagents to investigate:
Unit Definition
- Find what constitutes one experimental unit
- Is it a user, request, session, query, item, sample, trial, or instance?
- Look for: user, request, session, query, item, sample, trial, instance
Cluster & Group Structure
- Find groupings of units that might share treatment effects
- Identify natural clustering that predates treatment assignment
- Look for: cluster, group, shard, server, region, batch, household, team
Shared Resources
- Find infrastructure shared across treatment groups
- Identify components both treatment and control units touch
- Look for: cache, queue, pool, database, service, load_balancer, gpu, memory
Network & Social Connections
- Find connections between units that could transmit treatment effects
- Identify paths by which a treated unit could alter a control unit's experience
- Look for: network, graph, friend, neighbor, link, message, recommend, influence
Treatment Assignment Boundary
- Find where the treatment boundary is drawn
- Identify whether the assignment is at the unit level or a coarser level
- Look for: bucket, hash, experiment_id, variant, flag, feature_flag, rollout
Step 2: Map the Unit-Cluster-Resource Hierarchy
For each level of the hierarchy:
- Can treatment at one level affect outcomes at another?
- Identify specific spillover pathways between levels
- Assess whether SUTVA (stable unit treatment value assumption) is plausible at each level
Document:
- Unit Level: The atomic entity receiving treatment
- Cluster Level: Natural groupings of units with shared context
- System Level: Infrastructure shared across all groups
Step 3: Analyze Interference Pathways
CRITICAL — Analyze Interference Pathways:
For every shared resource or connection:
- Could treatment group A's behavior change the experience of control group B?
- Is this interference direct (shared cache hit rates) or indirect (market-level equilibrium effects)?
- What is the likely magnitude: negligible, moderate, or invalidating?
- Is there a mitigation strategy (cluster-level randomization, holdout, depletion correction)?
Rate each pathway:
- HIGH: Interference almost certainly contaminates the control group
- MEDIUM: Plausible interference under realistic usage patterns
- LOW: Theoretical but unlikely to affect measured outcomes
Step 4: Create the Diagram
Use flowchart with:
Direction: TB (units nested within clusters nested within the system)
Subgraphs:
- "EXPERIMENTAL UNITS" (the atomic entities being randomized)
- "CLUSTER STRUCTURE" (groupings above the unit level)
- "SHARED RESOURCES" (infrastructure accessible by both groups)
- "INTERFERENCE PATHWAYS" (explicit spillover routes)
Node Styling:
cli class: Experimental units
phase class: Cluster / group nodes
stateNode class: Shared resources
gap class: Interference pathways
handler class: Treatment assignment
detector class: SUTVA boundary
Step 5: Write Output
Write the diagram to: {{AUTOSKILLIT_TEMP}}/exp-lens-unit-interference/exp_diag_unit_interference_{YYYY-MM-DD_HHMMSS}.md (relative to the current working directory)
Pre-Diagram Checklist
Before creating the diagram, verify:
Related Skills
/autoskillit:make-experiment-diag - Parent skill for experimental lens selection
/autoskillit:mermaid - MUST BE LOADED before creating diagram
/autoskillit:exp-lens-causal-assumptions - For DAG-level causal structure analysis
/autoskillit:exp-lens-randomization-blocking - For randomization strategy and blocking design
1---2name: exp-lens-unit-interference3categories: [exp-lens]4activate_deps: [mermaid]5description: Create Unit Interference experimental design diagram showing unit hierarchy, cluster structure, shared resources, and SUTVA violation pathways. Causal-Structural lens answering "What is the unit, and can treatments spill over?"6hooks:7 PreToolUse:8 - matcher: "*"9 hooks:10 - type: command11 command: "echo 'Unit Interference Lens - Analyzing experimental units and spillover...'"12 once: true13metadata:14 source: https://skillmd.com/skills/talont-org/exp-lens-unit-interference15---1617# Unit Interference Experimental Design Lens1819**Philosophical Mode:** Causal-Structural20**Primary Question:** "What is the unit, and can treatments spill over?"21**Focus:** Experimental Unit, Cluster Structure, Shared Resources, Network Effects, SUTVA Violations2223## Arguments2425`/autoskillit:exp-lens-unit-interference [context_path] [experiment_plan_path]`2627- **context_path** (optional positional arg 1) — Absolute path to a lens context file28 containing IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria.29 If provided, read this file before beginning analysis to obtain structured context.30 If omitted, discover context by exploring the CWD.31- **experiment_plan_path** (optional positional arg 2) — Absolute path to the full32 experiment plan. If provided, read for complete experimental methodology and design.33 If omitted, locate the experiment plan by exploring the CWD.3435## When to Use3637- Online A/B tests with shared infrastructure38- Distributed systems where units share caches, queues, or services39- Social or network experiments where units are connected40- User invokes `/autoskillit:exp-lens-unit-interference` or `/autoskillit:make-experiment-diag unit`4142## Critical Constraints4344**NEVER:**45- Modify any source code files46- Do not litter the codebase with useless comments, TODO markers, or explanatory annotations — the skill output and diagram speak for themselves47- Create files outside `{{AUTOSKILLIT_TEMP}}/exp-lens-unit-interference/`48- Run subagents in the background (`run_in_background: true` is prohibited)4950**ALWAYS:**51- Focus on the unit definition and whether SUTVA is plausible52- Map the full unit-cluster-resource hierarchy before assessing interference53- Identify every shared resource that could transmit treatment effects across groups54- Distinguish direct spillover (shared cache) from indirect spillover (market equilibrium)55- BEFORE creating any diagram, LOAD the `/autoskillit:mermaid` skill using the Skill tool - this is MANDATORY56- If the Skill tool cannot be used (disable-model-invocation) or refuses this invocation, do NOT proceed with diagram creation. Abort this step and omit the diagram from output.57- Write output to `{{AUTOSKILLIT_TEMP}}/exp-lens-unit-interference/exp_diag_unit_interference_{YYYY-MM-DD_HHMMSS}.md`58- After writing the file, emit the structured output token as **literal plain text** with no59 markdown formatting on the token name (the adjudicator performs a regex match):6061 ```62 diagram_path = /absolute/path/to/{{AUTOSKILLIT_TEMP}}/exp-lens-unit-interference/exp_diag_unit_interference_{...}.md63 ```6465---6667## Analysis Workflow6869### Step 0: Parse optional arguments7071If positional arg 1 (context_path) is provided and the file exists, read it to obtain72IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria. If positional73arg 2 (experiment_plan_path) is provided and exists, read the experiment plan for full74methodology. Use this structured context as the foundation for Steps 1-5; skip the CWD75exploration for these fields if the context file supplies them.7677### Step 1: Launch Parallel Exploration Subagents7879Spawn Explore subagents to investigate:8081**Unit Definition**82- Find what constitutes one experimental unit83- Is it a user, request, session, query, item, sample, trial, or instance?84- Look for: user, request, session, query, item, sample, trial, instance8586**Cluster & Group Structure**87- Find groupings of units that might share treatment effects88- Identify natural clustering that predates treatment assignment89- Look for: cluster, group, shard, server, region, batch, household, team9091**Shared Resources**92- Find infrastructure shared across treatment groups93- Identify components both treatment and control units touch94- Look for: cache, queue, pool, database, service, load_balancer, gpu, memory9596**Network & Social Connections**97- Find connections between units that could transmit treatment effects98- Identify paths by which a treated unit could alter a control unit's experience99- Look for: network, graph, friend, neighbor, link, message, recommend, influence100101**Treatment Assignment Boundary**102- Find where the treatment boundary is drawn103- Identify whether the assignment is at the unit level or a coarser level104- Look for: bucket, hash, experiment_id, variant, flag, feature_flag, rollout105106### Step 2: Map the Unit-Cluster-Resource Hierarchy107108For each level of the hierarchy:109- Can treatment at one level affect outcomes at another?110- Identify specific spillover pathways between levels111- Assess whether SUTVA (stable unit treatment value assumption) is plausible at each level112113Document:114- **Unit Level**: The atomic entity receiving treatment115- **Cluster Level**: Natural groupings of units with shared context116- **System Level**: Infrastructure shared across all groups117118### Step 3: Analyze Interference Pathways119120**CRITICAL — Analyze Interference Pathways:**121For every shared resource or connection:122- Could treatment group A's behavior change the experience of control group B?123- Is this interference direct (shared cache hit rates) or indirect (market-level equilibrium effects)?124- What is the likely magnitude: negligible, moderate, or invalidating?125- Is there a mitigation strategy (cluster-level randomization, holdout, depletion correction)?126127Rate each pathway:128- **HIGH**: Interference almost certainly contaminates the control group129- **MEDIUM**: Plausible interference under realistic usage patterns130- **LOW**: Theoretical but unlikely to affect measured outcomes131132### Step 4: Create the Diagram133134Use flowchart with:135136**Direction:** `TB` (units nested within clusters nested within the system)137138**Subgraphs:**139- "EXPERIMENTAL UNITS" (the atomic entities being randomized)140- "CLUSTER STRUCTURE" (groupings above the unit level)141- "SHARED RESOURCES" (infrastructure accessible by both groups)142- "INTERFERENCE PATHWAYS" (explicit spillover routes)143144**Node Styling:**145- `cli` class: Experimental units146- `phase` class: Cluster / group nodes147- `stateNode` class: Shared resources148- `gap` class: Interference pathways149- `handler` class: Treatment assignment150- `detector` class: SUTVA boundary151152### Step 5: Write Output153154Write the diagram to: `{{AUTOSKILLIT_TEMP}}/exp-lens-unit-interference/exp_diag_unit_interference_{YYYY-MM-DD_HHMMSS}.md` (relative to the current working directory)155156---157158## Pre-Diagram Checklist159160Before creating the diagram, verify:161162- [ ] LOADED `/autoskillit:mermaid` skill using the Skill tool163- [ ] Using ONLY classDef styles from the mermaid skill (no invented colors)164- [ ] Diagram will include a color legend table165166---167168## Related Skills169170- `/autoskillit:make-experiment-diag` - Parent skill for experimental lens selection171- `/autoskillit:mermaid` - MUST BE LOADED before creating diagram172- `/autoskillit:exp-lens-causal-assumptions` - For DAG-level causal structure analysis173- `/autoskillit:exp-lens-randomization-blocking` - For randomization strategy and blocking design