Iterative Learning Experimental Design Lens
Philosophical Mode: Decision-Theoretic
Primary Question: "How does this maximize learning per cost?"
Focus: Factor Selection, Interaction Probing, Adaptive Allocation, Stopping Rules, Next-Experiment Planning
Arguments
/autoskillit:exp-lens-iterative-learning [context_path] [experiment_plan_path]
- context_path (optional positional arg 1) — Absolute path to a lens context file
containing IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria.
If provided, read this file before beginning analysis to obtain structured context.
If omitted, discover context by exploring the CWD.
- experiment_plan_path (optional positional arg 2) — Absolute path to the full
experiment plan. If provided, read for complete experimental methodology and design.
If omitted, locate the experiment plan by exploring the CWD.
When to Use
- Planning a sequence of experiments
- Optimizing hyperparameter search
- Ablation study design
- User invokes
/autoskillit:exp-lens-iterative-learning or /autoskillit:make-experiment-diag iterative
Critical Constraints
NEVER:
- Modify any source code files
- Recommend one-factor-at-a-time exploration when interactions are plausible
- Create files outside
{{AUTOSKILLIT_TEMP}}/exp-lens-iterative-learning/
- Run subagents in the background (
run_in_background: true is prohibited)
ALWAYS:
Evaluate exploration efficiency against the key uncertainty being reduced
Identify high-value unexplored regions of the factor space
Assess whether the stopping rule is principled or arbitrary
Surface interaction structure that one-factor-at-a-time designs would miss
BEFORE creating any diagram, LOAD the /autoskillit:mermaid skill using the Skill tool - this is MANDATORY
If the Skill tool cannot be used (disable-model-invocation) or refuses this invocation, do NOT proceed with diagram creation. Abort this step and omit the diagram from output.
Write output to {{AUTOSKILLIT_TEMP}}/exp-lens-iterative-learning/exp_diag_iterative_learning_{YYYY-MM-DD_HHMMSS}.md
After writing the file, emit the structured output token as literal plain text with no
markdown formatting on the token name (the adjudicator performs a regex match):
diagram_path = /absolute/path/to/{{AUTOSKILLIT_TEMP}}/exp-lens-iterative-learning/exp_diag_iterative_learning_{...}.md
Analysis Workflow
Step 0: Parse optional arguments
If positional arg 1 (context_path) is provided and the file exists, read it to obtain
IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria. If positional
arg 2 (experiment_plan_path) is provided and exists, read the experiment plan for full
methodology. Use this structured context as the foundation for Steps 1-5; skip the CWD
exploration for these fields if the context file supplies them.
Step 1: Launch Parallel Exploration Subagents
Spawn Explore subagents to investigate:
Factor Space
- Find all factors being varied across experiments
- Look for: factor, parameter, variable, condition, treatment, level, dimension
Interaction Structure
- Find evidence of interaction effects between factors
- Look for: interaction, joint, combined, synergy, cross, factorial
Cost & Resource Model
- Find cost per experiment and total budget
- Look for: cost, budget, time, compute, trials, epochs, samples
Sequential Decision Logic
- Find how next experiments are chosen based on previous results
- Look for: adaptive, sequential, bayesian, acquisition, exploration, exploitation, bandit
Learning Objectives
- Find what uncertainty is being reduced by the experiment sequence
- Look for: objective, uncertainty, information, knowledge, goal, optimize
Step 2: Map the Design Space
Map factors × levels, explored regions, probed interactions, next high-value experiments. Assess efficiency vs. key uncertainty.
Step 3: CRITICAL — Analyze Learning Efficiency
Per factor/round: Information gain, Interaction risk, Cost-efficiency, Exploration-exploitation, Stopping rule
Distinguish: Full factorial / Fractional factorial / One-factor-at-a-time / Adaptive/Bayesian
Step 4: Create the Diagram
Direction: LR. Subgraphs: FACTOR SPACE, EXPLORATION STRATEGY, RESULTS SO FAR, NEXT EXPERIMENTS, STOPPING CRITERIA
Step 5: Write Output
Write the diagram to: {{AUTOSKILLIT_TEMP}}/exp-lens-iterative-learning/exp_diag_iterative_learning_{YYYY-MM-DD_HHMMSS}.md (relative to the current working directory)
Pre-Diagram Checklist
Before creating the diagram, verify:
Related Skills
/autoskillit:make-experiment-diag - Parent skill/autoskillit:mermaid - MUST BE LOADED before creating diagram/autoskillit:exp-lens-sensitivity-robustness/autoskillit:exp-lens-error-budget
1---2name: exp-lens-iterative-learning3description: Create Iterative Learning experimental design diagram showing factor space exploration, adaptive allocation, and next-experiment recommendations. Decision-Theoretic lens answering "How does this maximize learning per cost?"4---56# Iterative Learning Experimental Design Lens78**Philosophical Mode:** Decision-Theoretic9**Primary Question:** "How does this maximize learning per cost?"10**Focus:** Factor Selection, Interaction Probing, Adaptive Allocation, Stopping Rules, Next-Experiment Planning1112## Arguments1314`/autoskillit:exp-lens-iterative-learning [context_path] [experiment_plan_path]`1516- **context_path** (optional positional arg 1) — Absolute path to a lens context file17 containing IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria.18 If provided, read this file before beginning analysis to obtain structured context.19 If omitted, discover context by exploring the CWD.20- **experiment_plan_path** (optional positional arg 2) — Absolute path to the full21 experiment plan. If provided, read for complete experimental methodology and design.22 If omitted, locate the experiment plan by exploring the CWD.2324## When to Use2526- Planning a sequence of experiments27- Optimizing hyperparameter search28- Ablation study design29- User invokes `/autoskillit:exp-lens-iterative-learning` or `/autoskillit:make-experiment-diag iterative`3031## Critical Constraints3233**NEVER:**34- Modify any source code files35- Recommend one-factor-at-a-time exploration when interactions are plausible36- Create files outside `{{AUTOSKILLIT_TEMP}}/exp-lens-iterative-learning/`37- Run subagents in the background (`run_in_background: true` is prohibited)3839**ALWAYS:**40- Evaluate exploration efficiency against the key uncertainty being reduced41- Identify high-value unexplored regions of the factor space42- Assess whether the stopping rule is principled or arbitrary43- Surface interaction structure that one-factor-at-a-time designs would miss44- BEFORE creating any diagram, LOAD the `/autoskillit:mermaid` skill using the Skill tool - this is MANDATORY45- If the Skill tool cannot be used (disable-model-invocation) or refuses this invocation, do NOT proceed with diagram creation. Abort this step and omit the diagram from output.46- Write output to `{{AUTOSKILLIT_TEMP}}/exp-lens-iterative-learning/exp_diag_iterative_learning_{YYYY-MM-DD_HHMMSS}.md`47- After writing the file, emit the structured output token as **literal plain text** with no48 markdown formatting on the token name (the adjudicator performs a regex match):4950 ```51 diagram_path = /absolute/path/to/{{AUTOSKILLIT_TEMP}}/exp-lens-iterative-learning/exp_diag_iterative_learning_{...}.md52 ```5354---5556## Analysis Workflow5758### Step 0: Parse optional arguments5960If positional arg 1 (context_path) is provided and the file exists, read it to obtain61IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria. If positional62arg 2 (experiment_plan_path) is provided and exists, read the experiment plan for full63methodology. Use this structured context as the foundation for Steps 1-5; skip the CWD64exploration for these fields if the context file supplies them.6566### Step 1: Launch Parallel Exploration Subagents6768Spawn Explore subagents to investigate:6970**Factor Space**71- Find all factors being varied across experiments72- Look for: factor, parameter, variable, condition, treatment, level, dimension7374**Interaction Structure**75- Find evidence of interaction effects between factors76- Look for: interaction, joint, combined, synergy, cross, factorial7778**Cost & Resource Model**79- Find cost per experiment and total budget80- Look for: cost, budget, time, compute, trials, epochs, samples8182**Sequential Decision Logic**83- Find how next experiments are chosen based on previous results84- Look for: adaptive, sequential, bayesian, acquisition, exploration, exploitation, bandit8586**Learning Objectives**87- Find what uncertainty is being reduced by the experiment sequence88- Look for: objective, uncertainty, information, knowledge, goal, optimize8990### Step 2: Map the Design Space9192Map factors × levels, explored regions, probed interactions, next high-value experiments. Assess efficiency vs. key uncertainty.9394### Step 3: CRITICAL — Analyze Learning Efficiency9596Per factor/round: Information gain, Interaction risk, Cost-efficiency, Exploration-exploitation, Stopping rule9798Distinguish: Full factorial / Fractional factorial / One-factor-at-a-time / Adaptive/Bayesian99100### Step 4: Create the Diagram101102**Direction:** LR. Subgraphs: FACTOR SPACE, EXPLORATION STRATEGY, RESULTS SO FAR, NEXT EXPERIMENTS, STOPPING CRITERIA103104### Step 5: Write Output105106Write the diagram to: `{{AUTOSKILLIT_TEMP}}/exp-lens-iterative-learning/exp_diag_iterative_learning_{YYYY-MM-DD_HHMMSS}.md` (relative to the current working directory)107108---109110## Pre-Diagram Checklist111112Before creating the diagram, verify:113114- [ ] LOADED `/autoskillit:mermaid` skill using the Skill tool115- [ ] Using ONLY classDef styles from the mermaid skill (no invented colors)116- [ ] Diagram will include a color legend table117118---119120## Related Skills121122- `/autoskillit:make-experiment-diag` - Parent skill123- `/autoskillit:mermaid` - MUST BE LOADED before creating diagram124- `/autoskillit:exp-lens-sensitivity-robustness`125- `/autoskillit:exp-lens-error-budget`
TalonT-Org