Iterative Learning Experimental Design Lens
Philosophical Mode: Decision-Theoretic
Primary Question: "How does this maximize learning per cost?"
Focus: Factor Selection, Interaction Probing, Adaptive Allocation, Stopping Rules, Next-Experiment Planning
Arguments
/autoskillit:exp-lens-iterative-learning [context_path] [experiment_plan_path]
- context_path (optional positional arg 1) — Absolute path to a lens context file
containing IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria.
If provided, read this file before beginning analysis to obtain structured context.
If omitted, discover context by exploring the CWD.
- experiment_plan_path (optional positional arg 2) — Absolute path to the full
experiment plan. If provided, read for complete experimental methodology and design.
If omitted, locate the experiment plan by exploring the CWD.
When to Use
- Planning a sequence of experiments
- Optimizing hyperparameter search
- Ablation study design
- User invokes
/autoskillit:exp-lens-iterative-learning or /autoskillit:make-experiment-diag iterative
Critical Constraints
NEVER:
- Modify any source code files
- Recommend one-factor-at-a-time exploration when interactions are plausible
- Create files outside
{{AUTOSKILLIT_TEMP}}/exp-lens-iterative-learning/
- Run subagents in the background (
run_in_background: true is prohibited)
ALWAYS:
Evaluate exploration efficiency against the key uncertainty being reduced
Identify high-value unexplored regions of the factor space
Assess whether the stopping rule is principled or arbitrary
Surface interaction structure that one-factor-at-a-time designs would miss
BEFORE creating any diagram, LOAD the /autoskillit:mermaid skill using the Skill tool - this is MANDATORY
If the Skill tool cannot be used (disable-model-invocation) or refuses this invocation, do NOT proceed with diagram creation. Abort this step and omit the diagram from output.
Write output to {{AUTOSKILLIT_TEMP}}/exp-lens-iterative-learning/exp_diag_iterative_learning_{YYYY-MM-DD_HHMMSS}.md
After writing the file, emit the structured output token as literal plain text with no
markdown formatting on the token name (the adjudicator performs a regex match):
diagram_path = /absolute/path/to/{{AUTOSKILLIT_TEMP}}/exp-lens-iterative-learning/exp_diag_iterative_learning_{...}.md
Analysis Workflow
Step 0: Parse optional arguments
If positional arg 1 (context_path) is provided and the file exists, read it to obtain
IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria. If positional
arg 2 (experiment_plan_path) is provided and exists, read the experiment plan for full
methodology. Use this structured context as the foundation for Steps 1-5; skip the CWD
exploration for these fields if the context file supplies them.
Step 1: Launch Parallel Exploration Subagents
Spawn Explore subagents to investigate:
Factor Space
- Find all factors being varied across experiments
- Look for: factor, parameter, variable, condition, treatment, level, dimension
Interaction Structure
- Find evidence of interaction effects between factors
- Look for: interaction, joint, combined, synergy, cross, factorial
Cost & Resource Model
- Find cost per experiment and total budget
- Look for: cost, budget, time, compute, trials, epochs, samples
Sequential Decision Logic
- Find how next experiments are chosen based on previous results
- Look for: adaptive, sequential, bayesian, acquisition, exploration, exploitation, bandit
Learning Objectives
- Find what uncertainty is being reduced by the experiment sequence
- Look for: objective, uncertainty, information, knowledge, goal, optimize
Step 2: Map the Design Space
Map factors × levels, explored regions, probed interactions, next high-value experiments. Assess efficiency vs. key uncertainty.
Step 3: CRITICAL — Analyze Learning Efficiency
Per factor/round: Information gain, Interaction risk, Cost-efficiency, Exploration-exploitation, Stopping rule
Distinguish: Full factorial / Fractional factorial / One-factor-at-a-time / Adaptive/Bayesian
Step 4: Create the Diagram
Direction: LR. Subgraphs: FACTOR SPACE, EXPLORATION STRATEGY, RESULTS SO FAR, NEXT EXPERIMENTS, STOPPING CRITERIA
Step 5: Write Output
Write the diagram to: {{AUTOSKILLIT_TEMP}}/exp-lens-iterative-learning/exp_diag_iterative_learning_{YYYY-MM-DD_HHMMSS}.md (relative to the current working directory)
Pre-Diagram Checklist
Before creating the diagram, verify:
Related Skills
/autoskillit:make-experiment-diag - Parent skill
/autoskillit:mermaid - MUST BE LOADED before creating diagram
/autoskillit:exp-lens-sensitivity-robustness
/autoskillit:exp-lens-error-budget
1---2name: exp-lens-iterative-learning3categories: [exp-lens]4activate_deps: [mermaid]5description: Create Iterative Learning experimental design diagram showing factor space exploration, adaptive allocation, and next-experiment recommendations. Decision-Theoretic lens answering "How does this maximize learning per cost?"6hooks:7 PreToolUse:8 - matcher: "*"9 hooks:10 - type: command11 command: "echo 'Iterative Learning Lens - Analyzing learning efficiency and design space...'"12 once: true13metadata:14 source: https://skillmd.com/skills/talont-org/exp-lens-iterative-learning15---1617# Iterative Learning Experimental Design Lens1819**Philosophical Mode:** Decision-Theoretic20**Primary Question:** "How does this maximize learning per cost?"21**Focus:** Factor Selection, Interaction Probing, Adaptive Allocation, Stopping Rules, Next-Experiment Planning2223## Arguments2425`/autoskillit:exp-lens-iterative-learning [context_path] [experiment_plan_path]`2627- **context_path** (optional positional arg 1) — Absolute path to a lens context file28 containing IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria.29 If provided, read this file before beginning analysis to obtain structured context.30 If omitted, discover context by exploring the CWD.31- **experiment_plan_path** (optional positional arg 2) — Absolute path to the full32 experiment plan. If provided, read for complete experimental methodology and design.33 If omitted, locate the experiment plan by exploring the CWD.3435## When to Use3637- Planning a sequence of experiments38- Optimizing hyperparameter search39- Ablation study design40- User invokes `/autoskillit:exp-lens-iterative-learning` or `/autoskillit:make-experiment-diag iterative`4142## Critical Constraints4344**NEVER:**45- Modify any source code files46- Recommend one-factor-at-a-time exploration when interactions are plausible47- Create files outside `{{AUTOSKILLIT_TEMP}}/exp-lens-iterative-learning/`48- Run subagents in the background (`run_in_background: true` is prohibited)4950**ALWAYS:**51- Evaluate exploration efficiency against the key uncertainty being reduced52- Identify high-value unexplored regions of the factor space53- Assess whether the stopping rule is principled or arbitrary54- Surface interaction structure that one-factor-at-a-time designs would miss55- BEFORE creating any diagram, LOAD the `/autoskillit:mermaid` skill using the Skill tool - this is MANDATORY56- If the Skill tool cannot be used (disable-model-invocation) or refuses this invocation, do NOT proceed with diagram creation. Abort this step and omit the diagram from output.57- Write output to `{{AUTOSKILLIT_TEMP}}/exp-lens-iterative-learning/exp_diag_iterative_learning_{YYYY-MM-DD_HHMMSS}.md`58- After writing the file, emit the structured output token as **literal plain text** with no59 markdown formatting on the token name (the adjudicator performs a regex match):6061 ```62 diagram_path = /absolute/path/to/{{AUTOSKILLIT_TEMP}}/exp-lens-iterative-learning/exp_diag_iterative_learning_{...}.md63 ```6465---6667## Analysis Workflow6869### Step 0: Parse optional arguments7071If positional arg 1 (context_path) is provided and the file exists, read it to obtain72IV/DV tables, H0/H1 hypotheses, controlled variables, and success criteria. If positional73arg 2 (experiment_plan_path) is provided and exists, read the experiment plan for full74methodology. Use this structured context as the foundation for Steps 1-5; skip the CWD75exploration for these fields if the context file supplies them.7677### Step 1: Launch Parallel Exploration Subagents7879Spawn Explore subagents to investigate:8081**Factor Space**82- Find all factors being varied across experiments83- Look for: factor, parameter, variable, condition, treatment, level, dimension8485**Interaction Structure**86- Find evidence of interaction effects between factors87- Look for: interaction, joint, combined, synergy, cross, factorial8889**Cost & Resource Model**90- Find cost per experiment and total budget91- Look for: cost, budget, time, compute, trials, epochs, samples9293**Sequential Decision Logic**94- Find how next experiments are chosen based on previous results95- Look for: adaptive, sequential, bayesian, acquisition, exploration, exploitation, bandit9697**Learning Objectives**98- Find what uncertainty is being reduced by the experiment sequence99- Look for: objective, uncertainty, information, knowledge, goal, optimize100101### Step 2: Map the Design Space102103Map factors × levels, explored regions, probed interactions, next high-value experiments. Assess efficiency vs. key uncertainty.104105### Step 3: CRITICAL — Analyze Learning Efficiency106107Per factor/round: Information gain, Interaction risk, Cost-efficiency, Exploration-exploitation, Stopping rule108109Distinguish: Full factorial / Fractional factorial / One-factor-at-a-time / Adaptive/Bayesian110111### Step 4: Create the Diagram112113**Direction:** LR. Subgraphs: FACTOR SPACE, EXPLORATION STRATEGY, RESULTS SO FAR, NEXT EXPERIMENTS, STOPPING CRITERIA114115### Step 5: Write Output116117Write the diagram to: `{{AUTOSKILLIT_TEMP}}/exp-lens-iterative-learning/exp_diag_iterative_learning_{YYYY-MM-DD_HHMMSS}.md` (relative to the current working directory)118119---120121## Pre-Diagram Checklist122123Before creating the diagram, verify:124125- [ ] LOADED `/autoskillit:mermaid` skill using the Skill tool126- [ ] Using ONLY classDef styles from the mermaid skill (no invented colors)127- [ ] Diagram will include a color legend table128129---130131## Related Skills132133- `/autoskillit:make-experiment-diag` - Parent skill134- `/autoskillit:mermaid` - MUST BE LOADED before creating diagram135- `/autoskillit:exp-lens-sensitivity-robustness`136- `/autoskillit:exp-lens-error-budget`