# Exp Lens Iterative Learning

> Create Iterative Learning experimental design diagram showing factor space exploration, adaptive allocation, and next-experiment recommendations. Decision-Theoretic lens answering "How does this maximize learning per cost?"

- Skill: `trecek/exp-lens-iterative-learning` (Agent Skill)
- Install (CLI): `npx skillmds add trecek/exp-lens-iterative-learning`
- Raw SKILL.md: https://api.skillmd.com/api/skills/trecek/exp-lens-iterative-learning/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: Trecek (https://skillmd.com/u/trecek)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/trecek/exp-lens-iterative-learning

---


# Iterative Learning Experimental Design Lens

**Philosophical Mode:** Decision-Theoretic
**Primary Question:** "How does this maximize learning per cost?"
**Focus:** Factor Selection, Interaction Probing, Adaptive Allocation, Stopping Rules, Next-Experiment Planning

## When to Use

- Planning a sequence of experiments
- Optimizing hyperparameter search
- Ablation study design
- User invokes `/exp-lens-iterative-learning` or `/make-experiment-diag iterative`

## Critical Constraints

**NEVER:**
- Modify any source code files
- Do not litter the codebase with useless comments, TODO markers, or explanatory annotations — the skill output and diagram speak for themselves
- Recommend one-factor-at-a-time exploration when interactions are plausible

**ALWAYS:**
- Evaluate exploration efficiency against the key uncertainty being reduced
- Identify high-value unexplored regions of the factor space
- Assess whether the stopping rule is principled or arbitrary
- Surface interaction structure that one-factor-at-a-time designs would miss
- BEFORE creating any diagram, LOAD the `/mermaid` skill using the Skill tool - this is MANDATORY

---

## Analysis Workflow

### Step 1: Launch Parallel Exploration Subagents

Spawn Explore subagents to investigate:

**Factor Space**
- Find all experimental factors and their ranges
- Look for: `hyperparameter`, `config`, `param`, `factor`, `level`, `range`, `grid`, `sweep`

**Interaction Structure**
- Find evidence of factor interactions
- Look for: `interaction`, `ablation`, `factorial`, `combination`, `cross`, `joint`

**Cost & Resource Model**
- Find compute/time/cost constraints
- Look for: `budget`, `cost`, `time`, `gpu`, `compute`, `resource`, `quota`, `limit`

**Sequential Decision Logic**
- Find adaptive or sequential experiment logic
- Look for: `bayesian_opt`, `optuna`, `hyperband`, `successive`, `early_stop`, `adaptive`, `bandit`

**Learning Objectives**
- Find what uncertainty the experiments aim to reduce
- Look for: `objective`, `goal`, `hypothesis`, `question`, `target`, `uncertainty`, `explore`

### Step 2: Map the Design Space

Map the design space (factors x levels), identify which regions have been explored, which interactions are probed, and what the next high-value experiment would be. Assess whether the experimental sequence is efficient (reducing key uncertainty) or wasteful (one-factor-at-a-time in interacting systems).

### Step 3: CRITICAL — Analyze Learning Efficiency

For every factor and experiment round:
- **Information gain**: What uncertainty does this experiment reduce? What does it leave unresolved?
- **Interaction risk**: Are there factors that likely interact? Does the design probe those interactions?
- **Cost-efficiency**: What is the cost per unit of learning? Is there a cheaper design with equivalent learning value?
- **Exploration-exploitation**: Is the strategy exploring new regions or exploiting known good regions prematurely?
- **Stopping rule**: Is the stopping criterion based on a principled uncertainty threshold, or is it arbitrary (fixed budget, time limit)?

Distinguish clearly:
- **Full factorial**: Explores all combinations — exhaustive but expensive, catches all interactions
- **Fractional factorial**: Efficient but aliases some interaction effects
- **One-factor-at-a-time**: Simple but misses interactions — inefficient in correlated spaces
- **Adaptive/Bayesian**: High efficiency when the response surface is smooth — risks local optima

### Step 4: Create the Diagram

Use flowchart with:

**Direction:** `LR` (learning flows from exploration to refinement)

**Subgraphs:**
- `FACTOR SPACE`
- `EXPLORATION STRATEGY`
- `RESULTS SO FAR`
- `NEXT EXPERIMENTS`
- `STOPPING CRITERIA`

**Node Styling:**
- `stateNode` class: factors/parameters
- `handler` class: exploration strategies
- `phase` class: experiment rounds
- `output` class: results/learnings
- `cli` class: objectives
- `gap` class: unexplored high-value regions
- `detector` class: stopping criteria

### Step 5: Write Output

Write the diagram to: `temp/exp-lens-iterative-learning/exp_diag_iterative_learning_{YYYY-MM-DD_HHMMSS}.md`

---

## Output Template

```markdown
# Iterative Learning Diagram: {System Name}

**Lens:** Iterative Learning (Decision-Theoretic)
**Question:** How does this maximize learning per cost?
**Date:** {YYYY-MM-DD}
**Scope:** {What was analyzed}

## Factor Space

| Factor | Range | Levels | Explored? | Interaction Risk |
|--------|-------|--------|-----------|-----------------|
| {Factor 1} | {min}–{max} | {n} | ✓ Full | Low |
| {Factor 2} | {min}–{max} | {n} | ✓ Partial | High |
| {Factor 3} | {min}–{max} | {n} | ✗ None | Unknown |

## Iterative Learning Design-Space Diagram

```mermaid
%%{init: {'flowchart': {'nodeSpacing': 50, 'rankSpacing': 60, 'curve': 'basis'}}}%%
flowchart LR
    %% CLASS DEFINITIONS %%
    classDef cli fill:#1a237e,stroke:#7986cb,stroke-width:2px,color:#fff;
    classDef stateNode fill:#004d40,stroke:#4db6ac,stroke-width:2px,color:#fff;
    classDef handler fill:#e65100,stroke:#ffb74d,stroke-width:2px,color:#fff;
    classDef phase fill:#6a1b9a,stroke:#ba68c8,stroke-width:2px,color:#fff;
    classDef newComponent fill:#2e7d32,stroke:#81c784,stroke-width:2px,color:#fff;
    classDef output fill:#00695c,stroke:#4db6ac,stroke-width:2px,color:#fff;
    classDef detector fill:#b71c1c,stroke:#ef5350,stroke-width:2px,color:#fff;
    classDef gap fill:#ff6f00,stroke:#ffa726,stroke-width:2px,color:#000;
    classDef integration fill:#c62828,stroke:#ef9a9a,stroke-width:2px,color:#fff;

    subgraph Factors ["FACTOR SPACE"]
        direction TB
        F1["{Factor 1}<br/>━━━━━━━━━━<br/>{range}, {levels} levels"]
        F2["{Factor 2}<br/>━━━━━━━━━━<br/>{range}, {levels} levels"]
        F3["{Factor 3}<br/>━━━━━━━━━━<br/>{range}, {levels} levels"]
    end

    subgraph Strategy ["EXPLORATION STRATEGY"]
        direction TB
        STRAT["{Strategy Type}<br/>━━━━━━━━━━<br/>{e.g., Bayesian/Grid/OFAT}"]
    end

    subgraph Rounds ["RESULTS SO FAR"]
        direction TB
        R1["{Round 1}<br/>━━━━━━━━━━<br/>{n} runs, {finding}"]
        R2["{Round 2}<br/>━━━━━━━━━━<br/>{n} runs, {finding}"]
    end

    subgraph Next ["NEXT EXPERIMENTS"]
        direction TB
        NX1["{Next Experiment 1}<br/>━━━━━━━━━━<br/>{rationale}"]
        NX2["{Next Experiment 2}<br/>━━━━━━━━━━<br/>{rationale}"]
    end

    subgraph Stopping ["STOPPING CRITERIA"]
        direction TB
        STOP["{Stopping Rule}<br/>━━━━━━━━━━<br/>{threshold or condition}"]
    end

    OBJ["{Learning Objective}<br/>━━━━━━━━━━<br/>{key uncertainty to reduce}"]
    UNEXPLORED["{High-Value Unexplored Region}<br/>━━━━━━━━━━<br/>{what is missing and why it matters}"]

    OBJ --> F1
    OBJ --> F2
    OBJ --> F3
    F1 --> STRAT
    F2 --> STRAT
    F3 --> STRAT
    STRAT --> R1
    R1 --> R2
    R2 --> NX1
    R2 --> NX2
    NX1 --> STOP
    NX2 --> STOP
    UNEXPLORED -.->|missed| STRAT

    %% CLASS ASSIGNMENTS %%
    class F1,F2,F3 stateNode;
    class STRAT handler;
    class R1,R2 phase;
    class NX1,NX2 output;
    class OBJ cli;
    class UNEXPLORED gap;
    class STOP detector;
```

**Color Legend:**
| Color | Category | Description |
|-------|----------|-------------|
| Dark Teal | Factors | Experimental factors and parameters |
| Orange | Strategy | Exploration strategy used |
| Purple | Rounds | Completed experiment rounds |
| Teal | Next | Recommended next experiments |
| Dark Blue | Objective | Learning objective |
| Yellow/Amber | Gaps | Unexplored high-value regions |
| Red | Stopping | Stopping criteria |

## Efficiency Assessment

| Round | Cost | Key Learning | Uncertainty Reduced | Efficiency Rating |
|-------|------|-------------|--------------------|--------------------|
| {Round 1} | {cost} | {finding} | {what resolved} | High/Med/Low |
| {Round 2} | {cost} | {finding} | {what resolved} | High/Med/Low |

## Recommended Next Steps

| Priority | Experiment | Factors | Expected Learning | Cost Estimate |
|----------|-----------|---------|------------------|---------------|
| 1 | {description} | {factors to vary} | {uncertainty reduced} | {cost} |
| 2 | {description} | {factors to vary} | {uncertainty reduced} | {cost} |
```

---

## Pre-Diagram Checklist

Before creating the diagram, verify:

- [ ] LOADED `/mermaid` skill using the Skill tool
- [ ] Using ONLY classDef styles from the mermaid skill (no invented colors)
- [ ] Diagram will include a color legend table

---

## Related Skills

- `/make-experiment-diag` - Parent skill for lens selection
- `/mermaid` - MUST BE LOADED before creating diagram
- `/exp-lens-sensitivity-robustness` - For factor sensitivity analysis
- `/exp-lens-error-budget` - For cost and resource constraint analysis