# Loop Driven Engineering

> Use at the start of any non-trivial engineering task (feature, bugfix touching more than one file, refactor with observable behavior change, incident response). Orchestrates LDD's four loops — inner (code), refinement (deliverable), outer (method), CoT (reasoning chain, v0.8.0) — with hard iteration budgets, dispatches the specialist skills at the right moments, and forbids declaring "done" without a synced doc-level mental model.

- Skill: `veegee82/loop-driven-engineering` (Agent Skill)
- Install (CLI): `npx skillmds@latest add veegee82/loop-driven-engineering`
- Raw SKILL.md: https://api.skillmd.com/api/skills/veegee82/loop-driven-engineering/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: veegee82 (https://skillmd.com/u/veegee82)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/veegee82/loop-driven-engineering

---


# Loop-Driven-Engineering

## The Metaphor

**The flight controller guiding four nested loops.** LDD is [Gradient Descent for Agents](../../docs/theory.md) and this skill is the coordinator. The *inner loop* is the aircraft correcting pitch per second — direct edits to the code (`∂L/∂code`). The *refinement loop* is the approach pattern — polishing the landing without changing the aircraft (`∂L/∂output`). The *outer loop* is the training program that adjusts how pilots are taught — changing the skill itself (`∂L/∂method`). The *CoT loop* is the pilot's own step-by-step reasoning during the descent — each thought gated before it commits (`∂L/∂thought`). Each loop has its own budget, its own instruments, its own failure mode. The controller's first duty: *name which loop is active.* Confusion about which loop you're in is the canonical engineering error.

## Overview

Engineering with an AI agent is **gradient descent across four parameter spaces**, not one. You plan, you try, you measure, you diagnose, you try again — on the **code axis** (inner loop, `θ`), the **deliverable axis** (refinement, `y`), the **method axis** (outer loop, `m`), and the **reasoning-chain axis** (CoT loop, `t`). This skill is the entry-point and coordinator: it picks which loop is active for the current task, dispatches the specialists, enforces the budget, and closes the loop only when regularizers hold.

The [thinking-levels auto-dispatch](../../docs/ldd/thinking-levels.md) runs *before* this skill and picks the rigor level (L0…L4) — that is the step-size scheduler for the whole optimizer. This skill runs *inside* the chosen level and orchestrates the actual descent.

**Core principle:** the loop closes when applicable gates are green **and** the doc-level mental model is current. Not when one test passes. Not when "it looks right." Not when the LLM sounds confident.

This skill is a **flexible pattern**, not a rigid procedure. Use judgment; adapt the structure to the task. But the budget, the escalation rule, and the loop-separation rule are hard.

See [`../../docs/ldd/convergence.md`](../../docs/ldd/convergence.md) for the four-loop model in full, [`../../diagrams/four-axes-gradient-descent.svg`](../../diagrams/four-axes-gradient-descent.svg) for the top-level picture, and [`../../diagrams/four-loops.svg`](../../diagrams/four-loops.svg) for the loop-nesting view across all four axes.

## When to Use

Invoke at the **start** of any non-trivial engineering task:

- Feature touching ≥2 files or ≥1 public surface
- Bugfix where the cause is not yet identified
- Refactor with observable behavior change (even if spec-preserving)
- Incident response
- Plan-driven work (a multi-step design under time pressure)
- Any work where you are tempted to "just start coding"

Do **not** use for:
- Trivial edits (rename a variable, fix a typo, delete dead code)
- Pure lookups or explanations
- Tasks already framed by a written plan you're executing verbatim

## The Four Loops

LDD has four loops, each on a different parameter axis. Pick **one** at the start; running multiple simultaneously produces unstable gradients. See [`../dialectical-cot/SKILL.md`](../dialectical-cot/SKILL.md) for the CoT loop (reasoning-chain axis, v0.8.0), which fires on verifiable multi-step reasoning tasks and is orthogonal to the three code-side loops below.

### Inner Loop (θ = code, the default)

```
read context (CLAUDE.md / README / relevant docs — only what the task needs)
  ↓
plan (dialectical-reasoning → concrete next step)
  ↓
loop(k ≤ K_MAX = 5):
    observe failing signal
    ↓
    reproducibility-first (is this real signal or noise?)
    ↓
    e2e-driven-iteration: run E2E → measure loss_k → diagnose → fix → rerun
    ↓
    if green AND task-complete: break
    if failed: invoke root-cause-by-layer + loss-backprop-lens, next iteration
    ↓
    if k == K_MAX: STOP — escalate with the required shape, do not grind on
  ↓
close: docs-as-definition-of-done → commit
```

### Refinement Loop (θ = deliverable)

Use when a deliverable exists, is "good enough but not great," and re-running from scratch would waste what worked. Dispatches `iterative-refinement`. Budget halves per iteration; stops on regression, plateau, or wall-time.

### Outer Loop (θ = skills / rubrics / prompts)

Use when the same rubric violation recurs across 3+ distinct tasks. Dispatches `method-evolution`. Requires a task suite and `Δloss_method` measurement; rollback on regression.

**Hard rule:** if you can't say which loop you're in, stop. The wrong loop doesn't just waste budget — it can regress the artifact it was meant to improve.

## The Budget: K_MAX = 5

**Five iterations per task, default.** If the loop has not closed after five, stop, write what failed and why, and escalate (to the user, to a colleague, to a different skill).

Why five: below that, you haven't explored; above it, you're flailing. Each extra iteration beyond five is a ~linear increase in sunk-cost bias and a ~geometric decrease in probability the next iteration will converge without a structural rethink.

**Hard rule:** when k hits K_MAX, produce the **escalation deliverable** below. It is a structured artifact, not a free-form apology. A 6th iteration without producing it is a process failure and is itself an outer-loop `method-evolution` trigger.

**Escalation deliverable (v0.13.1 — concrete template):**

1. **What was tried, per iteration.** One bullet per iteration, naming the skill that fired and the concrete change made (one line each). Example:
   - `i1: reproducibility-first → confirmed fail is real (3/3 reruns)`
   - `i2: root-cause-by-layer → layer 3, contract violation on UserPayload.status (nullable vs. required)`
   - `i3: added defensive isinstance check — passed locally, sibling test regressed`
   - `i4: reverted; tightened schema instead — same regression`
   - `i5: swapped to discriminated union — 2 more siblings now fail`

2. **What kept failing — verbatim.** Paste the exact error message / test name / gate output. No paraphrase. The reviewer needs to grep the artifact for this string.

3. **Layer-4/5 diagnosis** via `root-cause-by-layer`. Both layers must be named. Not "it's a design issue" — specify: "layer 4 (structural): `UserValidator` does two orthogonal jobs (schema check + behavior check) through one return type. layer 5 (conceptual): the domain has two invariants (shape, temporal order) but the code treats them as one."

4. **One-sentence concrete architectural proposal.** The sentence must name specific identifiers — file paths, class names, function names, data structures. Not "redesign module X", not "split the validator." Write the actual one-line design: `"Split UserValidator into UserSchemaValidator (enforces R-rules 1–7) and UserBehaviorValidator (enforces R-rules 8–12); wire UserBehaviorValidator after UserSchemaValidator in the checkout pipeline; remove the dual-return-type entirely."` If the sentence cannot be made concrete, the task is not ready for human escalation — it is ready for an `architect-mode` invocation first (the level scorer should have picked L3/L4 in that case).

5. **Explicit ask.** State what input you need, in one sentence: "Should I implement proposal (4) now, or do you want to see alternative architectures first via a `/ldd-architect` pass?" — not "let me know what you think."

**Loss-backprop-lens is diagnostic, not designer.** It tells you *the step size was wrong* (you were in a local minimum). It does not tell you *what the bigger step is*. That is the agent's job, in bullet (4). Escalations without a concrete bullet (4) get returned and the agent must re-do the K_MAX close before further work.

## The Test Pyramid (cheap → expensive)

Run gates in order of cost. Only escalate when the current tier is green.

| Tier | Examples | When to run |
|---|---|---|
| 0. **Local thought** | Read the code you're changing; grep for callers; skim docs | Always, before any edit |
| 1. **Schema / lint / type** | `ruff`, `tsc --noEmit`, JSON / YAML schema validation | After every code edit |
| 2. **Unit tests** | `pytest tests/unit`, `npm run test:unit` | After tier-1 green |
| 3. **Integration** | Hits a DB / filesystem / fake HTTP server | After tier-2 green, for changes crossing module boundaries |
| 4. **E2E** | Real external services, full-stack, long runtime | Only when tier-3 green AND the change warrants it (behavior change, pre-release, public-surface delta) |
| 5. **Live / production-like** | Canary, shadow traffic, staged rollout | Only after tier-4, only for high-blast-radius changes |

**Anti-pattern:** running E2E to catch a typo. It would have been caught by tier 1 in 2 seconds. Know your pyramid.

**Anti-pattern:** skipping tiers 1–3 because "E2E will catch everything." It won't, and when it fails you will have no idea at which layer it broke.

## Sub-Skill Dispatch

This is a composition skill. Dispatch the right sub-skill at the right moment. Entries from **this plugin** are always available. Entries from other plugins (shown as *external*) are optional — if not installed, apply the principle inline.

| Moment in the loop | Sub-skill | Source | Why |
|---|---|---|---|
| Task start, if `.ldd/trace.log` exists | `ldd_trace status --project .` | this plugin (v0.5.1 tool) | Recover prior iteration state before starting new iterations |
| Observing a failing signal, before any edit | `reproducibility-first` | this plugin | One sample is noise, not gradient |
| Inside a fix-loop, every iteration | `e2e-driven-iteration` | this plugin | Measure loss each iteration, don't guess |
| Debugging a failing gate / test / run | `root-cause-by-layer` | this plugin | Find the structural origin, not the surface symptom |
| Deciding edit size / one-off vs recurring | `loss-backprop-lens` | this plugin | Step size must match the loss pattern |
| Any non-trivial recommendation, plan, trade-off | `dialectical-reasoning` | this plugin | Force the counter-case before shipping an opinion |
| Deliverable is good-enough but not great | `iterative-refinement` | this plugin | Polish with a real gradient (y-axis) |
| Same rubric violation in 3+ tasks | `method-evolution` | this plugin | Evolve the skill itself (θ-axis) |
| **At iteration close, every iteration** | `ldd_trace append` | this plugin (v0.5.1 tool) | Per-iteration trace emission is mandatory (see using-ldd) — the tool is the cheap path |
| Release-candidate / weekly / before version bump | `drift-detection` | this plugin | Find cumulative drift that per-commit gates missed |
| Before committing / pushing / declaring done | `docs-as-definition-of-done` | this plugin | Sync docs to current behavior |
| At loop close, before handoff | `ldd_trace close` | this plugin (v0.5.1 tool) | Record terminal status + layer fix in `.ldd/trace.log` |
| Open-ended problem framing | `brainstorming` | external (superpowers) | Explore before committing |
| Writing a multi-step plan | `writing-plans` | external (superpowers) | Externalize the plan |
| Red-green discipline while coding | `test-driven-development` | external (superpowers) | Test before implementation |
| Before declaring "success" / "it's fixed" | `verification-before-completion` | external (superpowers) | Evidence before assertion |
| Before merging | `requesting-code-review` | external (superpowers) | Second set of eyes |

You are **not** required to invoke all of them on every task. Pick what the loop actually needs at this step. But you **are** required to invoke the right one at the right moment — skipping `reproducibility-first` on a flake, or skipping `docs-as-definition-of-done` on a behavior change, is a process failure. When an external sub-skill is not available, apply its principle inline (draft the plan in the conversation rather than loading a plan-writing skill).

## Red Flags — STOP, the loop is broken

- "I've tried this same fix 3 times, let me try a 4th variant" → local-minimum trap; invoke `loss-backprop-lens`, consider architectural step
- "The same pattern keeps happening across different tasks" → outer-loop signal; invoke `method-evolution`
- "I'll run E2E to see what breaks" (skipping tiers 1–3) → pyramid violation
- "I'll just commit and iterate in follow-ups" → budget discipline failure; write a plan
- "It failed / passed once, retry" → invoke `reproducibility-first`; one sample is not a gradient
- "This is iteration 6, one more should do it" → K_MAX violation; stop and escalate
- "Docs are fine for now, behavior change is subtle" → invoke `docs-as-definition-of-done`
- "I'll regenerate the doc from scratch instead of refining" → you conflated refinement with re-planning; invoke `iterative-refinement`
- "I'll edit the skill to match what just happened" → potential moving-target loss; invoke `method-evolution` with proper measurement
- "I don't need a plan, it's a small change" → "small" is the canonical rationalization for no-plan
- Same gate rejecting with the same reason ≥ 2 iterations → stagnation; abort, diagnose, restart with a fix at the structural origin
- No full drift scan in months on an active project → silent accumulation; invoke `drift-detection`

## Escalation on Non-Convergence

When you can't close the loop, the productive response is **not** to try harder — it's to communicate. The right escalation has:

- **What was tried** — concrete, per iteration.
- **What kept failing** — the gate, test, or symptom, verbatim.
- **The layer-4/5 diagnosis** (via root-cause-by-layer).
- **The step-size recommendation** (via loss-backprop-lens): is this a local-fix problem that needs fresh eyes, or a structural problem that needs a different design?
- **The explicit ask** — what input do you need to proceed?

A well-escalated non-convergence beats a silently-limping sixth iteration every time.

## How to Apply — checklist

1. **Context read** — only what the task needs. If the project has a `CLAUDE.md` / `AGENTS.md`, load it. Load the relevant docs. Do **not** read the whole repo.
2. **Plan** — one paragraph, run it through `dialectical-reasoning` if non-trivial. For multi-step work, write it out (use a plan-writing skill if available, otherwise draft the plan in the conversation).
3. **Enter loop.** k = 0, K_MAX = 5.
4. **For each iteration:** smallest coherent code change → fast gates → diagnose failures via sub-skills → decide step size.
5. **Close** — run docs-as-definition-of-done, verify completion with a verification skill if available, commit as one logical unit.
6. **If K_MAX hit** — escalate with the checklist above.

## Counter-Examples — when this skill is overkill

- You're renaming a local variable. Just rename it.
- You're fixing a typo in a README. Fix it, commit.
- You're answering a factual lookup question. Read and answer.

The skill is for **multi-step engineering under uncertainty**, not for every interaction.

## Real-World Shape

Good loop (feature: add `--dry-run` to `myapp deploy`):
1. Read `cli.py`, `deploy.py`, relevant README sections. (1 min)
2. Plan: dialectical pass — "thesis: add a flag, gate execution on it; antithesis: dry-run could leak partial state if deploy has side effects in plan-phase; synthesis: separate `plan()` from `execute()`, flag only skips `execute()`." (3 min)
3. k=1: refactor `plan()`/`execute()` split, unit tests for both. Tier-1+2 green. (15 min)
4. k=2: add flag, test dry-run path. Tier-1+2+3 green. (10 min)
5. Close: `docs-as-definition-of-done` — README L82 safety note, cli-reference table, user-guide preview section. (5 min)
6. Commit as one logical unit. Done in 2 iterations, budget to spare.

Bad loop (same task, no discipline):
1. Start coding the flag. k=1: works, but broke side-effect ordering. k=2: patch the ordering, broke a different test. k=3: patch that, broke idempotency. k=4: flaky. k=5: ship with retry. k=6: "one more fix." k=7: push through freeze window with TODO on README. Two weeks later, incident.

## Related / Composed

**In this plugin (always available):**
- **`reproducibility-first`** — gate before any gradient use
- **`root-cause-by-layer`** — the 5-layer debugging ladder
- **`loss-backprop-lens`** — edit-size calibration and signal-vs-noise
- **`e2e-driven-iteration`** — measure-every-iteration inner-loop rhythm
- **`dialectical-reasoning`** — planning and recommendations
- **`iterative-refinement`** — y-axis (deliverable) optimization
- **`method-evolution`** — θ-axis (skill / rubric) outer-loop optimization
- **`drift-detection`** — periodic full-repo scan for cumulative drift
- **`docs-as-definition-of-done`** — the close of the loop

**Optional companions** (Claude Code `superpowers` plugin, Codex built-ins, or equivalent):
- **`brainstorming`** — open-ended problem framing
- **`writing-plans`** — externalized multi-step plans
- **`test-driven-development`** — red-green discipline when writing code
- **`verification-before-completion`** — evidence before success claims
- **`systematic-debugging`** — overlaps with `root-cause-by-layer`; prefer the 5-layer ladder for the explicit discipline, the broader investigation framing when you need to *find* the bug rather than diagnose one already localized