# Espdl Quantize

> Iteratively tune esp-ppq QuantizationSetting to recover post-quantization accuracy on ESP-DL targets. Drives a closed loop of "baseline -> calibration × TQT(default) cartesian product -> distribution-aware residual fixes -> agent-driven open exploration -> re-evaluate" in the current Python environment, using a minimal user contract (calib dataloader + evaluate function). Generic across architectures (ResNet / EfficientNet / ViT / DETR / YOLO / LSTM and any esp-ppq-supported graph) — the search procedure is distribution-driven and does not depend on a specific network family. Method ordering is accuracy-first with a soft penalty for passes that slow down on-device inference; once the prescribed Phase-1/2/3 sequence exhausts, the skill hands control to the agent (Phase 5) with a structured history of improving levers + the per-iteration error artifacts to read, so the agent can compose multi-knob iterations (lever stacking, calibration cross-pollination, ablation, cost-trim) without a rigid template. LSQ on PO

- Skill: `espressif/espdl-quantize` (Agent Skill, multi-file: 24 files)
- Install (CLI): `npx skillmds@latest add espressif/espdl-quantize`
- Raw SKILL.md: https://api.skillmd.com/api/skills/espressif/espdl-quantize/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: espressif (https://skillmd.com/u/espressif)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/espressif/espdl-quantize

---


# ESP-DL Quantization Tuning Skill

This skill turns the human "stare at error report, guess setting, rerun" loop into a
structured, distribution-aware search. The user owns data loading and evaluation; the
skill owns `QuantizationSettingFactory.espdl_setting()` and the iteration loop.

> **About `<SKILL_DIR>` in shell snippets below.** This skill is agent-directory
> agnostic — it may be installed as `.cursor/skills/espdl-quantize/`,
> `.opencode/skills/espdl-quantize/`, or under any other agent's skills folder.
> Whenever you see `<SKILL_DIR>` in a shell command, substitute the **absolute path of
> the directory containing this `SKILL.md`** (the agent runtime gives you that path when
> it loads the skill). Setting it once at the start of a session makes the rest copy-pasteable:
>
> ```bash
> SKILL_DIR=/abs/path/to/espdl-quantize   # the directory holding this SKILL.md
> ```
>
> All in-skill markdown links (e.g. `[scripts/run_iteration.py](scripts/run_iteration.py)`)
> are already relative to `<SKILL_DIR>` and need no substitution.

## Generality boundary

This skill is a **general framework** for any esp-ppq-supported graph: ResNet,
EfficientNet, ViT, DETR, YOLO family, LSTM, custom CV/NLP backbones — anything
`espdl_quantize_torch` / `espdl_quantize_onnx` can ingest. The Phase 2 calib×TQT
cartesian product, the Phase 3 lever ordering, and the four Composition discipline
rules are all **distribution-driven**; none of them depends on a specific network
structure or family. The Worked example at the end uses MobileNet-V2 on ESP32-P4 as an
empirical demonstration — its concrete numbers are illustrative, not a model-selection
threshold.

## Why this skill exists

esp-ppq exposes a dozen tunable passes (calibration algorithm, layerwise equalization,
bias correction, weight split, mixed precision via dispatching table, TQT, LSQ, blockwise
reconstruction, ...). Each has 2-6 parameters. Trying them by hand is slow and error-prone.

What this skill brings to the table:

1. **Knowledge** — every esp-ppq method's principle, parameters, applicable scenarios, and
   anti-patterns are codified in [references/ppq_methods.md](references/ppq_methods.md).
2. **A decision rulebook** — given the top-K worst layers' input/weight/output distributions,
   [references/decision_playbook.md](references/decision_playbook.md) maps observed patterns to
   candidate methods.
3. **A fixed harness** — [scripts/run_iteration.py](scripts/run_iteration.py) takes the user's
   contract module plus a JSON setting and emits structured artifacts (metrics, layerwise error,
   per-layer stats) so the agent only has to read JSON to make the next decision.
4. **A search state machine** — [scripts/compare_iterations.py](scripts/compare_iterations.py)
   inspects what's already been tried and tells the agent which iteration to run next via
   `comparison.json["next_step_hint"]`. The hint embeds a complete `setting.json` template so
   the agent only has to fill in the rationale.
5. **Target-aware safety net** — the harness detects passes that conflict with the target's
   quantization policy:
   - **LSQ on POWER_OF_2 targets** (`esp32p4 / esp32s3 / c`) — auto-disabled. esp-ppq's
     `LSQDelegator` silently disables continuous-scale training under POWER_OF_2, so the
     pass would degenerate to weight-only tuning while paying full TQT-level PC time.
     Use TQT instead — it trains `log2_scale` and is POWER_OF_2-native.
   - **Layer-wise equalization on `esp32p4`** — **warn-only** (changed in this revision).
     esp-ppq officially marks the combination as "Not recommend"
      (see `esp-ppq/md_doc/Passes/LayerwiseEqualization.md`, "Usage" section),
     but empirical runs show some MobileNet-family / depthwise-separable networks still
     benefit. The harness now lets the pass run when `equalization.enabled=true` and
     emits a strong warning; the agent should treat it as a Phase 3 lever to try only
     after the calib×TQT cartesian product has settled.

## What the user has to provide

A single Python module (typically named `user_quant.py`) that exports:

- `QUANT_CONFIG` dict — model path, input shape, target chip, bits, primary_metric, etc.
- `create_calib_dataloader()` — returns the calibration `DataLoader`.
- `evaluate(quant_graph)` — returns a dict whose keys include `QUANT_CONFIG["primary_metric"]`.
- For torch flow only: `get_torch_model()` — returns the `nn.Module`.
- Optional: `collate_fn(batch)` and `evaluate_fast(quant_graph)`.

The full contract spec is in [references/contract.md](references/contract.md). Two ready-to-copy
examples live in [assets/user_quant_torch_example.py](assets/user_quant_torch_example.py) and
[assets/user_quant_onnx_example.py](assets/user_quant_onnx_example.py).

The skill never edits the contract module. All iteration state lives under `outputs/`.

---

## High-level flow

```mermaid
flowchart TD
    contract[user_quant.py] --> harness[run_iteration.py]
    setjson[outputs/iter_N/setting.json<br/>written by agent] --> harness
    harness --> ppqapi["esp_ppq.api.espdl_quantize_torch / _onnx"]
    ppqapi --> graph["esp-ppq BaseGraph<br/>(esp_ppq.IR.BaseGraph)"]
    graph --> lwerr["layerwise_error_analyse"]
    graph --> stat["statistical_analyse"]
    graph --> evalfn[user.evaluate]
    lwerr --> art[outputs/iter_N/]
    stat --> art
    evalfn --> art
    art --> compare[compare_iterations.py]
    compare --> hint["comparison.json<br/>next_step_hint"]
    hint --> phase{phase?}
    phase -- "1 / 2 / 3" --> agentTpl["agent fills rationale on embedded template"]
    phase -- "5 (open exploration)" --> agentFree["agent reads phase5_signals + artifacts,<br/>writes setting.json from scratch"]
    phase -- "4 (final)" --> finalize["outputs/best/ + outputs/final_report.md"]
    agentTpl -.writes next setting.-> setjson
    agentFree -.writes next setting.-> setjson
```

In Phases 1-3 the agent's job each round shrinks to: read
`comparison.json["next_step_hint"]`, copy the embedded `setting.json` template, fill in the
`rationale` field, run the harness. In Phase 5 the script stops prescribing settings — the
agent reads `phase5_signals` plus the per-iteration error artifacts and writes the next
`setting.json` from scratch (see "Phase 5 — Agent-driven exploration" below).

## Phases

### Phase 0 — Validate contract and environment

> **Important — ignore Docker / image / `/work` mentions you may see elsewhere.**
> Some user projects (and `user_quant.py` itself) still carry comments left over
> from an older Docker-based workflow — phrases like "build the image",
> "Phase 0 — docker 准备", `/work inside Docker`, or `docker run --gpus all`. Those
> are **legacy text only**, not steps to execute. The skill now runs entirely in
> the current Python interpreter.

Before any quantization, do these once per session:

1. Make sure the **current Python environment** has `esp_ppq` (with the `[cpu]` extra),
   `torch`, plus the small set of helpers the harness needs:

   ```bash
   pip install -e <path/to/esp-ppq>[cpu]
   pip install -r "$SKILL_DIR/assets/extra_requirements.txt"
   ```

   The skill is **environment-agnostic** — it does not require Docker. As long as
   `python -c "import esp_ppq, torch, onnx, onnxsim, pandas, scipy, tqdm"` succeeds, you are
   ready to go.

2. Validate the user's contract module imports cleanly and exposes the required functions/keys:

   ```bash
   python "$SKILL_DIR/scripts/run_iteration.py" \
     --user-quant <path/to/user_quant.py> \
     --output-dir <path/to/user_project>/outputs/contract_check \
     --check-contract
   ```

3. Make sure the iteration workdir exists (default: `<contract_dir>/outputs/`). The harness
   creates it on first run.

> The working directory for `python` should be the directory containing `user_quant.py` (or
> any directory — the harness resolves relative paths in `QUANT_CONFIG` against the
> contract module's directory).

### Phase 1 — Baseline (iter_0)

Run the default `QuantizationSettingFactory.espdl_setting()` once. The agent does NOT propose
any settings here — the harness uses a built-in baseline JSON when `--baseline` is passed.

```bash
python "$SKILL_DIR/scripts/run_iteration.py" \
  --user-quant <path/to/user_quant.py> \
  --output-dir <path/to/user_project>/outputs/iter_0 \
  --baseline
```

After it finishes, read these files:

- `outputs/iter_0/metrics.json` — what `evaluate()` returned, plus `_primary` shortcut.
- `outputs/iter_0/layerwise_error.json` — `{op_name: snr}` sorted descending by error.
  Covers **only `is_computing_op`** (Conv / Gemm / ConvTranspose / MatMul / Attention /
  PPQBiasFusedMatMul / LSTM); the SNR is the *isolated* contribution of that op when
  it alone is quantized.
- `outputs/iter_0/layer_stats.json` — `statistical_analyse` filtered by the layerwise
  top-K (legacy artifact; same coverage as layerwise).
- `outputs/iter_0/layer_stats_full.json` — **(new)** the full `statistical_analyse`
  output: every non-passive op's per-input/per-output distribution + cumulative SNR.
  This is the only artifact that includes Add / Concat / Resize / AveragePool /
  Sigmoid / Softmax / GRU / LayerNorm.
- `outputs/iter_0/non_computing_hot_ops.json` — **(new)** the top-K non-COMPUTING_OP
  layers ranked by max per-variable SNR, plus `inputs_float_std_ratio` (max/min
  Float Std of input variables, used by playbook rule R8).
- `outputs/iter_0/graphwise_jumps.json` — **(new)** adjacent computing-op pairs whose
  cumulative SNR gap is *not* explained by the downstream op's isolated contribution.
  Lists the intervening non-computing ops as suspected culprits.
- `outputs/iter_0/console.log` — full stdout/stderr.

Tell the user the baseline numbers, the top-5 error layer names from layerwise, and (if
non-empty) the top-3 entries from non_computing_hot_ops.json. The state machine in
`compare_iterations.py` decides when to finalize — do not stop here on your own even if
iter_0 looks like it hit `target_metric`; run the comparison once and let
`next_step_hint["phase"] == "phase-4-final-report"` confirm.

### Phase 2 — Calibration × TQT(default) cartesian product (mandatory)

This phase **must** run three iterations in strict sequence, each enabling exactly two
fields: `calib_algorithm` and `tqt_optimization` (with the esp-ppq default schedule). No
other pass is enabled. The cartesian product is what makes the search robust — calibration
in esp-dl quantization is **not separable** from the training pass: a calibration that
regresses standalone may become the strongest base when paired with TQT, and vice versa.
See the Worked example below for the empirical case that motivated this design.

The TQT default schedule is **strict**:

```json
{
  "lr": 1e-5,
  "steps": 500,
  "block_size": 4,
  "is_scale_trainable": true,
  "gamma": 0.0,
  "int_lambda": 0.0,
  "collecting_device": "cuda"
}
```

Iteration sequence:

| Iter | calib_algorithm | other passes | Purpose |
|------|-----------------|--------------|---------|
| `iter_1` | `kl` | TQT(default) | Pair iter_0(kl-only) and iter_1(kl+TQT) to read off the TQT-on-kl delta. If iter_1 hits target, stop. |
| `iter_2` | `mse` | TQT(default) | Same with mse. If hits target, stop. |
| `iter_3` | `percentile` | TQT(default) | Same with percentile. Often the hidden winner on heavy-tailed activations even when standalone percentile would regress. |

The **way to drive this** is to run [scripts/compare_iterations.py](scripts/compare_iterations.py)
between iterations:

```bash
python "$SKILL_DIR/scripts/compare_iterations.py" \
  --output-dir <path/to/user_project>/outputs
```

`comparison.json["next_step_hint"]` will be `phase-2-calib-tqt-sweep` until all three
calibrations are covered with TQT(default), and the embedded `setting.json` template can
be copied verbatim into `outputs/iter_<N>/setting.json` (only fill in the `rationale`).

> **Critical: iterations are strictly sequential — never run two in parallel.** Single
> GPU, calibration-data download race, and any of the three legs can short-circuit the
> rest if it hits `target_metric`. If you spawn parallel subagents the search breaks.

### Phase 3 — Residual fixes from best-so-far

After Phase 2 the best-so-far iteration is the one with the highest `primary_value` among
iter_0..3. `comparison.json["next_step_hint"]` switches to `phase-3-residual` (or
`phase-3-pivot` if the last two iterations both regressed vs best).

Each Phase 3 iteration **must** mutate from `comparison.json["best_iteration"]`'s
`setting.json` and change exactly **one** thing. The lever order below is the linear
default for `deploy_runtime_priority="balanced"`; lever 3a-3 is **conditional** (entered
only when one of two specific signals fires); the speed-priority reorder is described
under `Accuracy-first method ordering` below.

| Lever | Tier | On-device cost | What changes | When to use |
|-------|:----:|:--------------:|--------------|-------------|
| 3a-1 | A | 0 | TQT `steps: 500 → 1000` (lr=1e-5, block_size=4 unchanged) | Phase-2 winner is TQT-based and gap to target is non-trivial. One knob only — Composition discipline #2. |
| 3a-2 | A | 0 | TQT `lr: 1e-5 → 5e-5, steps: 1000 → 2000` (block_size=4 unchanged) | 3a-1 already gave a positive net effect. Do NOT push beyond this on the lr/steps axis (lr=1e-4 / steps=4000 stably regress on representative reproducers). |
| 3a-3 | A | 0 | TQT `block_size: 4 → 2` (lr/steps from best-so-far unchanged) | **CONDITIONAL — enter only on one of these two signals**: (1) **unstable fallback** — last iter was 3a-1 or 3a-2, regressed by < 0.5% relative AND introduced a new layer into the top-5 error list (TQT joint training perturbed a previously-quiet layer); or (2) **gap-shrink after convergence** — 3a-1/3a-2 both improved on best AND none of R5/R8/R3 structural triggers match in best's `layer_stats.json` / `non_computing_hot_ops.json`. Smaller block_size = closer to layerwise = more stable. Do not try block_size=1 (full layerwise, no upside) or block_size≥6 (overlaps lever 3g, unstable). |
| 3b | A | 0 | `bias_correct.enabled=true` | A top-error op's *output* row shows `|Noise Mean| > 0.1 × Noise Std`. |
| 3c | A | 0 | `fusion_alignment.align_elementwise_to = 'Align to Large'` (and friends) | R8 trigger fires on best's `non_computing_hot_ops.json`: a Concat/Add/Sub/Mul/Resize/AveragePool entry whose `max_snr ∈ (0.20, 0.30]` (primary Goldilocks band), OR `inputs_float_std_ratio > 5` (legacy reinforcement, fires outside the band too). Skipped above the band (`max_snr > 0.30` — residual too severe; use 3a-3/3d/3e), below the band (too little to fix), or via top-3 activation veto (`Relu/Swish/Sigmoid` `max_snr > 1.2× candidate` — activation-dominated, fix via TQT/int16). Constants live in `compare_iterations.py`; see references/decision_playbook.md §R8 for the empirical calibration. |
| 3d | A | 0 | enable `equalization` (full lever-3d template; do **not** abbreviate to `enabled=true` only — esp-ppq defaults `opt_level=1` while the template recommends `opt_level=2`, see Common pitfalls) | Conv→activation→Conv chain with weight per-channel `max/mean > 5`. **Per-tensor weight targets (`esp32s3 / c`) are the canonical use case; on `esp32p4` the pass is warn-only — esp-ppq officially "Not recommend" for per-channel weights but it can empirically help on some MobileNet-family / depthwise-separable nets.** |
| 3e | B | **+** | `dispatching_table` int16 on top 1-3 worst layers | One layer's SNR > 2× median of the top-5; structural fixes failed. ≤10% of total ops. **Permanent on-device runtime cost** (~2× cycles + ~2× activation memory on promoted ops). |
| 3f | B | **+** | `weight_split` on a single Conv with weight kurtosis > 10 | Equalization didn't fix it (or wasn't applicable on esp32p4). ≤3 split layers. **Permanent on-device runtime cost** (one extra Add op per split layer). |
| 3g | C | 0 | `blockwise_reconstruction` (last resort, **stacked on top of best**) | Tier A + Tier B all plateaued and gap > 5% absolute. GPU strongly recommended. The lever template no longer disables TQT — the engine runs `TrainedQuantizationThresholdPass` and then `AdaroundPass` sequentially (see `esp-ppq/esp_ppq/quantization/quantizer/base.py`), so the two passes coexist in the pipeline. PC quantization time roughly doubles vs the prior best, but accuracy attribution stays clean (blockwise is the only new variable). LSQ × {TQT, blockwise} remains hard-rejected by `apply_setting._check_mutex` because LSQ silently degenerates on POWER_OF_2 targets. |

The state machine in `compare_iterations.py` automatically picks the next lever per the
table above and the `deploy_runtime_priority` knob. The agent's job each Phase-3
iteration shrinks to: read `comparison.json["next_step_hint"]["advice"]`, copy the
embedded change snippet onto best-so-far's `setting.json`, fill `rationale` with the
specific layer-stats observation that drove the choice, run the harness.

Stop conditions are no longer the agent's call — when any of the following hold,
the state machine yields. Two of them finalise (Phase 4); two of them hand
control to the agent (Phase 5). Note that the Phase-3 cap fires at 5 iterations
even though the linear-order list has 8 levers — see "Why `_PHASE3_CAP=5` leaves
untried linear-order levers" in the Phase 5 section below for the trade-off and
how Phase 5 picks up the slack.

| Stop condition | Routes to | Why |
|----------------|-----------|-----|
| `primary_metric` reached `target_metric` | **phase-4-final-report** | Target met — keep poking is a waste. |
| Plateau: last 3 iterations all within 0.1% relative of best | **phase-4-final-report** | Accuracy stopped moving; Phase 5 has the same problem. The window is 3 (not 2) because real iteration histories often have a single sub-0.1% wobble in the middle of an otherwise-improving run; requiring 3 consecutive flat iterations rules out that false-positive. |
| `_PHASE3_CAP` (= 5) Phase-3 iterations run after Phase 2 AND target NOT reached | **phase-5-agent-driven** | The linear search ran out of cap-budget but the metric is still moving — let the agent explore. The unfilled tail of the linear list shows up in `phase5_signals.untried_phase3_levers`. |
| All linear-order Phase-3 levers (3a-1, 3a-2, 3b, 3c, 3d, 3e, 3f, 3g) tried or correctly skipped AND target NOT reached | **phase-5-agent-driven** | Same idea — the prescribed list is exhausted, but the model has more accuracy to give. |

Lever 3c is **correctly skipped** when R8 doesn't fire on best-so-far's
`non_computing_hot_ops.json` (see "R8 trigger" below): the data says fusion
alignment would regress, so the state machine doesn't burn an iteration on it
and instead advances to 3d. The skip is recorded in
`comparison.json["next_step_hint"]` so the agent (and the human reviewing later)
sees why the lever was bypassed.

### Phase 5 — Agent-driven exploration

Phase 5 exists because the Phase-3 linear search is, by design, surgically narrow.
Each Phase-3 lever changes exactly one thing on top of best-so-far. That's the right
shape when you're hunting for the next single biggest fix, but it can't find
**combinations**: a recipe that needs `percentile + TQT + equalization + 3-layer int16
+ bias_correct` (the `example_quantize_mobilenetv2_esp32p4/outputs/` winning configuration,
iter_14) is unreachable from any single Phase-3 lever applied to any single Phase-2 leg.
Worse, Composition discipline #4 (calibration is not separable from the training pass)
means even Phase 2 cannot tell you whether `percentile` becomes the best calibration once
equalization + int16 + bias are stacked on top of it.

In Phase 5 the state machine yields and the agent drives. The contract:

- The hint in `comparison.json["next_step_hint"]["advice"]` is **meta-guidance**, not a
  setting.json template. There is no `iteration_id` skeleton to fill in — you write the
  next setting.json from scratch.
- The hint is paired with a structured `comparison.json["next_step_hint"]["phase5_signals"]`
  block that summarises history: which iterations improved over their prior best (and by
  how much), which regressed (so you don't re-stack their changes), what calibration
  algorithms haven't been tried on top of the current lever stack, and pointers to the
  on-disk artifacts to consult before proposing the next change.
- Each Phase-5 iteration must still `mutate from best-so-far` (Composition discipline #1)
  and `stop escalating after one regression` (#3). Discipline #2 (one knob per iteration)
  is relaxed — see Composition discipline #5 below.

**Inspiration patterns** (these are starting points; let the actual data decide which
one applies on your model):

- **STACK improving levers.** When `phase5_signals.improving_levers` lists ≥2 entries (e.g.
  iter_5 enabled `tqt_optimization`, iter_9 enabled `equalization`), the first natural
  Phase-5 iteration is one that turns BOTH on at once. The mobilenetv2-p4 path went
  iter_11 (TQT + equalization + int16x3) → iter_13 (added calibration swap, +0.55%) →
  iter_14 (added bias_correct, +0.05%, final best).
- **CROSS-POLLINATE CALIB.** When `phase5_signals.untried_calib_swaps` is non-empty, run a
  single iteration that swaps the calibration on top of the current lever stack. The
  Phase-2 cartesian product evaluates calib × TQT(default) in isolation; once 3-4 levers
  are stacked the ranking can flip — exactly what happened on mobilenetv2-p4 when
  percentile, which had previously been considered a calibration loser, became the winner
  once stacked with TQT + equalization + int16.
- **ABLATE.** Once Phase 5 finds a new best, drop one component at a time and check
  whether accuracy stays above target. This produces a "minimal recipe": fewer passes,
  fewer surprises, shorter PC quantization time. Two ablation directions are particularly
  useful — drop the highest-cost component (e.g. one int16 op, or `weight_split`) for
  on-device speed; drop a Tier-A pass (TQT off, equalization off) to test which passes
  are actually load-bearing.
- **DIVE INTO ARTIFACTS.** When the above three don't produce an obvious next move, open
  best's `layerwise_error.json`, `layer_stats_full.json`, `non_computing_hot_ops.json`,
  and `graphwise_jumps.json`. Pick a layer with a concrete distribution observation
  (e.g. "Conv layer X has Float Std skew >5 but the weight per-channel max/mean is only
  2.3 — equalization won't help; this is a high-variance activation, try TQT escalation
  or int16 on this single op") and write the next iteration around that observation. Cite
  the file + the number in `rationale`.

**Stop signals** (each → finalize). Two are auto, two are agent-driven:

1. **`primary_metric` reached `target_metric`** — `compare_iterations.py` AUTO-finalizes
   via `phase-4-final-report`. `final_report.md` records `Stop reason category: target_reached`.
2. **Plateau** — last 3 iterations all within 0.1% relative of best.
   `compare_iterations.py` AUTO-finalizes via `phase-4-final-report`. `final_report.md`
   records `Stop reason category: plateau` plus the three plateau values.
3. **User-given iteration budget reached** — agent runs `--finalize --force-finalize`
   NOW, regardless of phase and regardless of remaining untried patterns/levers. **User
   budget is the hard ceiling.** `--force-finalize` is the explicit opt-in that confirms
   "this early stop is intentional"; `final_report.md` records
   `Stop reason category: force_finalize_phase5` plus the untried lists so the user can
   see what was skipped.
4. **Coverage-exhausted "ran out of ideas"** — STRICT. Only fires when ALL the following
   hold simultaneously:
   - the user did NOT give a specific iteration budget;
   - `phase5_signals.untried_phase5_patterns` is empty (every pattern attempted at least
     once);
   - `phase5_signals.untried_phase3_levers` is empty (every linear-order Phase-3 lever
     either tried or correctly skipped by its trigger);
   - `phase5_signals.untried_5beta_reapply` is empty (every calib swap re-tested on the
     current deepest stack — see "5β-reapply" below);
   - the most recent iterations did not produce a new best.

   **If signal (3) is in play, signal (4) is disabled.** Keep iterating until the user
   budget is met, drawing fresh variations from the untried lists. Phase 5 has NO hard
   iteration cap from the state machine; the user is the cap.

The `comparison.json["early_finalize_command"]` field always contains the one-line
`--finalize` invocation; the stdout "Tip" block reprints it after every run.

**Hard-reject of premature `--finalize`**: if you run `--finalize` while still in
`phase-5-agent-driven` and neither target nor plateau is met AND you do NOT pass
`--force-finalize`, `compare_iterations.py` PRINTS THE REJECTION BLOCK, **REFUSES TO
WRITE** `outputs/best/` or `outputs/final_report.md`, and **EXITS WITH CODE 1**. The
agent must either (a) re-run without `--finalize` and keep iterating, or (b) pass
`--force-finalize` to confirm intentional early stop. This is the operational
enforcement of the user-budget contract in signal (3); see "How premature finalize is
prevented" below for the rationale.

**5β-reapply** (the high-leverage coverage gap that needs explicit tracking):
Composition discipline #4 says calib-only ranking does not predict the combined ranking,
but Phase 2 runs before any levers are on. The corollary is that an early 5β
CROSS-POLLINATE attempt on a *shallow* stack can produce a misleading verdict — the
same calib swap on the *current deepest* stack may behave very differently. The
canonical example is `example_quantize_mobilenetv2_esp32p4` iter_13: percentile lost
to kl on the Phase-2 stack but won by +0.55% on the deepest lever stack. The skill
tracks this via `phase5_signals.untried_5beta_reapply`: a list of calibrations that
appeared as 5β targets earlier in history but were not re-tested on the current best's
stack. The Phase 5 hint surfaces the list explicitly, and stop signal (4) is blocked
while it is non-empty.

**Tunable-params soft advisory**: the Phase 5 hint includes a "Tunable parameters in
current best" section listing the parameter knobs available inside each enabled pass
(TQT `lr` / `steps` / `block_size`, blockwise `lr` / `steps` / `block_size`,
equalization `opt_level` / `iterations`, fusion_alignment direction, percentile
calibration `percentile`) with common value ranges drawn from `references/ppq_methods.md`.
This is a SOFT advisory — NOT part of coverage. The agent reads the section, decides
whether the layerwise / non_computing_hot_ops data justifies a knob change, and proposes
the next iteration accordingly. Tuning a parameter within an already-enabled pass is a
valid Phase-5 move; not every variation requires turning a pass on/off.

### User-budget enforcement in Phase 5

Phase 5's "no hard cap" property means the state machine will keep emitting hints
forever if you let it. The user budget is what bounds the loop. Concretely:

- Track the user-budget count in your head (or in scratchpad). Increment after each
  iteration completes.
- After the N-th iteration where N == user budget, run the `--finalize` command and
  stop.
- **Never** invoke stop signal (4) when a user budget is in play — even if all
  patterns and levers are covered. Spend the remaining budget on variations of
  attempted patterns: re-stack improving levers in different combinations, try the
  same pattern on a different layer subset, ablate a different component, dive into
  a different artifact than last time. Variation under user budget is REQUIRED;
  fabricating defensible variations is part of the Phase 5 contract.
- The mechanism that prevents the wrong call here is **rationale citation discipline
  #5**: every iteration must name the iter id(s) whose data motivated the change. If
  you cannot name any prior iter that motivates the next change AND the user budget
  remains, look at a different artifact, find a number you can name, and use that
  as your rationale — do not finalize.

### Why `_PHASE3_CAP=5` leaves untried linear-order levers (by design)

The Phase-3 linear-order lever list has 8 entries (`3a-1, 3a-2, 3b, 3c, 3d, 3e, 3f,
3g`), but `_PHASE3_CAP=5` caps the number of structured single-knob Phase-3 iterations
at 5. The trade-off this cap encodes:

- **Pro**: Phase 5 can start exploring sooner (cross-pollination + ablation + stacking
  are higher-leverage moves than the tail of the linear list in many real cases — see
  the mobilenetv2-p4 iter_13 +0.55% jump).
- **Pro**: when `target_metric` is hit early in Phase 3, the cap is irrelevant — the
  short-circuit fires first.
- **Con**: levers near the end of the linear order (typically 3d / 3e / 3f / 3g) are
  often left untried when the cap fires. The 3a-3 conditional path can also occupy
  a cap slot, making this worse.

The skill closes the gap by treating those untried levers as **first-class Phase 5
coverage targets**. `phase5_signals.untried_phase3_levers` lists them by id; the hint
explicitly directs the agent to STACK each onto best-so-far as a Phase-5 iteration
before stop signal (4) can fire. Functionally a Phase-3 single-knob mutation and a
Phase-5 STACK iteration produce the same setting (both `mutate from best + flip one
lever`), so coverage is preserved — just under a different label and with a slightly
larger lever-stack baseline.

**Worked example**: in `example_quantize_mobilenetv2_bad_esp32s3/outputs/`, the
20-iteration run only produced 12 iterations because the agent (a) interpreted the old
"Pretending there's a 5th idea when there isn't one is worse than finalising" line as
permission to bail, and (b) had no visibility into the fact that `3f weight_split` and
`3g blockwise_reconstruction` were untouched and `5gamma ABLATE` + `5delta DIVE-INTO-
ARTIFACTS` were untried Phase-5 patterns. Under the current contract:

- The "Pretending..." sentence is gone — replaced by the hard rule that user budget
  trumps signal (4).
- The hint now surfaces `untried_phase3_levers=[3f, 3g]` and
  `untried_phase5_patterns=[5gamma, 5delta]` as named targets.
- Premature `--finalize` in phase-5 (target not met, no plateau) is **HARD REJECTED**:
  `compare_iterations.py` refuses to write `outputs/best/` / `outputs/final_report.md`
  and exits with code 1. The agent must either continue iterating or explicitly pass
  `--force-finalize`. The earlier soft-warning version of this guardrail was a load-
  bearing failure mode that produced both this 12-of-20 run and the later 18-of-30 /
  21-of-30 runs in `example_quantize_mobilenetv2_esp32p4_tmp/outputs/`.

These three together make the iteration-budget mismatch self-correcting.

### How premature finalize is prevented

The hard-reject contract for `--finalize` is the operational enforcement of the user-
budget rule in stop signal (3). Concretely:

1. The agent (or user) runs `compare_iterations.py --finalize` while
   `phase == phase-5-agent-driven` and the script computes `target_metric` is NOT
   reached and the recent iterations are NOT a plateau.
2. `compare_iterations.py` prints the rejection block, listing
   `untried_phase5_patterns`, `untried_phase3_levers`, and `untried_5beta_reapply`.
3. `comparison.json` is still written (so the agent can re-read the hint), but
   `outputs/best/` and `outputs/final_report.md` are NOT touched.
4. The script exits with code 1.
5. The agent picks ONE of:
   - re-run without `--finalize` and continue iterating (use the printed untried lists
     as concrete next targets); OR
   - pass `--force-finalize` alongside `--finalize` to confirm intentional early stop.
     The resulting `final_report.md` has a `## Stop reason` section with category
     `force_finalize_phase5` plus the untried lists so the user can see exactly what
     was skipped.

Why hard-reject (not soft warning): the soft warning was ignored both in the bad-
esp32s3 12-of-20 run and in the esp32p4_tmp 18-of-30 / 21-of-30 runs. Making the
reject load-bearing means the agent literally cannot produce a final_report.md by
accident in phase-5 — a positive write requires `--force-finalize`, which the agent
will only emit when the user budget rationale is solid.

### Phase 4 — Final report

**Two ways to enter Phase 4:**

* **State-machine trigger** (machine view): `comparison.json["next_step_hint"]["phase"] == "phase-4-final-report"`. The state machine emits this when target reached, plateau, Phase-3 cap, or all linear-order levers tried (see Phase 3's stop-condition list).
* **User-budget trigger** (human view): the user gave the agent a specific iteration budget — phrasings like "iterate 3 times", "迭代 3 轮", "只跑 N 轮", "iterate N times", "最多 N 轮", "only N iterations". When this budget is hit, **the user-budget trigger always wins** even if the state machine still wants to keep going.

**Why the auto-finalize is bullet-proof.** `compare_iterations.py` writes `outputs/best/` and `outputs/final_report.md` whenever **either** trigger fires:

* **Automatic** when `phase == "phase-4-final-report"` — every invocation of `compare_iterations.py --output-dir <outputs>` checks this and finalizes if true. Agents reading the script's stdout will see a `[compare] phase-4 detected; finalize results: ...` block.
* **On demand** via the `--finalize` flag at any time, regardless of phase. This is the escape hatch for the user-budget case — agents should copy the command from `comparison.json["early_finalize_command"]` (or the printed "Tip: how to wrap up at any time" block at the bottom of `compare_iterations.py`'s stdout) and run it after the last user-budgeted iteration completes.

The generated `final_report.md` carries an HTML marker comment on its first line. Subsequent finalize runs detect the marker → safely refresh the report (no data loss). If an agent has hand-edited the file (and removed the marker), subsequent finalize preserves it untouched unless `--force` is passed. Sections `## Key findings` and `## Remaining gap (if target not met)` are seeded with auto-bullets but agents are explicitly invited (via the marker comment) to expand them with concrete distribution interpretations from `layer_stats.json` / `non_computing_hot_ops.json` / `graphwise_jumps.json`.

**Iteration history table — new columns.** The auto-generated table now includes:

* **`rank`** — dense ranking by `primary_value` (1 = best). Recomputed from disk on every finalize, so adding more iterations later won't reverse the relative order of any pre-existing pair (this is asserted by the unit tests). Columns visible in the report and in `comparison.json["iteration_ranks"]`.
* **`affects inference speed`** — `"No"` for almost all settings; `"Yes (...)"` only when the iteration enables `dispatching_table` int16 promotion or `weight_split` (the only two passes with permanent on-device runtime cost; see "On-device runtime cost cheat-sheet"). When the **best** iteration has `affects inference speed = Yes`, **inspect the rank-2/3 rows** — if they trade < 0.1% accuracy for `affects inference speed = No`, the user may prefer the runner-up for production deployment.

**Recommended agent workflow after finalize:**

1. Read `outputs/final_report.md`. The Summary, Iteration history (with rank + speed columns), Best setting, Python snippet are auto-generated; expand `## Key findings` and `## Remaining gap` with concrete bullets if the model warrants.
2. **Run a single full-eval re-check** to replace the iteration loop's `evaluate_fast()` number with the user's real `evaluate()`:
   `python {SKILL_DIR}/scripts/run_iteration.py --user-quant <...> --setting outputs/best/setting.json --output-dir outputs/iter_<NEW> --use-full-eval`.
   If the resulting `<primary_metric>` differs from what's in the Summary, update the Summary line in `final_report.md`. The marker line keeps the file refreshable; once you remove the marker (or pass `--force` from the script), subsequent automated runs won't clobber edits.
3. If you ever need to regenerate the report from scratch (e.g. after fixing a bug in an iteration), run:
   `python {SKILL_DIR}/scripts/compare_iterations.py --output-dir <outputs> --finalize --force`.

> **Legacy fallback.** The pre-auto-finalize workflow (manually run `--write-best`, hand-write `outputs/final_report.md`) is still supported for completeness — if for any reason `compare_iterations.py` does not emit the artifacts (e.g. broken iteration data on disk), the agent can fall back to that flow. The `--write-best` flag now writes only `outputs/best/`; the report remains the agent's responsibility in that fallback.

**Final-report template (auto-emitted by the script, for reference / audit):**

```
<!-- auto-generated marker line — agents may edit Key findings / Remaining gap -->
# Final Report: <model> on <target>

## Summary
- Best iteration: iter_<N>
- <primary_metric>: <value> (target_metric=<target or "not set">)
- _Note: value comes from evaluate_fast(); run --use-full-eval to refresh._
- On-device speed cost vs baseline (best): <No | Yes (...)>
- Other metrics: <copy from outputs/best/metrics.json>

## Iteration history
| iter | method changed | <primary_metric> | delta | outcome | rank | affects inference speed |

## Best setting
<inline the FULL outputs/best/setting.json>

## Python snippet
<auto-translated QuantizationSettingFactory.espdl_setting() recipe>

## Key findings
<auto-bullets — agents extend with concrete distribution interpretations>

## Remaining gap (if target not met)
<auto-bullets — agents replace boilerplate with model-specific recommendations>
```

---

## Composition discipline (read before every iteration)

These rules govern the iteration loop. Violating any of them = current iteration is
discarded, agent rolls back to best-so-far and re-runs.

1. **Mutate from best-so-far, not the last iteration.** Always start the next
   `setting.json` from `comparison.json["best_iteration"]["dir"]/setting.json`. The most
   recent run can be a regression you should not inherit.

2. **One new method (or one parameter change) per iteration (Phases 1-3 only).**
   Calibration algorithm and `tqt_optimization` with the default schedule are treated as
   the **conjoined Phase 2 base** — they enter and leave together inside Phase 2. Outside
   Phase 2 and inside Phase 3, change exactly one knob. If two changes are stacked and
   the iteration regresses, you can't tell which one hurt. (Phase 5 relaxes this rule —
   see #5.)

3. **Stop escalating after one regression.** If iter_N raises a TQT hyper-parameter (or
   tightens a lever) and the metric drops, do not push further on that axis; pivot to a
   different lever from the Phase 3 table.

4. **Never retire a calibration algorithm based on its calib-only score.** Calibration is
   the input distribution shaper for downstream passes (especially TQT in esp-dl, where
   POWER_OF_2 makes TQT the only available training-based pass). Calib-only accuracy does
   **not** predict combined accuracy: percentile may regress standalone but become the
   strongest TQT base, because tail-clipping leaves more "training space" for TQT to
   recover. Phase 2 must always evaluate calibration with `calib × TQT(default)` cartesian
   product, never with calib-only ranking. The same principle applies in Phase 5:
   calibration ranked against the Phase-3 lever stack can flip vs the Phase-2 cartesian
   ranking — re-test untried calibrations after the lever stack settles. See Worked
   example below for the concrete reproducer.

5. **Phase 5 multi-knob changes are allowed iff the rationale cites historical evidence.**
   The "one knob per iteration" rule (#2) is relaxed in Phase 5 — an iteration may stack
   2+ passes — but only when each pass change names the specific iteration whose data
   motivates it (e.g. "iter_5 showed lever 3a-3 stabilised the perturbed layer; iter_9
   showed equalization improved bottleneck Conv chains; combining them tests whether the
   gains compound"). Without citations, a multi-knob change is a guess and must be split
   into single-knob st

…(truncated)