evolve — Darwin Mode self-improvement
kimi-k3-harness ships with Darwin Mode (@metaharness/darwin, ADR-070…146): the model
is frozen; the harness evolves. Each generation mutates ONE of the 7 surface files
(planner, contextBuilder, reviewer, retry/tool/memory/score policy), sandboxes each
child, scores it, and keeps only variants that measurably improve — building an
archive of successful descendants.
Run it
npm run evolve # real substrate: runs your test command per variant (deterministic mutator — no API key, no network)
npm run evolve:dry # mock substrate: fast, fully offline, no test execution
Or directly:
npx metaharness-darwin evolve . --sandbox real --generations 3 --children 4
Safety (secure by default)
- Deterministic mutator is the default — no network, no API key, air-gapped.
- Every mutation passes the
validateGeneratedCode gate: no new imports, network,
filesystem, shell, env access, or dependencies — pure refactor/tuning only.
- Mutations run in a sandbox; only variants that pass your tests are archived.
- Nothing is promoted without measured improvement (guard against Goodharting).
See @metaharness/darwin for selection strategies (--selection, --crossover,
--curriculum), statistical gates (--fdr, --bench), and the real-LLM mutator (library API).
What the benchmarks taught us (measured, full SWE-bench Lite 300)
Defaults worth carrying into how you evolve and run this harness (full evidence + CIs in
@metaharness/darwin's LEARNINGS.md / bench/results/RESULTS.md):
- Closed-loop repair is the #1 lever (~2×). Feeding test/compiler failure back and retrying took
resolve-rate 7.7% → 15.3% on the same cheap model. Iterate against ground truth, don't single-shot.
- Cheap-first + cost-aware routing. Track $/resolve, not just resolve-rate; a cheap model
resolved 31× cheaper per fix than a frontier one. Reserve frontier for measured capability gaps.
- Tier the models (Barbarian & Scholar). Cheap sweep + frontier on only the residual = 33.3%
at ~6× lower cost than running frontier everywhere.
- Put the output-format contract in a system message + example, and size prompts to the model's
real context window — this alone took a weak local model from 0% to ~50% valid output.
- Only trust batch evaluation of the final artifact — in-loop counters drift 1.5–5×.
- The harness multiplies the model; it can't rescue one below the task's reasoning floor. Pick
the smallest model above the floor, then let evolution do the rest.
1---2name: evolve3description: Evolve this harness with Darwin Mode — frozen model, evolving harness (real, sandboxed, safety-gated).4---56# evolve — Darwin Mode self-improvement78`kimi-k3-harness` ships with **Darwin Mode** (`@metaharness/darwin`, ADR-070…146): the model9is frozen; the *harness* evolves. Each generation mutates ONE of the 7 surface files10(planner, contextBuilder, reviewer, retry/tool/memory/score policy), sandboxes each11child, scores it, and keeps only variants that *measurably* improve — building an12archive of successful descendants.1314## Run it1516```bash17npm run evolve # real substrate: runs your test command per variant (deterministic mutator — no API key, no network)18npm run evolve:dry # mock substrate: fast, fully offline, no test execution19```2021Or directly:2223```bash24npx metaharness-darwin evolve . --sandbox real --generations 3 --children 425```2627## Safety (secure by default)2829- **Deterministic mutator** is the default — **no network, no API key, air-gapped**.30- Every mutation passes the `validateGeneratedCode` gate: no new imports, network,31 filesystem, shell, env access, or dependencies — pure refactor/tuning only.32- Mutations run in a **sandbox**; only variants that pass your tests are archived.33- Nothing is promoted without measured improvement (guard against Goodharting).3435See `@metaharness/darwin` for selection strategies (`--selection`, `--crossover`,36`--curriculum`), statistical gates (`--fdr`, `--bench`), and the real-LLM mutator (library API).3738## What the benchmarks taught us (measured, full SWE-bench Lite 300)3940Defaults worth carrying into how you evolve and run this harness (full evidence + CIs in41`@metaharness/darwin`'s `LEARNINGS.md` / `bench/results/RESULTS.md`):42431. **Closed-loop repair is the #1 lever (~2×).** Feeding test/compiler failure back and retrying took44 resolve-rate 7.7% → 15.3% on the *same cheap model*. Iterate against ground truth, don't single-shot.452. **Cheap-first + cost-aware routing.** Track **$/resolve**, not just resolve-rate; a cheap model46 resolved 31× cheaper per fix than a frontier one. Reserve frontier for *measured* capability gaps.473. **Tier the models (Barbarian & Scholar).** Cheap sweep + frontier on *only the residual* = 33.3%48 at ~6× lower cost than running frontier everywhere.494. **Put the output-format contract in a system message + example**, and size prompts to the model's50 real context window — this alone took a weak local model from 0% to ~50% valid output.515. **Only trust batch evaluation of the final artifact** — in-loop counters drift 1.5–5×.526. **The harness multiplies the model; it can't rescue one below the task's reasoning floor.** Pick53 the smallest model *above* the floor, then let evolution do the rest.
ruvnet