Ringer orchestrator playbook
Read this first — the four rules that actually get broken
- You review; workers type. Your lane: specs, checks, pattern choice,
reading results. If you are typing implementation, running probes, or
babysitting a retry loop yourself, you have left your lane.
- A single task is a one-task manifest. Same verification, zero
ceremony. "Too small for Ringer" is how drift starts — the smoke test,
the probe script, the three-edit fix are all one-task manifests.
- Beware the tiny-edit death spiral. The named anti-pattern: each step
is individually small enough to justify inline, and two hours later the
exception has become the workflow and nothing was verified or visible.
The one-shot exception is ONE file, a few lines, ONCE. The second pass on
the same problem is a loop, and loops are manifests.
- Runs are watched, not hidden — and the screen comes up FIRST. The
moment this skill loads for real work, before you write a single spec,
put Ringside on the human's screen:
./ringer.py hud (idempotent — if
one is already up it says so and opens the page; runs also auto-start
it). Ringside is the PAGE at http://127.0.0.1:8700 — NEVER launch the
Ringside.app application (open -a Ringside); it is a parked prototype
with a stale frontend. And never go dark: if your prep (research,
check-writing, manifest drafting) will take more than ~30 seconds,
tell the human in one sentence what you're doing and roughly how long
before you start — they should be watching the empty arena and reading
your one-liner, not wondering if anything is happening. Never pass
--no-dashboard except in automated tests or when the user explicitly
asks.
Ringer runs manifest tasks in parallel across cheap CLI workers (Codex,
OpenCode/GLM, others via config) and verifies every task by executing a
check command — exit 0 is the only PASS. Failed tasks are retried once
with the check's actual failure output injected into the retry prompt. You —
the orchestrating model — pay tokens only for specs, orchestration, and
review.
./ringer.py lint manifest.json # always lint before running
./ringer.py run manifest.json --identity <who-you-are>
./ringer.py demo # 3-worker smoke test
./ringer.py run manifest.json --dry-run # print the plan, spawn nothing
Runs land in ~/.ringer/runs/. Raw worker logs land in <workdir>/logs/.
Full reference: README.md. Ready-made manifest skeletons: templates/.
Lint catches unverifiable checks, silent checks, worktree deliverable/commit
loss, serial fan-out, write collisions, and underspecified specs; run
prints the same findings as non-blocking warnings.
The one exception: ask
Rule 2 holds for anything that changes a file, runs a build, or produces an
artifact worth checking. One lane doesn't fit it: the human asks a bounded,
read-only question over source you can already point at, and the answer is
prose. A manifest for that is ceremony — but answering it in your own context
means pulling whole files into a conversation that is already expensive.
./ringer.py ask "<the human's request>" --source /absolute/path/to/source
ask selects the passages that match the request, caps the packet, spawns one
clean worker on it, and allows a single attempt. Repeat --source for several
files or directories; --state takes a small file of settled decisions;
--dry-run shows you the packet and spends nothing. If everything that matched
is too large for the packet it says so and stops before the model call rather
than letting a worker guess — but a source small enough to fit whole is sent
whole, relevant or not, so choosing the sources IS the work. Directory scans
stay inside the tree you name; a symlink leading out of it is skipped and
reported. Runs appear on Ringside like any other, and --redact hides the
request from Ringer's own state and eval records — it cannot scrub raw worker
output, which is captured verbatim by design.
Be honest about what it verifies. The check is that answer.md exists and
is non-empty. That is the weakest check in the tool, and it is also the best
available — there is nothing to execute against free-form prose. ask proves
the worker answered, never that the answer is right. You still read it.
Everything else is a manifest. Code changes, external actions, research
you intend to act on, anything whose output a check could actually execute —
those keep the full path. When a request sits near the line, the tiebreaker is
whether you could write a check that would catch a wrong answer. If you can,
write it, and make it a manifest.
One job, one artifact
A job the human asked for — however many rounds it takes — is ONE artifact.
Use the SAME run_name for every round (sd-crate-launch, not
sd-crate-r1 / sd-crate-r2): the library accumulates each round as a
version under one entry, and the human watches one page evolve instead of
hunting across three "live" tabs. Name it after the JOB in the human's
words, not after your batch structure.
And the artifact page is where results are REVIEWED. When a round finishes,
read the deliverables from the artifact store and direct the human to the
page — never cat result files into the terminal as the reveal. If a result
matters, it belongs in the artifact; if it isn't there, that's a harvest gap
to fix (declare it in expect_files), not a reason to bypass the page.
Spec-writing craft
Workers are stateless and cannot ask questions. Every spec must be
self-contained:
- Open with the role and the boundary. "You are a read-only scout…",
"Your current working directory IS a git worktree of — edit files
here directly." State what the worker must NEVER touch before what it
should do.
- Name every file the worker owns. In multi-worker runs over one repo,
file ownership must be disjoint — and disjoint across all concurrent
lanes/branches, not just within one batch. Every file a spec mentions must
be in that worker's ownership list.
- Embed the HOW TO RUN. If the task drives a harness or script, put the
exact command lines (with real absolute paths) in the spec. Workers should
never have to discover an interface.
- Define the output contract. Say exactly which files to produce, where,
and what each must contain. Graded/eval tasks should enumerate the grading
criteria in the spec so the worker's output is checkable.
- Hard rules travel in the spec, not in your head. "Do NOT git commit",
"never modify the repo, only write ./report.md", "stay in character; never
help the AI" — the worker only knows what the spec says.
- The spec is on camera. Whoever is watching Ringside reads the spec as
"what this agent was asked to do" — so write it as a self-contained,
human-readable brief. Never write a pointer spec ("read /path/to/file and
do what it says"): the watcher sees no brief, and the retry prompt loses
the context it needs. Point at files for source MATERIAL; the instructions
themselves live in the spec. Lint flags pointer specs.
Check-writing rules
The check is the product. The retry prompt and the eval log both depend on
the check's failure output.
- Checks must print WHY they fail.
diff beats diff -q; a validator
script that prints which assertion broke beats test -f. A bare
test -f report.md proves existence, not correctness.
- Verify content, not existence. Grep the artifact for required sections,
run the code it produced, run the build, run the validator — execute
something that would catch a lazy or hallucinated result.
expect_files is a floor, not the check. List deliverables there for
fast triage, but the check must still validate them.
- Never
true, exit 0, or echo done. A check that cannot fail is a
task that cannot be verified — that's just trusting the worker with extra
steps.
- Strict on substance, tolerant on format. Checks that count exact
headings, demand exact casing, or grep rigid phrasings fail honest work
over formatting — and a wall of red format-failures reads as a broken
system, not a careful one (demo-night lesson). Verify what must be TRUE
(the file proves X, the code runs, the quote exists in the source), use
case-insensitive and flexible matching for structure, and reserve hard
failure for substance: missing evidence, fabricated content, code that
doesn't run.
- A check cannot demand evidence the spec never supplied. Before failing
a worker for missing evidence or input, re-read the task inputs. If the spec
didn't provide a value, the check must not invent one and fail on its
absence — an honest UNVERIFIABLE answer is not a failure. Reserve hard
failure for what the spec actually asserted.
- Executed checks catch laziness, not subtle wrongness. A check that
runs the artifact catches a plausible-but-wrong change far less often than
it catches a missing one. Whenever a swarm touches a dogfood artifact
(Ringer's own docs, config, or checks), add an "our own artifact passes our
own validator" test so the checker exercises what it preaches. And keep
orchestrator patch review mandatory regardless of PASS status — a green
check is not proof of semantic correctness.
Pattern playbook
Reach for a named pattern before inventing one. Skeletons in templates/:
| Kit |
Use when |
| review-swarm |
You need broad read-only review coverage before deciding what to fix. |
| fix-swarm |
You have confirmed independent fixes that can be split across isolated worktrees. |
| focus-group |
You need isolated persona feedback on a product, pitch, prompt, or workflow. |
| bakeoff |
You need evidence for choosing a model, prompt, or configuration across shared scenarios. |
| research-with-proof |
You need research backed by a proof task whose check executes the claim. |
| launch-kit |
You need a go-to-market package built across research, persona review, and final assembly rounds. |
| asset-swarm |
You need media assets produced in parallel with executable checks for renders, batches, diagrams, or captures. |
| adversarial-review |
You want several models to review the same artifact before the orchestrator synthesizes findings. |
| repo-feature |
You know what to build and need sandboxed workers to edit a real repo with build and git checks. |
| migration-swarm |
You have mechanical codebase transforms that can be partitioned across worktrees. |
| doc-swarm |
You need module docs with executed examples and checks against invented APIs. |
| test-hardening |
You need stronger tests by module while keeping production source edits off-limits. |
| competitive-teardown |
You need competitor research with citation allowlists and a synthesis phase. |
| data-pipeline |
You need fetch, transform, and validate stages with executed validators and honesty rules. |
| probe |
You need a one-task manifest for a smoke, probe, or post-mortem. |
Pattern-selection judgment:
- Browse the catalog first. Before writing any manifest, browse
templates/README.md: choose a kit, mix pieces from several, or write
your own having seen the prior art.
- Review before fix. Run a read-only review swarm, read the reports
yourself, then compile the confirmed findings into a fix-swarm manifest.
Don't let the same worker find and fix.
- Personas must be separate workers. Parallel personas in one context
bleed into each other. One persona per task, one session dir per task.
- Iterating on a prompt/product? Re-run the same panel. A fixed persona
panel across rounds tells you whether a change fixed what the panel
actually complained about.
- Probes, smokes, and diagnosis loops are manifests too. A model-calling
smoke test is a one-task manifest with the transcript as
expect_files
and a validator as the check. Diagnosing a failed worker's output is a
read-only scout task. If it calls a model, it runs under Ringer — that is
what makes it visible, verified, and logged.
Engine selection
The engine choice belongs to the human — but the recommendation comes
from THEIR evidence. Before the FIRST run of a job: read what's wired up
([engines.<name>] blocks in ~/.config/ringer/config.toml), run
./ringer.py models --task-type <this job's type> for the local scoreboard,
and glance at ./ringer.py catalog --changes for anything newly free or
newly cheap. Then ask the user which model should do the typing — top 2–3
options with the NUMBERS in the pitch and a recommendation, e.g.: "GLM is
6/6 first-try on persona work here at ~2¢/task — recommended. Codex is also
100% but ~8x the tokens. And kimi went free on OpenRouter yesterday — want
it auditioning one of the small tasks?" Honor their pick via the per-task
engine/model fields; don't re-ask every round of the same job unless
the mix isn't working. This is per-user by design: the scoreboard learns
THIS user's workload — never import another machine's conclusions or
recommend from a different user's numbers.
Explore or the scoreboard fossilizes. Always recommending the proven
pick means never learning a new one. In any run of 3+ tasks that has a
low-stakes lane (docs sweeps, mechanical edits, persona reviews — strong
executed check, retry to absorb failure), assign roughly ONE task to an
exploration candidate from ./ringer.py models --explore --task-type <type>
(untested + cheap or free, text-capable, decent context). Free promos from
catalog --changes jump the queue — a temporarily-free model is a zero-cost
experiment. Never explore on time-critical work, never with more than a
small slice of a batch, and name the experiment when presenting the engine
ask so the human can veto it. Promotion ladder (computed by --explore):
untested → probation (some evidence) → proven for a task_type (3+ tasks,
first-try ≥ 0.67). Proven models earn bigger lanes in that type and an
audition one rung up in adjacent types; repeated first-attempt failures end
the audition — record the demotion in MODEL-NOTES so the next orchestrator
doesn't re-run the experiment.
OpenCode is the harness; the model is a manifest field. Unless a model
ships its own first-class harness (Codex does), it runs through the
opencode engine with the task's "model" field set to the OpenRouter
slug — e.g. "engine": "opencode", "model": "openrouter/moonshotai/kimi-k2.7-code".
This holds even when someone — including the user, in the heat of a run —
says to "call kimi directly" or reach for the model's own CLI: the harness
is what provides the sandbox, raw logs, token counts, and executed
verification, so routing around it silently drops all four. Never clone an
engine block or splice -m through engine_args to change models; that's
what the model field is for, and a bakeoff is only real when the MANIFEST
names each competitor (2026-07-06 lesson: an engine block with a hard-coded
model ran one model under three competitors' names).
Engines are config blocks ([engines.<name>] in config.toml), selectable
per task via the manifest engine field. Defaults are deliberate:
- codex (default): strongest general worker. Use per-task
engine_args
to set reasoning effort — spend it on hard tasks, not boilerplate.
- opencode: the universal lane — any OpenRouter model via the
model
field (engine model_default is GLM-5.2, the cheap-intelligence pick).
Validate a model new to you with a trivial one-task manifest before
trusting it with a batch.
- Small/flash-class models are the first to choke on long conversational or
multi-turn harness tasks — watch their retry counts before scaling them.
- Match
timeout_s to the task: conversational harness tasks and
build-and-test checks need far more than file edits.
- Check the evidence before assigning models to tasks. Run
./ringer.py models (optionally --task-type <type>) — the local
scoreboard aggregating every executed-check outcome per (model,
task_type): first_try_pass_rate is the routing signal; pass_rate includes
retry rescues. Then read docs/MODEL-NOTES.md (in the ringer repo) for
the judgment the numbers can't carry. Routing is grounded in performance,
not vibes (Jon directive 2026-07-06).
- "Show me the scoreboard" is one command. When the human asks to see
the model scoreboard, rankings, model costs, or "which models work best,"
run
./ringer.py models --open — it renders the full scoreboard (tiers,
first-try rates, est. $/task, usage, MODEL-NOTES excerpts, free-promo
watchlist) as a zero-LLM HTML page in the artifact library and opens it
in their browser. Costs no tokens; never hand-summarize the numbers when
the page can show them.
- Give every task a
task_type (canonical vocabulary in the README —
code-feature, code-fix, code-review, research, persona-review, site-build,
image-gen, docs, probe, bakeoff, ...). Untyped tasks bucket as (untyped)
and teach the scoreboard nothing; lint nudges you when it's missing.
Worktrees-mode footguns (learned the hard way)
Run-level "worktrees": true gives each task an isolated git worktree of
repo, detached at HEAD. Three consequences:
- Passing tasks get their worktree DELETED. Deliverables must land
outside the task worktree, or the check must export them first.
- Worker commits die with the worktree. Pattern that works: the worker
leaves changes uncommitted; the check runs
git add -A && git diff --cached > <path-outside-worktree>.patch and
validates the patch. You apply and commit on your branch after review.
- Logs survive (they go to
<workdir>/logs/), so post-mortems work
even on deleted worktrees.
- Gitignored outputs silently vanish from patch exports.
git add -A
cannot stage ignored files (build dirs like dist/), so a worker's edits
there pass its checks, export an incomplete patch, and die with the
worktree. If a task touches any gitignored path, the check must cp
those files to a path outside the worktree explicitly — verify the patch
AND the copies before trusting the run.
And on your own side of the fence: when integrating patches into the real
repo, stage specific paths — never git add -A in a checkout that may hold
someone's untracked scratch files.
Post-run review ritual
- Read the run JSON in
~/.ringer/runs/ — statuses, retries, durations.
- For any retried or failed task, read the raw worker log in
<workdir>/logs/ before deciding anything. Retries that passed on
attempt 2 often reveal a spec ambiguity worth fixing in your next
manifest.
- Spot-check at least one PASSING task's artifact per run. The check
catches most laziness; you catch the rest.
- Failures with useless error messages mean your CHECK needs work, not
(only) the worker.
- Update
docs/MODEL-NOTES.md (in the ringer repo) when a run taught
you something about a model: one dated line under the model — task type,
what happened (attempts, tokens, failure mode), what you'd do
differently. Only what the executed checks and raw logs support. The raw
numbers took care of themselves — every attempt already landed in the
local model log (./ringer.py models to see the updated scoreboard).
Spend your own context deliberately
The scoreboard exists so that worker tokens buy evidence. Your own tokens are
not free either, and nothing in the tool constrains them:
- Reach for code before a model. Counting, sorting, exact-text search,
field extraction, format conversion, file comparison, validation —
rg,
jq, a parser, a two-line script. A model imitating grep is an expensive
way to get a worse grep.
- Select passages; don't load files. Search first, then read what matched.
Loading a whole transcript because the answer is somewhere inside it is how
a cheap question turns expensive.
ask does this for you; when you are not
using ask, do it by hand.
- Load a tool when the job needs it — not every connector and schema at
the top of a session on the chance that one gets used.
- Answer the question that was asked. A sentence when a sentence was asked
for. No process diary, no restating the human's request back to them, no
unrequested options.
- Never retry into a limit. A token- or usage-limit failure is not a
transient error; retrying it just burns the budget faster. Reduce the input
or take a cheaper path.
When you claim a saving, count the whole job — every call, including your own
planning and review. Moving tokens from your context into a worker's is only a
saving if the total came down.
Baked-in invariants (preserve in any change to ringer.py)
Stdin closed (< /dev/null); sandbox mode explicit; verification executes
the artifact; logs carry raw worker output only. These are load-bearing —
engine and invocation changes must keep all four.
1---2name: ringer3description: >-4 Orchestrator playbook and routing rules for Ringer, the verified-swarm5 delegation tool (ringer.py). TRIGGER — load BEFORE acting, not after —6 whenever: you are about to run ANY script or command that calls a model or7 drives a conversational/eval harness (probe, smoke test, simulation,8 grader, persona conversation) outside a live Ringer run; you are about to9 start an edit→test→edit loop or a batch of similar edits across files; you10 are about to do a "quick check" that spawns a model or a CLI agent; you are11 reviewing or diagnosing failed worker or model output; you catch yourself12 thinking a task is "small enough to just do myself" — that thought IS the13 trigger (a single task is a one-task manifest, and a bounded read-only14 question is `ringer.py ask`); or you are writing or15 reviewing a manifest, choosing a swarm pattern (review swarm, fix swarm,16 focus group, bakeoff, research-with-proof), picking a worker engine, or17 debugging a failed run. SKIP only for: reading or searching files, git18 operations, a one-file few-line ONE-SHOT edit (once — if you are back for a19 second pass, that is a loop: TRIGGER), authoring prose/specs/docs straight20 from your own context, or pure conversation.21metadata:22 source: https://skillmd.com/skills/natebjones-projects/ringer23---2425# Ringer orchestrator playbook2627## Read this first — the four rules that actually get broken28291. **You review; workers type.** Your lane: specs, checks, pattern choice,30 reading results. If you are typing implementation, running probes, or31 babysitting a retry loop yourself, you have left your lane.322. **A single task is a one-task manifest.** Same verification, zero33 ceremony. "Too small for Ringer" is how drift starts — the smoke test,34 the probe script, the three-edit fix are all one-task manifests.353. **Beware the tiny-edit death spiral.** The named anti-pattern: each step36 is individually small enough to justify inline, and two hours later the37 exception has become the workflow and nothing was verified or visible.38 The one-shot exception is ONE file, a few lines, ONCE. The second pass on39 the same problem is a loop, and loops are manifests.404. **Runs are watched, not hidden — and the screen comes up FIRST.** The41 moment this skill loads for real work, before you write a single spec,42 put Ringside on the human's screen: `./ringer.py hud` (idempotent — if43 one is already up it says so and opens the page; runs also auto-start44 it). Ringside is the PAGE at http://127.0.0.1:8700 — NEVER launch the45 Ringside.app application (`open -a Ringside`); it is a parked prototype46 with a stale frontend. And never go dark: if your prep (research,47 check-writing, manifest drafting) will take more than ~30 seconds,48 tell the human in one sentence what you're doing and roughly how long49 before you start — they should be watching the empty arena and reading50 your one-liner, not wondering if anything is happening. Never pass51 `--no-dashboard` except in automated tests or when the user explicitly52 asks.5354Ringer runs manifest tasks in parallel across cheap CLI workers (Codex,55OpenCode/GLM, others via config) and verifies every task by **executing a56check command** — exit 0 is the only PASS. Failed tasks are retried once57with the check's actual failure output injected into the retry prompt. You —58the orchestrating model — pay tokens only for specs, orchestration, and59review.6061```bash62./ringer.py lint manifest.json # always lint before running63./ringer.py run manifest.json --identity <who-you-are>64./ringer.py demo # 3-worker smoke test65./ringer.py run manifest.json --dry-run # print the plan, spawn nothing66```6768Runs land in `~/.ringer/runs/`. Raw worker logs land in `<workdir>/logs/`.69Full reference: `README.md`. Ready-made manifest skeletons: `templates/`.70Lint catches unverifiable checks, silent checks, worktree deliverable/commit71loss, serial fan-out, write collisions, and underspecified specs; `run`72prints the same findings as non-blocking warnings.7374## The one exception: `ask`7576Rule 2 holds for anything that changes a file, runs a build, or produces an77artifact worth checking. One lane doesn't fit it: the human asks a bounded,78read-only question over source you can already point at, and the answer is79prose. A manifest for that is ceremony — but answering it in your own context80means pulling whole files into a conversation that is already expensive.8182```bash83./ringer.py ask "<the human's request>" --source /absolute/path/to/source84```8586`ask` selects the passages that match the request, caps the packet, spawns one87clean worker on it, and allows a single attempt. Repeat `--source` for several88files or directories; `--state` takes a small file of settled decisions;89`--dry-run` shows you the packet and spends nothing. If everything that matched90is too large for the packet it says so and stops before the model call rather91than letting a worker guess — but a source small enough to fit whole is sent92whole, relevant or not, so choosing the sources IS the work. Directory scans93stay inside the tree you name; a symlink leading out of it is skipped and94reported. Runs appear on Ringside like any other, and `--redact` hides the95request from Ringer's own state and eval records — it cannot scrub raw worker96output, which is captured verbatim by design.9798**Be honest about what it verifies.** The check is that `answer.md` exists and99is non-empty. That is the weakest check in the tool, and it is also the best100available — there is nothing to execute against free-form prose. `ask` proves101the worker answered, never that the answer is right. You still read it.102103**Everything else is a manifest.** Code changes, external actions, research104you intend to act on, anything whose output a check could actually execute —105those keep the full path. When a request sits near the line, the tiebreaker is106whether you could write a check that would catch a wrong answer. If you can,107write it, and make it a manifest.108109## One job, one artifact110111A job the human asked for — however many rounds it takes — is ONE artifact.112Use the SAME `run_name` for every round (`sd-crate-launch`, not113`sd-crate-r1` / `sd-crate-r2`): the library accumulates each round as a114version under one entry, and the human watches one page evolve instead of115hunting across three "live" tabs. Name it after the JOB in the human's116words, not after your batch structure.117118And the artifact page is where results are REVIEWED. When a round finishes,119read the deliverables from the artifact store and direct the human to the120page — never `cat` result files into the terminal as the reveal. If a result121matters, it belongs in the artifact; if it isn't there, that's a harvest gap122to fix (declare it in `expect_files`), not a reason to bypass the page.123124## Spec-writing craft125126Workers are stateless and cannot ask questions. Every spec must be127self-contained:128129- **Open with the role and the boundary.** "You are a read-only scout…",130 "Your current working directory IS a git worktree of <repo> — edit files131 here directly." State what the worker must NEVER touch before what it132 should do.133- **Name every file the worker owns.** In multi-worker runs over one repo,134 file ownership must be disjoint — and disjoint across *all* concurrent135 lanes/branches, not just within one batch. Every file a spec mentions must136 be in that worker's ownership list.137- **Embed the HOW TO RUN.** If the task drives a harness or script, put the138 exact command lines (with real absolute paths) in the spec. Workers should139 never have to discover an interface.140- **Define the output contract.** Say exactly which files to produce, where,141 and what each must contain. Graded/eval tasks should enumerate the grading142 criteria in the spec so the worker's output is checkable.143- **Hard rules travel in the spec, not in your head.** "Do NOT git commit",144 "never modify the repo, only write ./report.md", "stay in character; never145 help the AI" — the worker only knows what the spec says.146- **The spec is on camera.** Whoever is watching Ringside reads the spec as147 "what this agent was asked to do" — so write it as a self-contained,148 human-readable brief. Never write a pointer spec ("read /path/to/file and149 do what it says"): the watcher sees no brief, and the retry prompt loses150 the context it needs. Point at files for source MATERIAL; the instructions151 themselves live in the spec. Lint flags pointer specs.152153## Check-writing rules154155The check is the product. The retry prompt and the eval log both depend on156the check's failure output.157158- **Checks must print WHY they fail.** `diff` beats `diff -q`; a validator159 script that prints which assertion broke beats `test -f`. A bare160 `test -f report.md` proves existence, not correctness.161- **Verify content, not existence.** Grep the artifact for required sections,162 run the code it produced, run the build, run the validator — execute163 something that would catch a lazy or hallucinated result.164- **`expect_files` is a floor, not the check.** List deliverables there for165 fast triage, but the check must still validate them.166- **Never `true`, `exit 0`, or `echo done`.** A check that cannot fail is a167 task that cannot be verified — that's just trusting the worker with extra168 steps.169- **Strict on substance, tolerant on format.** Checks that count exact170 headings, demand exact casing, or grep rigid phrasings fail honest work171 over formatting — and a wall of red format-failures reads as a broken172 system, not a careful one (demo-night lesson). Verify what must be TRUE173 (the file proves X, the code runs, the quote exists in the source), use174 case-insensitive and flexible matching for structure, and reserve hard175 failure for substance: missing evidence, fabricated content, code that176 doesn't run.177- **A check cannot demand evidence the spec never supplied.** Before failing178 a worker for missing evidence or input, re-read the task inputs. If the spec179 didn't provide a value, the check must not invent one and fail on its180 absence — an honest UNVERIFIABLE answer is not a failure. Reserve hard181 failure for what the spec actually asserted.182- **Executed checks catch laziness, not subtle wrongness.** A check that183 *runs* the artifact catches a plausible-but-wrong change far less often than184 it catches a missing one. Whenever a swarm touches a dogfood artifact185 (Ringer's own docs, config, or checks), add an "our own artifact passes our186 own validator" test so the checker exercises what it preaches. And keep187 orchestrator patch review mandatory regardless of PASS status — a green188 check is not proof of semantic correctness.189190## Pattern playbook191192Reach for a named pattern before inventing one. Skeletons in `templates/`:193194| Kit | Use when |195|---|---|196| [review-swarm](../../../templates/review-swarm/) | You need broad read-only review coverage before deciding what to fix. |197| [fix-swarm](../../../templates/fix-swarm/) | You have confirmed independent fixes that can be split across isolated worktrees. |198| [focus-group](../../../templates/focus-group/) | You need isolated persona feedback on a product, pitch, prompt, or workflow. |199| [bakeoff](../../../templates/bakeoff/) | You need evidence for choosing a model, prompt, or configuration across shared scenarios. |200| [research-with-proof](../../../templates/research-with-proof/) | You need research backed by a proof task whose check executes the claim. |201| [launch-kit](../../../templates/launch-kit/) | You need a go-to-market package built across research, persona review, and final assembly rounds. |202| [asset-swarm](../../../templates/asset-swarm/) | You need media assets produced in parallel with executable checks for renders, batches, diagrams, or captures. |203| [adversarial-review](../../../templates/adversarial-review/) | You want several models to review the same artifact before the orchestrator synthesizes findings. |204| [repo-feature](../../../templates/repo-feature/) | You know what to build and need sandboxed workers to edit a real repo with build and git checks. |205| [migration-swarm](../../../templates/migration-swarm/) | You have mechanical codebase transforms that can be partitioned across worktrees. |206| [doc-swarm](../../../templates/doc-swarm/) | You need module docs with executed examples and checks against invented APIs. |207| [test-hardening](../../../templates/test-hardening/) | You need stronger tests by module while keeping production source edits off-limits. |208| [competitive-teardown](../../../templates/competitive-teardown/) | You need competitor research with citation allowlists and a synthesis phase. |209| [data-pipeline](../../../templates/data-pipeline/) | You need fetch, transform, and validate stages with executed validators and honesty rules. |210| [probe](../../../templates/probe/) | You need a one-task manifest for a smoke, probe, or post-mortem. |211212Pattern-selection judgment:213214- **Browse the catalog first.** Before writing any manifest, browse215 `templates/README.md`: choose a kit, mix pieces from several, or write216 your own having seen the prior art.217- **Review before fix.** Run a read-only review swarm, read the reports218 yourself, then compile the confirmed findings into a fix-swarm manifest.219 Don't let the same worker find and fix.220- **Personas must be separate workers.** Parallel personas in one context221 bleed into each other. One persona per task, one session dir per task.222- **Iterating on a prompt/product? Re-run the same panel.** A fixed persona223 panel across rounds tells you whether a change fixed what the panel224 actually complained about.225- **Probes, smokes, and diagnosis loops are manifests too.** A model-calling226 smoke test is a one-task manifest with the transcript as `expect_files`227 and a validator as the check. Diagnosing a failed worker's output is a228 read-only scout task. If it calls a model, it runs under Ringer — that is229 what makes it visible, verified, and logged.230231## Engine selection232233**The engine choice belongs to the human — but the recommendation comes234from THEIR evidence.** Before the FIRST run of a job: read what's wired up235(`[engines.<name>]` blocks in `~/.config/ringer/config.toml`), run236`./ringer.py models --task-type <this job's type>` for the local scoreboard,237and glance at `./ringer.py catalog --changes` for anything newly free or238newly cheap. Then ask the user which model should do the typing — top 2–3239options with the NUMBERS in the pitch and a recommendation, e.g.: *"GLM is2406/6 first-try on persona work here at ~2¢/task — recommended. Codex is also241100% but ~8x the tokens. And kimi went free on OpenRouter yesterday — want242it auditioning one of the small tasks?"* Honor their pick via the per-task243`engine`/`model` fields; don't re-ask every round of the same job unless244the mix isn't working. This is per-user by design: the scoreboard learns245THIS user's workload — never import another machine's conclusions or246recommend from a different user's numbers.247248**Explore or the scoreboard fossilizes.** Always recommending the proven249pick means never learning a new one. In any run of 3+ tasks that has a250low-stakes lane (docs sweeps, mechanical edits, persona reviews — strong251executed check, retry to absorb failure), assign roughly ONE task to an252exploration candidate from `./ringer.py models --explore --task-type <type>`253(untested + cheap or free, text-capable, decent context). Free promos from254`catalog --changes` jump the queue — a temporarily-free model is a zero-cost255experiment. Never explore on time-critical work, never with more than a256small slice of a batch, and name the experiment when presenting the engine257ask so the human can veto it. Promotion ladder (computed by --explore):258untested → probation (some evidence) → proven for a task_type (3+ tasks,259first-try ≥ 0.67). Proven models earn bigger lanes in that type and an260audition one rung up in adjacent types; repeated first-attempt failures end261the audition — record the demotion in MODEL-NOTES so the next orchestrator262doesn't re-run the experiment.263264**OpenCode is the harness; the model is a manifest field.** Unless a model265ships its own first-class harness (Codex does), it runs through the266`opencode` engine with the task's `"model"` field set to the OpenRouter267slug — e.g. `"engine": "opencode", "model": "openrouter/moonshotai/kimi-k2.7-code"`.268This holds even when someone — including the user, in the heat of a run —269says to "call kimi directly" or reach for the model's own CLI: the harness270is what provides the sandbox, raw logs, token counts, and executed271verification, so routing around it silently drops all four. Never clone an272engine block or splice `-m` through `engine_args` to change models; that's273what the `model` field is for, and a bakeoff is only real when the MANIFEST274names each competitor (2026-07-06 lesson: an engine block with a hard-coded275model ran one model under three competitors' names).276277Engines are config blocks (`[engines.<name>]` in config.toml), selectable278per task via the manifest `engine` field. Defaults are deliberate:279280- **codex** (default): strongest general worker. Use per-task `engine_args`281 to set reasoning effort — spend it on hard tasks, not boilerplate.282- **opencode**: the universal lane — any OpenRouter model via the `model`283 field (engine `model_default` is GLM-5.2, the cheap-intelligence pick).284 Validate a model new to you with a trivial one-task manifest before285 trusting it with a batch.286- Small/flash-class models are the first to choke on long conversational or287 multi-turn harness tasks — watch their retry counts before scaling them.288- Match `timeout_s` to the task: conversational harness tasks and289 build-and-test checks need far more than file edits.290- **Check the evidence before assigning models to tasks.** Run291 `./ringer.py models` (optionally `--task-type <type>`) — the local292 scoreboard aggregating every executed-check outcome per (model,293 task_type): first_try_pass_rate is the routing signal; pass_rate includes294 retry rescues. Then read `docs/MODEL-NOTES.md` (in the ringer repo) for295 the judgment the numbers can't carry. Routing is grounded in performance,296 not vibes (Jon directive 2026-07-06).297- **"Show me the scoreboard" is one command.** When the human asks to see298 the model scoreboard, rankings, model costs, or "which models work best,"299 run `./ringer.py models --open` — it renders the full scoreboard (tiers,300 first-try rates, est. $/task, usage, MODEL-NOTES excerpts, free-promo301 watchlist) as a zero-LLM HTML page in the artifact library and opens it302 in their browser. Costs no tokens; never hand-summarize the numbers when303 the page can show them.304- **Give every task a `task_type`** (canonical vocabulary in the README —305 code-feature, code-fix, code-review, research, persona-review, site-build,306 image-gen, docs, probe, bakeoff, ...). Untyped tasks bucket as (untyped)307 and teach the scoreboard nothing; lint nudges you when it's missing.308309## Worktrees-mode footguns (learned the hard way)310311Run-level `"worktrees": true` gives each task an isolated git worktree of312`repo`, detached at HEAD. Three consequences:3133141. **Passing tasks get their worktree DELETED.** Deliverables must land315 outside the task worktree, or the check must export them first.3162. **Worker commits die with the worktree.** Pattern that works: the worker317 leaves changes uncommitted; the check runs318 `git add -A && git diff --cached > <path-outside-worktree>.patch` and319 validates the patch. You apply and commit on your branch after review.3203. **Logs survive** (they go to `<workdir>/logs/`), so post-mortems work321 even on deleted worktrees.3224. **Gitignored outputs silently vanish from patch exports.** `git add -A`323 cannot stage ignored files (build dirs like `dist/`), so a worker's edits324 there pass its checks, export an incomplete patch, and die with the325 worktree. If a task touches any gitignored path, the check must `cp`326 those files to a path outside the worktree explicitly — verify the patch327 AND the copies before trusting the run.328329And on your own side of the fence: when integrating patches into the real330repo, stage specific paths — never `git add -A` in a checkout that may hold331someone's untracked scratch files.332333## Post-run review ritual3343351. Read the run JSON in `~/.ringer/runs/` — statuses, retries, durations.3362. For any retried or failed task, read the raw worker log in337 `<workdir>/logs/` before deciding anything. Retries that passed on338 attempt 2 often reveal a spec ambiguity worth fixing in your next339 manifest.3403. Spot-check at least one PASSING task's artifact per run. The check341 catches most laziness; you catch the rest.3424. Failures with useless error messages mean your CHECK needs work, not343 (only) the worker.3445. **Update `docs/MODEL-NOTES.md`** (in the ringer repo) when a run taught345 you something about a model: one dated line under the model — task type,346 what happened (attempts, tokens, failure mode), what you'd do347 differently. Only what the executed checks and raw logs support. The raw348 numbers took care of themselves — every attempt already landed in the349 local model log (`./ringer.py models` to see the updated scoreboard).350351## Spend your own context deliberately352353The scoreboard exists so that worker tokens buy evidence. Your own tokens are354not free either, and nothing in the tool constrains them:355356- **Reach for code before a model.** Counting, sorting, exact-text search,357 field extraction, format conversion, file comparison, validation — `rg`,358 `jq`, a parser, a two-line script. A model imitating `grep` is an expensive359 way to get a worse `grep`.360- **Select passages; don't load files.** Search first, then read what matched.361 Loading a whole transcript because the answer is somewhere inside it is how362 a cheap question turns expensive. `ask` does this for you; when you are not363 using `ask`, do it by hand.364- **Load a tool when the job needs it** — not every connector and schema at365 the top of a session on the chance that one gets used.366- **Answer the question that was asked.** A sentence when a sentence was asked367 for. No process diary, no restating the human's request back to them, no368 unrequested options.369- **Never retry into a limit.** A token- or usage-limit failure is not a370 transient error; retrying it just burns the budget faster. Reduce the input371 or take a cheaper path.372373When you claim a saving, count the whole job — every call, including your own374planning and review. Moving tokens from your context into a worker's is only a375saving if the total came down.376377## Baked-in invariants (preserve in any change to ringer.py)378379Stdin closed (`< /dev/null`); sandbox mode explicit; verification executes380the artifact; logs carry raw worker output only. These are load-bearing —381engine and invocation changes must keep all four.