Tribunal
The orchestrator builds nothing and scores nothing itself — it slices the work, dispatches
a separate doer, convenes separate verifiers, and adjudicates on evidence. Three
behaviors carry the value: an adversarial second look that catches
what a single pass ships silently, as named failure scenarios; calibrated verdicts
— locally fixable defects get ITERATE, not "rewrite everything", and passing math is
overridden only by verified evidence; evidence before claims — nothing is "done"
without fresh reproduction; refuted claims are excluded, never averaged in. Announce
at start: "Running this through the tribunal pattern: doer -> verifier panel -> consensus."
When to use
Multi-part or high-stakes deliverables where a shipped defect costs more than a panel.
Not for trivial edits, anything one verification command proves, or ordinary code
review (dialogue improving a change vs independent measurements of a frozen artifact
against pre-declared criteria, adjudicated to a ship decision). Never nest tribunals
— no role runs the protocol on its own output.
The loop
- Freeze acceptance criteria and verification commands BEFORE implementation; derive
scoring dimensions, weights, and a pass target from them, recorded so re-panels
reuse the rubric. Identify the operative skills in play — those bearing on how the
artifact is built or judged (e.g. production-grade); never the tribunal skill itself
nor pure orchestration skills — and fold their standards into the criteria. Verify-only
entry (artifact already exists): write criteria from the original request — never
reverse-engineered from it — freeze, start at step 3.
- Spawn a SEPARATE doer agent (never the orchestrator itself) with the full slice spec
pasted in (never "read the plan file") and the operative skills named with an
instruction to load them (load production-grade, etc.) — if it cannot load a named
skill it says so rather than proceeding. If the orchestrator writes or edits the
deliverable, there is no independent artifact to verify and the run collapses to one
context. The doer implements, runs the verification commands, materializes the artifact
durably — committed, pushed, or published so it outlives the doer's session — and
reports a diff summary, verbatim output, the artifact's fetchable address, and exactly
one status (table below). DONE without an address is not DONE.
- Check the report against the actual diff yourself; dispatch the panel in parallel, context-walled.
- Adjudicate per consensus-mechanics.md; record
verdict, dissents, caveats, and round count in the ledger.
- SHIP -> next slice. ITERATE -> findings become a fix list for a fresh doer;
re-panel with fresh verifiers re-scoring every dimension, prior findings as risks.
Invariants (low freedom — these are the skill)
- Context wall. Four roles, four separate sessions: the orchestrator (slices,
dispatches, adjudicates, owns the ledger — produces NO deliverable content and assigns
NO scores itself), the doer, and each verifier. One agent never plays two roles (shared
context = shared blind spots = no triangulation, the entire point). The doer finishes
first; then the
verifiers run in parallel, each RECEIVING exactly: frozen criteria; the artifact by
fetchable address — a commit SHA, branch, or URI a verifier in a different
sandbox can retrieve on its own, plus the diff (or the document and its predecessor)
inline; bare working-tree paths are insufficient, since they resolve only inside the
doer's session and die with it; reference materials;
permission to run the verification commands; known risks; the operative skills
(named, to load). Verifiers NEVER receive:
the doer's reasoning or self-assessment, design rationale, each other's first-round
views or identities, expected or prior scores. Naming shared standards is not a wall
breach — the wall withholds the doer's reasoning and scores, not the bar everyone is
measured against.
- Evidence anchoring. High scores require a verbatim quote from the artifact;
failure claims require file:line or command output; evidence-free findings are
discarded. Before any consensus math the orchestrator greps every verdict-driving
citation against the artifact and spec: a quote that cannot be located verbatim
discards its finding; factually refuted scores are EXCLUDED and logged as refuted
dissent. Adjudicate on evidence — never average disagreement away.
- Explicit opposition. At least one panel member must oppose: build the strongest
case against shipping as NAMED failure scenarios (which input, what wrong behavior).
Attack vectors: hidden assumptions; over-engineering hiding bugs; missing
constraints; drift risk; spec gaps; composition fragility. Generic skepticism is
worthless — discard it. Confirm every accusation against the spec text first; never
flag behavior the spec mandates. An unrebutted correctness or safety scenario from
any member is never silently overruled: surface it (ESCALATE).
- Criteria precede the artifact — frozen before implementation, or from the original request in verify-only mode.
- No completion claims without fresh verification output. "Should pass" is not
evidence; agent-reported success is checked against the actual artifact and diff.
- Bounded iteration. Max 3 panel rounds per slice (fresh verifiers each round)
plus a separate doer-dispatch budget (default 5, complexity-weighted); whichever
exhausts first -> ESCALATE to the human with the full evidence package. Both counters
travel in the handoff or a durable ledger, never in orchestrator context alone — every
dispatch states the round index and the remaining budget. A cap an orchestrator restart
or context reset forgets is not a cap: the run re-panels indefinitely.
| Doer status |
Orchestrator handling |
| DONE |
Verify the diff exists, the commands ran, and the reported address resolves; a DONE with no fetchable address is NEEDS_CONTEXT — send it back to materialize. Then panel. |
| DONE_WITH_CONCERNS |
Correctness/scope concerns: address before panel. Observations: log, proceed. |
| NEEDS_CONTEXT |
Supply the missing context; re-dispatch the same model. |
| BLOCKED |
Triage in order: more context, a more capable model, decompose the slice, escalate. Never retry the same model unchanged — change at least one of context, model, task size. |
| Verdict |
Trigger |
| SHIP |
Overall at or above target; no dimension blocked. |
| SHIP_WITH_CAVEATS |
In the caveat band [target − 0.10, target), only non-blocking caveats. During a build this is an ITERATE trigger: each caveat fixed and re-verified, or deferred with a logged reason — final only if the human accepts. |
| ITERATE |
Below the caveat band; a surviving blocking caveat (one asserting a correctness/safety defect, or lacking a concrete fix or logged deferral); or a blocked dimension whose causes are all localized, enumerable fixes (regardless of overall) — with a mandatory fix list. Math decides unless a verified (unrefuted) failure scenario overrides. |
| BLOCK |
The defect set is structural: remediation needs redesign, not an enumerable list of localized fixes — or proceeding is unsafe. Low scores alone never force BLOCK: a majority-blocked dimension whose causes are all localized fixes is ITERATE with a mandatory fix list. |
| ESCALATE |
Deadlock, an unrebutted opposition scenario, or budgets exhausted. Orchestrator-only — never a panel recommendation. |
Calibration: BLOCK is structural. Many severe defects that are each a localized fix in
a sound architecture make ITERATE, not BLOCK — severity sets fix priority, never the
verdict class. A blocked dimension (majority at 3 or below) never lets a passing overall
slip through as SHIP: with localized causes it is ITERATE plus a mandatory fix list, even
when overall ≥ target; only structural causes reach BLOCK. Consistency check before
issuing BLOCK: if the report itself describes every defect as locally fixable, BLOCK
contradicts the evidence — issue ITERATE. One member's low score is a logged dissent, not
a block.
Panel composition (derive per artifact)
Independent vantage points triangulate ground truth; the rest is sizing. Default three
lenses: quality (spec compliance first — missing / extra / misunderstood — then
correctness), fitness-for-purpose (real consumer, edge cases, integration), and the
adversary (must oppose). Scale to stakes — two reviewers for a small low-risk slice,
five lenses for a high-stakes irreversible one — picking lenses from the artifact's
actual risk surface. Example: a data-migration script risks data loss and partial
failure -> transform-correctness, a rollback/idempotency adversary, operator
experience; an API doc -> consistency, consumer ergonomics, a breaking-change adversary.
Never give the adversary a weaker model than the rest of the panel; when available, a
different model family on the panel buys uncorrelated blind spots.
Verifier prompt skeleton (write each prompt from this)
Role: <lens> verifier. You have NOT seen the builder's reasoning or any other
reviewer's views. The report you judge may be wrong — verify everything against
the artifact itself, as-is.
[Adversary only: You MUST oppose — build the strongest case against shipping;
name the exact failure scenario per concern. Write "ESCALATE: <reason>" for a
correctness/safety concern you believe cannot be rebutted.]
Inputs: frozen criteria; dimensions/weights/target; artifact (fetchable address — commit/branch/URI — plus the diff inline); round index and remaining budget; references; verification commands (you may run them); known risks; operative skills (named — load them; if you cannot, say so).
Rules: score each dimension separately, 1-10, with confidence 0.0-1.0; every
claim cites a verbatim quote, file:line, or command output. Citations WILL be
grepped against the artifact and spec — one that does not exist verbatim
discards the finding.
Output: per-dimension score + confidence + evidence; top concern;
recommendation SHIP|SHIP_WITH_CAVEATS|ITERATE|BLOCK; escalation if any.
Records
The deliverable is the ONLY durable artifact a tribunal run produces. Report the
verdict, dissents (incl. refuted, verbatim), caveats (fixed | deferred + reason), and
panel-round counts in your closing summary — and fold them into the PR description or
commit message when one exists. Delivering that report to the requester, over the
channel the request arrived on, is a step of the loop, not a side effect of finishing —
a verdict nobody received is not a verdict. Never leave loose files in the tree. If a run
genuinely needs working files (a running ledger across many slices, temp fixtures), put them ALL
inside a single gitignored .tribunal/ directory and never commit it; nothing else is
written. Never silently drop a caveat.
Plan vetting (same machinery, earlier)
Before executing a non-trivial plan, run it past three independent lenses: will it work
(are the checks specific?), can the doer follow it, is it the cheapest sufficient path.
2-of-3 approval proceeds; otherwise merge the critiques into exactly ONE refinement
pass, re-vet once, then escalate. Vetting freezes the criteria the panel later audits.
Platforms
With parallel subagents, dispatch the panel as parallel calls in one message — separate
agents, real independence. Without parallel dispatch, run each role as its own sequential
fresh-context session (still distinct agents), enforcing the wall by what each session
receives. One agent playing every role forfeits the triangulation benefit — measured
equal to no panel at all (a weak model self-simulating a panel scores at its solo floor);
use only when no separate-session option exists, and label the verdict "single-context
(no independence)".
Detached orchestration (the run outlives, or is suspended by, any single turn): where the
runtime's wait primitive ends the orchestrator's turn, whatever is passed to that
primitive is bookkeeping, not communication — nobody has been told anything yet. Carry
the artifact address, round index and remaining budget in the handoff, since the resumed
orchestrator may not be the one that wrote them. On resume, re-establish what the
requester has actually received before deciding the next move: a run ending is not
delivery.
End-to-end example: worked-example.md.
Failure catalogue: anti-patterns.md.
1---
2name: tribunal
3version: 0.0.3
4license: MIT
5description: >-
6 Runs a doer -> verifier-panel -> consensus loop to verify a deliverable before it ships.
7 An orchestrator freezes acceptance criteria before implementation, dispatches a doer, then
8 convenes a context-walled panel of independent verifiers - including an adversary with an
9 explicit must-oppose mandate - for evidence-anchored review adjudicated to a SHIP /
10 SHIP_WITH_CAVEATS / ITERATE / BLOCK / ESCALATE verdict logged to a ledger. Use for
11 multi-agent verification of any artifact - code slices, plans, documents, audits -
12 whenever asked to verify a deliverable, vet a plan, run a consensus review or independent
13 review, set up a doer-verifier loop, or gate a ship decision. Works on any platform with
14 parallel subagents; degrades to sequential fresh-context sessions without them; on
15 detached, sandboxed or asynchronous runtimes the artifact is handed over by fetchable
16 address and the budgets travel in the handoff. Not for trivial single-file edits or
17 ordinary code review.
18metadata:
19 source: https://skillmd.com/skills/a-tokyo/tribunal
20---
21
22# Tribunal
23
24The orchestrator builds nothing and scores nothing itself — it slices the work, dispatches
25a *separate* doer, convenes *separate* verifiers, and adjudicates on evidence. Three
26behaviors carry the value: an **adversarial second look** that catches
27what a single pass ships silently, as named failure scenarios; **calibrated verdicts**
28— locally fixable defects get ITERATE, not "rewrite everything", and passing math is
29overridden only by verified evidence; **evidence before claims** — nothing is "done"
30without fresh reproduction; refuted claims are excluded, never averaged in. Announce
31at start: "Running this through the tribunal pattern: doer -> verifier panel -> consensus."
32
33## When to use
34
35Multi-part or high-stakes deliverables where a shipped defect costs more than a panel.
36Not for trivial edits, anything one verification command proves, or ordinary code
37review (dialogue improving a change vs independent measurements of a frozen artifact
38against pre-declared criteria, adjudicated to a ship decision). Never nest tribunals
39— no role runs the protocol on its own output.
40
41## The loop
42
431. Freeze acceptance criteria and verification commands BEFORE implementation; derive
44 scoring dimensions, weights, and a pass target from them, recorded so re-panels
45 reuse the rubric. Identify the operative skills in play — those bearing on how the
46 artifact is built or judged (e.g. production-grade); never the tribunal skill itself
47 nor pure orchestration skills — and fold their standards into the criteria. Verify-only
48 entry (artifact already exists): write criteria from the original request — never
49 reverse-engineered from it — freeze, start at step 3.
502. Spawn a SEPARATE doer agent (never the orchestrator itself) with the full slice spec
51 pasted in (never "read the plan file") and the operative skills named with an
52 instruction to load them (load production-grade, etc.) — if it cannot load a named
53 skill it says so rather than proceeding. If the orchestrator writes or edits the
54 deliverable, there is no independent artifact to verify and the run collapses to one
55 context. The doer implements, runs the verification commands, materializes the artifact
56 durably — committed, pushed, or published so it outlives the doer's session — and
57 reports a diff summary, verbatim output, the artifact's fetchable address, and exactly
58 one status (table below). DONE without an address is not DONE.
593. Check the report against the actual diff yourself; dispatch the panel in parallel, context-walled.
604. Adjudicate per [consensus-mechanics.md](references/consensus-mechanics.md); record
61 verdict, dissents, caveats, and round count in the ledger.
625. SHIP -> next slice. ITERATE -> findings become a fix list for a fresh doer;
63 re-panel with fresh verifiers re-scoring every dimension, prior findings as risks.
64
65## Invariants (low freedom — these are the skill)
66
671. **Context wall.** Four roles, four separate sessions: the orchestrator (slices,
68 dispatches, adjudicates, owns the ledger — produces NO deliverable content and assigns
69 NO scores itself), the doer, and each verifier. One agent never plays two roles (shared
70 context = shared blind spots = no triangulation, the entire point). The doer finishes
71 first; then the
72 verifiers run in parallel, each RECEIVING exactly: frozen criteria; the artifact by
73 **fetchable address** — a commit SHA, branch, or URI a verifier in a *different*
74 sandbox can retrieve on its own, plus the diff (or the document and its predecessor)
75 inline; bare working-tree paths are insufficient, since they resolve only inside the
76 doer's session and die with it; reference materials;
77 permission to run the verification commands; known risks; the operative skills
78 (named, to load). Verifiers NEVER receive:
79 the doer's reasoning or self-assessment, design rationale, each other's first-round
80 views or identities, expected or prior scores. Naming shared standards is not a wall
81 breach — the wall withholds the doer's reasoning and scores, not the bar everyone is
82 measured against.
832. **Evidence anchoring.** High scores require a verbatim quote from the artifact;
84 failure claims require file:line or command output; evidence-free findings are
85 discarded. Before any consensus math the orchestrator greps every verdict-driving
86 citation against the artifact and spec: a quote that cannot be located verbatim
87 discards its finding; factually refuted scores are EXCLUDED and logged as refuted
88 dissent. Adjudicate on evidence — never average disagreement away.
893. **Explicit opposition.** At least one panel member must oppose: build the strongest
90 case against shipping as NAMED failure scenarios (which input, what wrong behavior).
91 Attack vectors: hidden assumptions; over-engineering hiding bugs; missing
92 constraints; drift risk; spec gaps; composition fragility. Generic skepticism is
93 worthless — discard it. Confirm every accusation against the spec text first; never
94 flag behavior the spec mandates. An unrebutted correctness or safety scenario from
95 any member is never silently overruled: surface it (ESCALATE).
964. **Criteria precede the artifact** — frozen before implementation, or from the original request in verify-only mode.
975. **No completion claims without fresh verification output.** "Should pass" is not
98 evidence; agent-reported success is checked against the actual artifact and diff.
996. **Bounded iteration.** Max 3 panel rounds per slice (fresh verifiers each round)
100 plus a separate doer-dispatch budget (default 5, complexity-weighted); whichever
101 exhausts first -> ESCALATE to the human with the full evidence package. Both counters
102 travel in the handoff or a durable ledger, never in orchestrator context alone — every
103 dispatch states the round index and the remaining budget. A cap an orchestrator restart
104 or context reset forgets is not a cap: the run re-panels indefinitely.
105
106| Doer status | Orchestrator handling |
107|---|---|
108| DONE | Verify the diff exists, the commands ran, and the reported address resolves; a DONE with no fetchable address is NEEDS_CONTEXT — send it back to materialize. Then panel. |
109| DONE_WITH_CONCERNS | Correctness/scope concerns: address before panel. Observations: log, proceed. |
110| NEEDS_CONTEXT | Supply the missing context; re-dispatch the same model. |
111| BLOCKED | Triage in order: more context, a more capable model, decompose the slice, escalate. Never retry the same model unchanged — change at least one of context, model, task size. |
112
113| Verdict | Trigger |
114|---|---|
115| SHIP | Overall at or above target; no dimension blocked. |
116| SHIP_WITH_CAVEATS | In the caveat band [target − 0.10, target), only non-blocking caveats. During a build this is an ITERATE trigger: each caveat fixed and re-verified, or deferred with a logged reason — final only if the human accepts. |
117| ITERATE | Below the caveat band; a surviving blocking caveat (one asserting a correctness/safety defect, or lacking a concrete fix or logged deferral); or a blocked dimension whose causes are all localized, enumerable fixes (regardless of overall) — with a mandatory fix list. Math decides unless a verified (unrefuted) failure scenario overrides. |
118| BLOCK | The defect set is structural: remediation needs redesign, not an enumerable list of localized fixes — or proceeding is unsafe. Low scores alone never force BLOCK: a majority-blocked dimension whose causes are all localized fixes is ITERATE with a mandatory fix list. |
119| ESCALATE | Deadlock, an unrebutted opposition scenario, or budgets exhausted. Orchestrator-only — never a panel recommendation. |
120
121Calibration: BLOCK is structural. Many severe defects that are each a localized fix in
122a sound architecture make ITERATE, not BLOCK — severity sets fix priority, never the
123verdict class. A blocked dimension (majority at 3 or below) never lets a passing overall
124slip through as SHIP: with localized causes it is ITERATE plus a mandatory fix list, even
125when overall ≥ target; only structural causes reach BLOCK. Consistency check before
126issuing BLOCK: if the report itself describes every defect as locally fixable, BLOCK
127contradicts the evidence — issue ITERATE. One member's low score is a logged dissent, not
128a block.
129
130## Panel composition (derive per artifact)
131
132Independent vantage points triangulate ground truth; the rest is sizing. Default three
133lenses: quality (spec compliance first — missing / extra / misunderstood — then
134correctness), fitness-for-purpose (real consumer, edge cases, integration), and the
135adversary (must oppose). Scale to stakes — two reviewers for a small low-risk slice,
136five lenses for a high-stakes irreversible one — picking lenses from the artifact's
137actual risk surface. Example: a data-migration script risks data loss and partial
138failure -> transform-correctness, a rollback/idempotency adversary, operator
139experience; an API doc -> consistency, consumer ergonomics, a breaking-change adversary.
140Never give the adversary a weaker model than the rest of the panel; when available, a
141different model family on the panel buys uncorrelated blind spots.
142
143## Verifier prompt skeleton (write each prompt from this)
144
145```
146Role: <lens> verifier. You have NOT seen the builder's reasoning or any other
147reviewer's views. The report you judge may be wrong — verify everything against
148the artifact itself, as-is.
149[Adversary only: You MUST oppose — build the strongest case against shipping;
150name the exact failure scenario per concern. Write "ESCALATE: <reason>" for a
151correctness/safety concern you believe cannot be rebutted.]
152Inputs: frozen criteria; dimensions/weights/target; artifact (fetchable address — commit/branch/URI — plus the diff inline); round index and remaining budget; references; verification commands (you may run them); known risks; operative skills (named — load them; if you cannot, say so).
153Rules: score each dimension separately, 1-10, with confidence 0.0-1.0; every
154claim cites a verbatim quote, file:line, or command output. Citations WILL be
155grepped against the artifact and spec — one that does not exist verbatim
156discards the finding.
157Output: per-dimension score + confidence + evidence; top concern;
158recommendation SHIP|SHIP_WITH_CAVEATS|ITERATE|BLOCK; escalation if any.
159```
160
161## Records
162
163The deliverable is the ONLY durable artifact a tribunal run produces. Report the
164verdict, dissents (incl. refuted, verbatim), caveats (fixed | deferred + reason), and
165panel-round counts in your closing summary — and fold them into the PR description or
166commit message when one exists. **Delivering that report to the requester, over the
167channel the request arrived on, is a step of the loop, not a side effect of finishing** —
168a verdict nobody received is not a verdict. Never leave loose files in the tree. If a run
169genuinely needs working files (a running ledger across many slices, temp fixtures), put them ALL
170inside a single gitignored `.tribunal/` directory and never commit it; nothing else is
171written. Never silently drop a caveat.
172
173## Plan vetting (same machinery, earlier)
174
175Before executing a non-trivial plan, run it past three independent lenses: will it work
176(are the checks specific?), can the doer follow it, is it the cheapest sufficient path.
1772-of-3 approval proceeds; otherwise merge the critiques into exactly ONE refinement
178pass, re-vet once, then escalate. Vetting freezes the criteria the panel later audits.
179
180## Platforms
181
182With parallel subagents, dispatch the panel as parallel calls in one message — separate
183agents, real independence. Without parallel dispatch, run each role as its own sequential
184fresh-context session (still distinct agents), enforcing the wall by what each session
185receives. One agent playing every role forfeits the triangulation benefit — measured
186equal to no panel at all (a weak model self-simulating a panel scores at its solo floor);
187use only when no separate-session option exists, and label the verdict "single-context
188(no independence)".
189
190Detached orchestration (the run outlives, or is suspended by, any single turn): where the
191runtime's wait primitive ends the orchestrator's turn, whatever is passed to that
192primitive is bookkeeping, not communication — nobody has been told anything yet. Carry
193the artifact address, round index and remaining budget in the handoff, since the resumed
194orchestrator may not be the one that wrote them. On resume, re-establish what the
195requester has actually received before deciding the next move: a run ending is not
196delivery.
197
198End-to-end example: [worked-example.md](references/worked-example.md).
199Failure catalogue: [anti-patterns.md](references/anti-patterns.md).