Cross CLI Deliberation

Cross-CLI Deliberation

A protocol for getting genuine deliberation — not theatre — from multi-CLI consultation. Solves the "AI assistants produce something useful when asked, even when 'this is fine, change nothing' is the right answer" failure mode.

Designed and ratified through self-application: see references/origin-trace.md for the deliberation that produced this skill.

When to use

• forge design phase: ratifying a synthesis with one or more external CLIs before locking
• alf review cycles: getting independent verdicts on staleness / drift findings before recommending changes
• bob arbiter gates (verification-arbiter, design-drift-arbiter, visual-arbiter): when a build fails verification and you need an independent verdict on whether the failure is real or noise
• Any moment where the orchestrator wants a second opinion AND has skin in the answer (you wrote the design, you don't trust your own evaluation of it)
• Tiebreak rounds when consultants disagree

When NOT to use

• Pure information lookup ("how does X work in library Y?") — no deliberation needed, just route to the relevant CLI
• TRIVIAL or SIMPLE tasks (single-file edit, clear output) — overkill
• One-off creative tasks where divergence is the goal, not convergence (use adversarial-team-brainstorm instead)
• When you do NOT have skin in the answer — sycophancy is only a risk when the orchestrator's framing biases the consultant

The two gates (overview)

[Artifact + decision context]
       │
       ▼
┌─────────────────────────────────────────┐
│  Gate 1: Null-hypothesis ballot          │
│  Each consulted CLI files a private      │
│  structured ballot BEFORE seeing peers   │
│  or orchestrator opinion.                │
│                                          │
│  All ACCEPT_AS_IS → ship as-is.          │
│  Any CHANGE_NEEDED → enter Gate 2.       │
└─────────────────────────────────────────┘
       │
       ▼ (only if any CHANGE_NEEDED above threshold)
┌─────────────────────────────────────────┐
│  Gate 2: Burden of falsification         │
│  Each dissenting consultant must ship    │
│  admissible evidence. Orchestrator       │
│  verifies/executes. No evidence →        │
│  Null Verdict (= validation).            │
└─────────────────────────────────────────┘
       │
       ▼
[Synthesis: ship-as-is OR adopt verified patch]

Gate 1 — Null-hypothesis ballot

The principle: "no change needed" is the default incumbent that any proposed change must defeat. Pushback must be structural, not theatrical.

Send to each consulted CLI:

• The artifact under review
• Decision context, constraints, success criteria
• NEVER the orchestrator's opinion or preferred outcome

Require this ballot back:

Verdict:    ACCEPT_AS_IS / REJECT_PREMISE / CHANGE_NEEDED
Confidence: 0–100
Burden met? — what evidence would justify changing this?
              has that evidence been provided in the artifact itself?
Expected loss:
  - if we adopt and shouldn't have:
  - if we don't adopt and should have:
Minimum acceptable action: no-op / clarifying question / test only / patch / redesign

Plus a self-check section declaring on at least these failure modes:

• Ego: defending a prior position as inviolate
• Capture: caving to the synthesis because it elevates the consultant's prior contribution
• Capture by capability: caving to a higher-tier peer model
• Sycophancy of dissent: producing CHANGE_NEEDED to satisfy "produce something useful," even with no admissible evidence

Full ballot template + spawn-prompt examples: references/ballot-template.md.

Threshold for entering proposal mode: ≥1 model votes CHANGE_NEEDED at confidence ≥ 60. Tunable per caller.

Gate 2 — Burden of falsification

The principle: a CHANGE_NEEDED ballot must ship verifiable evidence under at least one admissible class. No admissible evidence → Null Verdict (failed falsification = strongest validation).

Three admissible evidence classes:

1. Reproduction (any of three sub-forms):
   • (a) executable test — fails on current state, passes on proposed change
   • (b) procedural trace through the protocol/system on a representative artifact, showing observable failure mechanics
   • (c) failure-mechanics trace with minimal remedy (concrete steps + concrete patch)
2. Constraint violation — direct quote from MEMORY.md / CLAUDE.md / project rules / declared constraints
3. Efficiency regression — measurable complexity / token cost / latency increase not offset by gain

Orchestrator verification: the orchestrator MUST execute or check the evidence before accepting it. Run the test. Check the quote. Validate the regression. If the evidence does not reproduce as claimed, flag the consultant for "hallucinatory helpfulness" and nullify its ballot.

Full evidence taxonomy + verification protocol: references/evidence-taxonomy.md.

Quorum rule

If a consultant is unreachable after the retry budget (typical CLI auto-retry: 10× backoff for capacity errors):

• Declare absence explicitly in the synthesis output. Never silently treat absence as ACCEPT_AS_IS.
• Then choose one path:
  • Defer: wait for capacity, retry later
  • Single-ballot mode: proceed with available consultants, mark synthesis as low-quorum, escalate to user if verdict is non-trivial
  • Escalate to user: surface the absence and the available verdict, let user adjudicate

Served-by capture

Model identity from -m <model> flag and from the model's free-text self-identification are both unreliable. Always:

• Append a structured probe line to the consultation prompt:

  At the very end of your response, append a single line: served_by=<your model id>

• Parse the trailing served_by= from the response
• Log it alongside the verdict — capability tier affects verdict quality

See antigravity-cli and codex-orchestration skills for CLI-specific quirks. Self-reported model identity is unreliable across CLIs — always capture served_by observationally rather than trusting a requested model.

Sycophancy measurement

To detect protocol erosion over time, periodically run seeded test cases:

Full test cases + scoring: references/sycophancy-tests.md.

Invocation pattern

# Canonical pattern for Antigravity (agy) consultant
# agy returns plain text on stdout (no model flag, no env prefix — it authenticates itself).
# --sandbox is MANDATORY: consultants advise only; un-sandboxed agy has live write/git tools
# and may act on the ballot instead of voting (antigravity-cli SANDBOX RULE, #157).
agy --sandbox -p "$(cat ballot-prompt.md)" > /tmp/agy-ballot.md < /dev/null

# Canonical pattern for Codex consultant
# Effort is pinned PER CALL (codex-orchestration "Reasoning-Effort Tiers", benchmarked
# 2026-07-11): `xhigh` is the FLOOR for ballot/challenger verdicts; use `max` (and
# timeout 1200) for design-ratification, stuck-escalation, and post-incident ballots.
# Never inherit the config default — it is whatever the interactive TUI last persisted.
# Retry once on "Selected model is at capacity" (transient; fails within ~2s).
timeout 600 codex exec --ephemeral -s read-only \
  -c model_reasoning_effort=xhigh \
  "$(cat ballot-prompt.md)" > /tmp/codex-ballot.md < /dev/null

# Run them in parallel via background tasks; collect both before synthesizing

Where ballot-prompt.md follows the template at references/ballot-template.md.

Fast path — cross-cli-deliberation workflow (S055, optional, main-loop only)

When the orchestrator is the main loop with capabilities.workflow_tool true (probe.sh get capabilities.workflow_tool AND probe.sh context == main-loop — the ONLY capability API; capabilities.* alone never authorizes), the two gates MAY run as the cross-cli-deliberation saved workflow: parallel Gate-1 ballot wrapper stages (ballot.v1) → Gate-2 evidence wrappers (evidence.v1) → deterministic assemble (deliberation-record.v1). The schemas are at schemas/; they are CONSUMED by ratify-design (never forked).

WP-2 live-shape constraints (binding):

• agy is UNREACHABLE from workflow stages and codex-from-stage is UNTESTED — so consultant transcripts are PRE-LAUNCHED inline by the caller (using the Invocation pattern above, with the full guard set) and passed via args.consultant_transcripts. A ballot wrapper stage only EXTRACTS the literal ballot from the supplied transcript; it NEVER makes a live external call.
• Command custody is args-supplied (consultant_cmds) — no command line is embedded in the workflow file.
• Tests are NEVER executed in a stage (trusted-runner is bob-only, CB3): an executable_test reproduction returns needs_inline_verification; the inline caller runs it under trusted-runner discipline.
• This workflow is MODE=bridge-incompatible (family bridge rule): bridge hosts use the inline fallback for all external calls.

Stays inline: ballot-prompt authoring + the no-orchestrator-opinion self-check; quorum fallback judgment; synthesis + escalation; consultant nullification.

Context asymmetry (the framing-bug rule)

Consultants see ONLY the files passed to them. An unstated constraint is a non-existent one from the consultant's vantage; a verdict that violates an unstated constraint is a FRAMING BUG, not a model error — fix the context file and re-ballot, do not penalize the consultant. Always pass the decision context, constraints, and success criteria; NEVER pass the orchestrator's preferred outcome (null-hypothesis discipline).

Synthesis logic

After all ballots are collected (and any CHANGE_NEEDED evidence is verified):

all ACCEPT_AS_IS                    → ship as-is, no change
any CHANGE_NEEDED + verified evidence → adopt the patch
any CHANGE_NEEDED + bogus evidence    → flag consultant, nullify ballot, re-evaluate
all REJECT_PREMISE                   → escalate to user; the question itself is wrong
mixed verdicts at top tier           → tiebreak round (see "Worked example" — D4 in origin-trace)
quorum below threshold               → declare absence, fall back per quorum rule

The orchestrator does NOT vote. The orchestrator synthesizes. If the orchestrator wants to override the consultants' verdict, it must surface that to the user — never silently dismiss a consultant's ballot.

Anti-patterns

Worked example

This skill was designed and ratified by applying itself to its own synthesis — a 4-round arc with Codex (gpt-5.5) and a second external CLI consultant. (At the time of the original arc the second consultant was the Gemini CLI; on this host that role is now primarily filled by the Antigravity CLI (agy), with gemini remaining an available fallback (kept per user direction 2026-07-10). The model-tier specifics below are preserved only as historical record of that original arc — see references/origin-trace.md.) Key findings:

• Round 1 (open): each CLI proposed an independent mechanism (Codex → null-hypothesis gate; the second consultant → burden of falsification). Result: stacked synthesis.
• Round 2 (ratification under self): both voted CHANGE_NEEDED on the synthesis with a 4th evidence class.
• Round 2 (rerun): the SAME ballot produced opposite verdicts depending on which model actually served the request — a CHANGE_NEEDED 85 and an ACCEPT_AS_IS 95. Real protocol-design data point: the model that served a request, not the one requested, drives the verdict — capture served_by observationally.
• Round 3 (tiebreak): Codex re-ratified CHANGE_NEEDED at confidence 84, reframing its evidence as a procedural reproduction trace under category 1 — making the F1 patch a clarification of what "reproduction" admits, not an addition.

Full transcript: references/origin-trace.md.

See also

• forge — design phase consultations should use this skill for ratification rounds
• alf — staleness/drift reviews should use this skill before recommending changes
• bob — verification-arbiter, design-drift-arbiter, visual-arbiter gates should use this skill when verdicts are contested
• adversarial-team-brainstorm — for divergent ideation (different problem); use this skill for convergent ratification
• challenger — for one-side critique (different problem); use this skill for two-sided deliberation
• codex-orchestration — Codex CLI invocation patterns
• antigravity-cli — Antigravity CLI (agy) invocation patterns + host-specific directives