zavelinski multi-agent-debate
Several agents answer independently, then read each other's answers and critique/revise across a round or two, converging on a better-reasoned result. Debate surfaces flaws a single pass (or a blind majority vote) misses.
Why this exists (evidence)
- Multiagent debate (Du et al., arXiv:2305.14325): multiple LLM instances proposing and debating over rounds improves factuality and reasoning vs single-agent and vs self-consistency on several benchmarks; the critique step catches errors that independent sampling alone leaves in.
- It differs from self-consistency: there, samples never interact and you majority-vote; here, agents SEE and challenge each other, so a well-argued correction can flip the group.
When to use
- High-stakes design/architecture decisions with multiple defensible options.
- Disputed root-cause analysis (two plausible diagnoses).
- Claims where the QUALITY of the argument matters, not just the modal answer.
- NOT for cheap/clear tasks: debate is the most expensive technique here (N agents x R rounds).
The method
- Propose: N agents (2-4) independently answer the question with reasoning.
- Debate round: each agent reads the others' answers and either defends, concedes, or revises, citing specific flaws. 1-2 rounds is usually enough.
- Converge: stop when they agree or positions stabilize. Synthesize the converged answer + note any unresolved dissent.
- Verify the winner: debate improves reasoning, not ground truth, confirm critical facts/behavior (compose with cite-guard / adversarial-verify).
How to run it
- Workflow: a propose stage (parallel N agents, structured answers), then a debate stage where each agent gets the others' outputs and revises, then a synthesis stage. Optionally give agents distinct lenses (correctness / security / simplicity) so the debate is diverse, not echo.
- Route proposers to a cheaper model; keep the synthesis/judge sharp (model-router).
Composes with
self-consistency: cheaper first pass; escalate to debate only when the vote is split or the call is high-stakes.run-cost: N x R is the priciest pattern here; budget before launching.orchestrate: debate the design at the GATE before building.
Honest limits
- Most expensive technique in this collection (multiplies cost by agents x rounds). Reserve for decisions worth it.
- Agents can converge on a confident shared error (groupthink); diverse lenses + a verification step mitigate, not eliminate.
- Gains are benchmark-specific; measure your own.