Agent Debate

Agent Debate - Truth Through Disagreement

Agent Debate implements the multi-agent debate paradigm for improving accuracy through structured disagreement.

The core insight: Multiple agents answering independently, then critiquing each other, produces more accurate results than single agents—especially on complex reasoning tasks.

The Analogy: Panel Discussion

When to Use Agent Debate

Perfect for:

• ✅ Fact-checking (Is this claim true?)
• 🔢 Complex reasoning (math, logic puzzles)
• ⚖️ Trade-off analysis (pros/cons of each option)
• 📊 Multi-faceted questions (no single right answer)
• 🎯 High-stakes decisions (where accuracy matters)
• 🧠 Reducing hallucinations (when wrong answers sound right)

Skip for:

• ⚡ Simple facts ("What's 2+2?")
• 🎨 Pure creativity (brainstorming, creative writing)
• 🏃 Speed-critical tasks (debate adds latency)

The Debate Pattern

┌─────────────────────────────────────────────────────────────┐
│  ROUND 1: Independent Answers                                 │
│  • Agent A answers question                                   │
│  • Agent B answers same question                              │
│  • Agent C answers same question (optional)                   │
├─────────────────────────────────────────────────────────────┤
│  ROUND 2: Critique                                            │
│  • Each agent reviews others' answers                         │
│  • Identifies errors, omissions, weak reasoning              │
│  • Scores or ranks the answers                               │
├─────────────────────────────────────────────────────────────┤
│  ROUND 3: Refined Answers (optional)                          │
│  • Agents revise based on critique                           │
│  • Incorporate valid criticisms                              │
│  • Defend positions if critique is wrong                     │
├─────────────────────────────────────────────────────────────┤
│  ROUND 4: Consensus/Verdict (optional)                        │
│  • Agents discuss to reach consensus                          │
│  • Or: Majority vote on best answer                          │
│  • Or: Meta-agent synthesizes best parts                     │
└─────────────────────────────────────────────────────────────┘

The Workflow

Step 1: Generate Independent Answers

Each agent answers WITHOUT seeing others' responses:

AGENT A:
Q: "What caused the 2008 financial crisis?"
A: "The immediate trigger was the collapse of Lehman Brothers in 
September 2008. This was caused by the subprime mortgage crisis, 
where banks made risky loans to borrowers who couldn't repay them."

AGENT B:
Q: "What caused the 2008 financial crisis?"
A: "Multiple interconnected factors: (1) Housing bubble fueled by 
low interest rates and lax lending standards, (2) Financial 
innovations like CDOs that spread risk opaquely, (3) Regulatory 
gaps allowing excessive leverage, (4) Lehman Brothers collapse 
amplifying panic."

AGENT C:
Q: "What caused the 2008 financial crisis?"
A: "Primarily deregulation in the 1990s-2000s that allowed banks 
to take bigger risks. The Gramm-Leach-Bliley Act and Commodity 
Futures Modernization Act created the conditions."

⚠️ Critical: Agents must NOT see each other's answers before this step.

Step 2: Structured Critique

Each agent reviews all answers:

AGENT A CRITIQUES:

Reviewing Agent B:
✓ Good: Comprehensive, multiple factors
⚠️ Missing: Doesn't mention specific deregulation acts
⚠️ Weak: "amplifying panic" vague

Reviewing Agent C:
✓ Good: Specific legislation mentioned
✗ Error: Overemphasizes deregulation, understates housing bubble
✗ Missing: Doesn't explain the immediate trigger mechanism

SELF-CRITIQUE:
✗ My answer too narrow - focused only on Lehman/subprime
✗ Missing: Financial instruments, leverage, regulatory context

AGENT B CRITIQUES:

Reviewing Agent A:
✗ Error: "immediate trigger was Lehman" - actually Lehman was 
   a symptom, not root cause
✗ Too narrow: Focuses only on subprime, misses derivatives
⚠️ Missing: Regulatory failures

Reviewing Agent C:
✓ Good: Legislation specifics
⚠️ Incomplete: Deregulation enabled but didn't cause crisis alone
✗ Missing: Housing bubble mechanics

SELF-CRITIQUE:
✓ Comprehensive
⚠️ Could be clearer on sequence: bubble → instruments → panic

AGENT C CRITIQUES:

Reviewing Agent A:
✗ Error: Lehman collapse was proximate cause, not root cause
✗ Missing: The deregulation that enabled risky behavior
⚠️ Shallow: Doesn't explain why subprime spread systemically

Reviewing Agent B:
✓ Good: Captures complexity
✓ Good: Mentions leverage
⚠️ Missing: Specific legislation (I should check if that's important)

SELF-CRITIQUE:
✗ Too focused on deregulation, understates housing bubble
✗ Missing: The mechanism of contagion

Step 3: Refined Answers (Optional)

Agents revise incorporating valid critiques:

AGENT A REVISED:
"Multiple factors caused the 2008 crisis: (1) Housing bubble from 
low rates and lax lending, (2) Financial innovations (CDOs, CDS) 
that spread risk opaquely, (3) High leverage, (4) Regulatory gaps. 
Lehman Brothers' Sept 2008 collapse was the proximate trigger that 
amplified panic, not the root cause."

[Note: Agent A improved comprehensiveness]

AGENT B REVISED:
"The 2008 crisis resulted from: (1) Housing bubble (1997-2006) 
fueled by low interest rates and lax standards, (2) Financial 
innovations (CDOs, CDS) enabled by deregulation like Commodity 
Futures Modernization Act 2000, (3) Excessive leverage (30:1 
ratios), (4) Regulatory failures. Lehman collapse (Sept 15, 2008) 
triggered systemic panic."

[Note: Agent B added specifics on deregulation and timeline]

AGENT C REVISED:
"The crisis stemmed from interconnected factors: (1) Housing bubble 
(1997-2006) where subprime lending grew from 8% to 20% of mortgages, 
(2) Financial engineering (CDOs, CDS) that obscured risk, enabled by 
deregulation, (3) High bank leverage, (4) Shadow banking system. 
Lehman bankruptcy triggered the acute phase."

[Note: Agent C acknowledged bubble and added data]

Step 4: Consensus or Verdict

Option A: Meta-Synthesis (Recommended)

META-AGENT SYNTHESES:

"After reviewing all three expert analyses and their critiques:

CONSENSUS ELEMENTS (all agree):
- Housing bubble was fundamental cause
- Financial instruments (CDOs/CDS) amplified and spread risk
- High leverage made system fragile
- Multiple factors, not single cause
- Lehman collapse was the triggering event

KEY DEBATE POINTS:
- Agent C emphasized deregulation most strongly
- Agent B provided best timeline and mechanics
- Agent A initially focused narrowly but improved

SYNTHESIZED ANSWER:
The 2008 financial crisis resulted from the interaction of:

1. HOUSING BUBBLE (1997-2006): Low interest rates and lax lending 
   standards created bubble. Subprime mortgages grew from 8% to 20%.

2. FINANCIAL ENGINEERING: CDOs and CDS instruments spread risk 
   opaquely across the system. Enabled by deregulation (Commodity 
   Futures Modernization Act 2000, etc.).

3. LEVERAGE: Banks operated at 30:1 ratios, making them vulnerable.

4. REGULATORY GAPS: Shadow banking system escaped oversight.

TRIGGER: Lehman Brothers bankruptcy (Sept 15, 2008) triggered 
systemic panic and credit freeze.

CONFIDENCE: High (consistent themes across independent analyses)"

Option B: Majority Vote

VOTING ROUND:

"Which answer best captures the causes of the 2008 crisis?"

Agent A: Votes for Agent B (most comprehensive)
Agent B: Votes for Agent B (best balance)
Agent C: Votes for Agent B (fair to all factors)

RESULT: Agent B wins 3/3 votes

Option C: Consensus Building

CONSENSUS DISCUSSION:

Agent A: "I think we all agree housing bubble was fundamental. 
Agent C, do you accept that deregulation alone wasn't sufficient?"

Agent C: "Yes, I overstated that. The bubble was necessary. But 
deregulation was what let banks create those risky instruments."

Agent B: "Agreed. So consensus: bubble created conditions, 
engineering amplified it, Lehman triggered it."

Agents A, C: "Agreed."

CONSENSUS: [Synthesized statement]

Debate Formats

Format 1: Fact-Check (Binary Verdict)

QUESTION: "Is this claim true: 'The Great Wall of China is visible 
from space'?"

Agent A: "False. Astronauts confirm it's too narrow."
Agent B: "False. NASA says it's not visible with naked eye."
Agent C: "False. Urban myths debunked by actual space photos."

[Short debate - consensus reached quickly]

VERDICT: FALSE
CONFIDENCE: Very High (unanimous, evidence-based)

Format 2: Reasoning (Step-by-Step)

QUESTION: "If train A leaves at 2pm going 60mph..."

Agent A: Step-by-step solution
Agent B: Different approach, same answer
Agent C: Third verification method

DEBATE: Compare methods, verify calculations

VERDICT: Answer is X, confirmed by 3 independent methods
CONFIDENCE: Very High

Format 3: Trade-Off Analysis (Multi-Factor)

QUESTION: "Which database should we choose: PostgreSQL or MongoDB?"

Agent A (advocates PostgreSQL)
Agent B (advocates MongoDB)
Agent C (neutral evaluator)

DEBATE: Each presents case, critiques others

VERDICT: Trade-off summary with recommendation
CONFIDENCE: Medium (legitimate trade-offs exist)

Format 4: Error Detection (Hallucination Check)

CLAIM: "Our system can handle 1 million concurrent users."

Agent A: "Likely false. Current architecture shows..."
Agent B: "False. Load testing data shows..."
Agent C: "Questionable. No evidence of testing at that scale."

DEBATE: Each finds flaws in the claim

VERDICT: CLAIM UNVERIFIED
CONFIDENCE: High (multiple independent concerns)

Best Practices

1. Agents Must Be Independent

❌ Bad: Agent B sees Agent A's answer first

✅ Good: All agents answer simultaneously, isolated

2. Critique Must Be Specific

❌ Weak critique: "This is wrong"

✅ Strong critique: "Step 3 has math error: 150/3=50, not 60"

3. Acknowledge Valid Criticisms

❌ Defensive: "No, I'm right"

✅ Constructive: "You're correct about X, I revise to include Y"

4. Don't Force Consensus

If agents legitimately disagree:

"Agents A and B disagree on X. A argues [reasoning]. B argues 
[reasoning]. Both positions are defensible. Decision depends on 
prioritizing [factor] vs [factor]."

5. Use Odd Number of Agents

Breaks ties in voting:

• 3 agents: Simple, effective
• 5 agents: More thorough, higher cost

Examples

Example 1: Fact-Check

Claim to verify: "Python was created in 1995."

## Agent Debate Session: Python Creation Date

### Round 1: Independent Answers

**Agent A:**
"Python was created in 1995 by Guido van Rossum."

**Agent B:**
"Python was first released in 1991 by Guido van Rossum."

**Agent C:**
"Python was created in 1989, first public release 1991, version 1.0 in 1994."

### Round 2: Critique

**Agent A Critiques:**
- Self: My answer of 1995 is wrong. That's probably when I started using it.
- Agent B: Correct about 1991 release
- Agent C: Most accurate with timeline

**Agent B Critiques:**
- Agent A: Incorrect. 1995 is wrong.
- Agent C: Correct and most detailed
- Self: I got main date right (1991) but missed creation year (1989)

**Agent C Critiques:**
- Agent A: Significantly wrong (4 years off)
- Agent B: Mostly correct, missing creation context
- Self: Correct and complete

### Round 3: Verdict

**Meta-Synthesis:**
Agent A was incorrect (1995). Agent B was partially correct (1991 release). 
Agent C was fully correct (1989 creation, 1991 release).

**VERDICT: FALSE** - Python was not created in 1995.
**CORRECT ANSWER:** Python created 1989, first released 1991.
**CONFIDENCE:** Very High (2/3 agents correct, 1 self-corrected)

Example 2: Complex Reasoning

Question: "If a bat and ball cost $11 total, and the bat costs $10 more than the ball, how much is the ball?"

## Agent Debate Session: Bat and Ball

### Round 1: Independent Answers

**Agent A:**
"Bat = $10, Ball = $1. Total $11. Bat costs $10 more than ball. ✓"

**Agent B:**
"Let ball = x. Bat = x + 10. Total: x + (x+10) = 11. 2x + 10 = 11. 
2x = 1. x = 0.50. Ball = $0.50."

**Agent C:**
"Ball = $1. Because $10 bat + $1 ball = $11."

### Round 2: Critique

**Agent A Critiques:**
- Self: Wait, if ball is $1 and bat is $10 more, bat should be $11. 
  But then total is $12. Error!
- Agent B: Correct algebra
- Agent C: Same error I made

**Agent B Critiques:**
- Agent A: Wrong. Didn't check: $10 more than $1 is $11, total $12.
- Agent C: Same error.
- Self: Correct. Verify: Ball $0.50, bat $10.50 (which is $10 more). 
  Total $11. ✓

**Agent C Critiques:**
- Self: I didn't verify! $10 more than $1 is $11, total would be $12.
- Agent A: Same mistake.
- Agent B: Correct with algebra.

### Round 3: Refined Answers

**Agent A Revised:**
"I made an error. Correct answer: Ball = $0.50, Bat = $10.50."

**Agent C Revised:**
"Corrected. Ball = $0.50."

### Verdict

**CORRECT ANSWER:** Ball costs $0.50
**CONFIDENCE:** Very High (3/3 agents now agree)
**NOTE:** Common cognitive bias (System 1 thinking) caught through debate.

Example 3: Trade-Off Analysis

Question: "Should we build or buy our analytics platform?"

## Agent Debate Session: Build vs Buy Analytics

### Round 1: Independent Positions

**Agent A (Pro-Build):**
"Build custom. Reasons: (1) Full control over features, (2) No 
vendor lock-in, (3) Data stays in-house, (4) Customizable to our 
exact needs, (5) Long-term cost savings."

**Agent B (Pro-Buy):**
"Buy existing solution. Reasons: (1) 6 months vs 18 months to 
market, (2) $50K/year vs $800K engineering cost, (3) Battle-tested 
reliability, (4) Support included, (5) Focus engineering on core product."

**Agent C (Evaluator):**
"Hybrid approach: Buy core (Segment), build custom reporting on top."

### Round 2: Critique

**Agent A Critiques:**
- Agent B: Underestimates integration costs. Vendor APIs change.
- Agent C: Hybrid has worst of both - still vendor dependency.
- Self: I may underestimate time - custom analytics is complex.

**Agent B Critiques:**
- Agent A: Underestimates cost. Analytics is hard - need team of 3-4.
- Agent C: Hybrid is reasonable middle ground.
- Self: May overestimate vendor reliability - check SLA.

**Agent C Critiques:**
- Agent A: Correct that build gives control, but timeline?
- Agent B: Timeline advantage is real, but customization limited?
- Self: Hybrid works if our needs are mostly standard reporting.

### Round 3: Consensus Discussion

**Key Points of Agreement:**
- Timeline: Buy fastest, Build slowest
- Cost: Build expensive upfront, Buy has ongoing costs
- Control: Build wins, Buy loses

**Unresolved:**
- Agent A: Custom needs require build
- Agent B: Standard needs met by buy

### Final Recommendation

"After debate:

**IF** analytics needs are standard (tracking, basic reporting): **BUY**
- Faster time to value
- Lower risk
- Engineering focus on core product

**IF** analytics is competitive differentiator (custom ML, unique 
metrics): **BUILD**
- Required for differentiation
- Long-term cost justified

**HYBRID** for middle ground: Buy foundation (Segment), build 
custom reporting layer.

Given current context [unknown], recommend: Start with BUY, reassess 
in 12 months based on actual customization needs."

Example 4: Hallucination Detection

Claim: "Our API can process 100,000 requests per second."

## Agent Debate Session: Verify API Capacity Claim

### Round 1: Independent Assessment

**Agent A:**
"Doubtful. Current architecture: single Node.js instance. Node can 
handle ~10K req/s per core. We have 4 cores = ~40K theoretical max. 
Database would bottleneck before that."

**Agent B:**
"False. Load testing last month showed 15K req/s before degradation. 
100K is 6x higher than tested capacity."

**Agent C:**
"Questionable. No evidence provided. Would require:
- Horizontal scaling (not implemented)
- Database optimization (not done)
- Caching layer (not deployed)

None of these are in place."

### Round 2: Critique

**Agent A:**
- B confirms my calculation with empirical data
- C adds architectural requirements
- Consensus emerging: claim is false

**Agent B:**
- A's theoretical calc matches my empirical finding
- C identifies what's missing for 100K
- All agree claim is unsupported

**Agent C:**
- A and B agree on actual capacity (~15K)
- My architectural analysis confirms gap
- Unanimous: claim cannot be true

### Verdict

**VERDICT: CLAIM UNVERIFIED / LIKELY FALSE**

**EVIDENCE:**
- Empirical test: 15K req/s (Agent B)
- Theoretical max: 40K req/s (Agent A)
- Architecture gap: Missing horizontal scaling (Agent C)

**ACTUAL CAPACITY:** ~15,000 req/s (tested)
**CLAIMED:** 100,000 req/s (6.6x overstatement)

**CONFIDENCE:** Very High (unanimous agreement)

**RECOMMENDATION:**
1. Retract claim pending load testing
2. Conduct proper load testing to find actual limit
3. Implement horizontal scaling if 100K truly needed
4. Add monitoring/alerts before approach limits

Integration with Other Skills

Agent Debate + Plan First:

• Debate: Which approach is best?
• Plan First: Execute the winning approach

Agent Debate + Self-Critique:

• Independent answers with self-critique built in
• Peer critique adds external validation

Agent Debate + Team Code:

• Debate: Which architecture pattern?
• Team Code: Build the consensus choice

Quick Start Template

## Agent Debate: [Question]

### Round 1: Independent Answers

**Agent A:**
[Answer]

**Agent B:**
[Answer]

**Agent C:**
[Answer]

### Round 2: Critique

**Agent A Critiques:**
- Self: [What I got wrong]
- Agent B: [Their errors/omissions]
- Agent C: [Their errors/omissions]

**Agent B Critiques:**
...

**Agent C Critiques:**
...

### Round 3: Refined Answers (optional)

**Agent A Revised:**
...

### Verdict

**Consensus:** [What we agree on]
**Disagreement:** [What's disputed]
**Answer:** [Final synthesized answer]
**Confidence:** [High/Medium/Low]

References

• Research: "Improving Factuality and Reasoning in Language Models through Multiagent Debate" (Du et al., 2023)
• Related: Ensemble methods, Wisdom of Crowds, Delphi method
• See references/examples.md for detailed debate examples