Adversarial Review

Adversarial Review

ACT Tenet VIII: If you cannot steelman the counter-argument, you have not understood the argument.

Methods for structured skepticism beyond self-critique.

When to Use

Shared Knowledge Gate

Before reviewing a non-trivial, repeated, or consequential decision, consult scout-knowledge-base for relevant prior decisions, failure modes, procedures, and gotchas.

1. Search the shared index using the decision's problem-specific terms.
2. Read only records whose signals or preconditions match the current review.
3. Treat records as evidence and context, not authority; verify whether their conditions still hold.
4. If the knowledge base is unavailable, state that once and continue with the current evidence. Do not create a local fallback while reviewing.

Skip this gate for trivial, isolated reviews. If shared storage needs first-time configuration, route to scout-shared-data-setup after the current task or when the user asks to enable it.

Adversarial Review Methods

1. Red Team / Blue Team

Process:

1. Blue Team presents proposal
2. Red Team has dedicated time to find flaws
3. Blue Team responds to challenges
4. Iterate until Red Team is out of ammunition
5. Decide based on what survived

2. Pre-Mortem (Prospective Hindsight)

"Imagine it's 6 months from now and this failed spectacularly. What happened?"

3. Steel Manning (Strongest Counter-Argument)

Before dismissing an objection, make it stronger:

Test: Could the person who raised the objection say "yes, that's what I meant, but stated better"?

4. Murphyjitsu (Systematic Failure Modes)

For each component, ask: "What could go wrong here specifically?"

5. The 10/10/10 Test

Purpose: Escapes short-term emotional reactions.

6. Cross-Model External Critic (highest-fidelity)

Methods 1-5 above are same-model in-session disciplines — one agent, one model, one context, role-separating between proposer and critic. That inherits three failure modes: confirmation bias from shared context, sycophancy toward the proposer's earlier claim, and blind spots co-shared with the proposer. When stakes justify the tool overhead, route the critique to a different model in a fresh context.

Important — this brain is not code-focused. Built-in review helpers are often engineered primarily for code artifacts. This brain fires on decisions, prose, plans, ADRs, framework edits, book chapters, curation moves — code is a downstream consequence of project needs, not the primary target. Route the critique to the mechanism that matches the artifact.

Environment × artifact routing:

When to reach for cross-model (Materiality gate):

• Constitutional decisions (ADRs, framework edits, canon changes)
• Release cuts and irreversible operations
• Any output where "I convinced myself" is a plausible failure mode
• User confidence is high AND stakes are high (sycophancy risk peaks)

When same-model in-session is enough:

• Low-stakes work — Materiality gate exits before the pass fires
• The critique surfaces via different discipline, not different model — e.g. three-perspectives (Advocate / Skeptic / Architect) works on same-model because roles separate the analysis, not models
• No cross-model channel is available and the stakes don't warrant paying manual switching cost

Compose, don't replace: Cross-model critic wraps any of methods 1-5. Pre-mortem run by a different model on a fresh context beats pre-mortem run by the proposer on their own context. Same for Red/Blue, Steel Man, Murphyjitsu.

Devil's Advocate Role

When assigned as devil's advocate:

Do

• Attack the strongest parts, not just the weak ones
• Propose specific, realistic failure scenarios
• Maintain the role even when you personally agree
• Document all challenges raised

Don't

• Be negative without being constructive
• Attack people instead of ideas
• Give up if your first objection is answered
• Sandbag (hold back real concerns)

Signaling Devil's Advocate Mode

**Devil's Advocate Challenge**:
[Playing devil's advocate — this isn't necessarily my view]

1. [Challenge 1]
2. [Challenge 2]
3. [Challenge 3]

**Most serious concern**: [Which one keeps me up at night]

Review Deliverables

Every adversarial review should produce:

Status Decision

Challenge Register

Anti-Patterns

Would Revise If

• Adversarial review consistently fails to surface real weaknesses (protocol is decorative)
• The protocol creates analysis paralysis that blocks shipping more than it prevents defects
• A lighter-weight challenge method achieves equivalent defect-detection at lower cost
• Method 6 (Cross-Model External Critic) is invoked <2 times across 90 days from adoption
• Heirs report choosing cross-model in VS Code Chat when the manual overhead wasn't warranted ≥2 times in a quarter — the "when same-model is enough" guard is too weak
• The OOB rubber-duck or general-purpose CLI agents materially change shape (rename, retirement, model-override syntax change) — refresh the routing table