# Multi Model Validation

> Runs the same task across multiple AI models in parallel and aggregates verdicts. Use when the user wants a second opinion, multi-expert validation, or consensus from Grok, Gemini, GPT-5, or Kimi.

- Skill: `madappgang/multi-model-validation-2` (Agent Skill)
- Install (CLI): `npx skillmds@latest add madappgang/multi-model-validation-2`
- Raw SKILL.md: https://api.skillmd.com/api/skills/madappgang/multi-model-validation-2/raw
- Safety review: pending (external: skill-scanner PASS, skillspector WARNING)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: MadAppGang (https://skillmd.com/u/madappgang)
- Updated: 2026-09-21
- Page: https://skillmd.com/skills/madappgang/multi-model-validation-2

---


# Multi-Model Validation

**Version:** 3.3.0
**Purpose:** Patterns for running multiple AI models in parallel via Claudish proxy with **context-aware preferences**, dynamic model discovery, session-based workspaces, and performance statistics
**Status:** Production Ready

## Overview

Multi-model validation is the practice of running multiple AI models (Grok, Gemini, GPT-5, DeepSeek, etc.) in parallel to validate code, designs, or implementations from different perspectives. This achieves:

- **3-5x speedup** via parallel execution (15 minutes → 5 minutes)
- **Consensus-based prioritization** (issues flagged by all models are CRITICAL)
- **Diverse perspectives** (different models catch different issues)
- **Cost transparency** (know before you spend)
- **Free model discovery** (NEW v3.0) - find high-quality free models from trusted providers
- **Performance tracking** - identify slow/failing models for future exclusion
- **Data-driven recommendations** - optimize model shortlist based on historical performance

**Key Innovations:**

1. **Context-Aware Preferences** (NEW v3.3.0) - Automatically use saved model preferences per task type (debug/research/coding/review) from `.claude/multimodel-team.json`
2. **Dynamic Model Discovery** (v3.0) - Read the live catalog (`list_models`) for current available models (live, 24h cache)
3. **Session-Based Workspaces** (v3.0) - Each validation session gets a unique directory to prevent conflicts
4. **4-Message Pattern** - Ensures true parallel execution by using only Agent tool calls in a single message
5. **Pattern 7-8** - Statistics collection and data-driven model recommendations

This skill is extracted from the `/review` command and generalized for use in any multi-model workflow.

---

## ⚠️ MANDATORY: Learn and Reuse User Preferences

> **Model preferences are learned per context and reused automatically.**
>
> - First time a context is used → ASK user → SAVE to that context
> - Next time same context → VALIDATE the saved IDs against `list_models`, then use
>   the survivors automatically (no asking). "No asking" applies to the *selection*,
>   never to the catalog check — saved IDs go stale and must be re-checked every run.
> - User explicitly says "change models" or "different models" → ASK and UPDATE

```bash
# FIRST STEP - Read preferences file
cat .claude/multimodel-team.json 2>/dev/null
```

**Flow:**

```
1. Detect context from task keywords
   - "debug", "error", "bug", "fix" → debug
   - "research", "analyze", "investigate" → research
   - "implement", "build", "create", "code" → coding
   - "review", "audit", "check" → review

2. Check if contextPreferences[context] exists and is non-empty

   IF EXISTS (has models saved):
   → Call: list_models (claudish MCP) and KEEP ONLY the saved IDs it still lists
     Saved preferences are user policy, not a catalog snapshot — they go stale
     silently. This applies to defaultModels and contextPreferences alike; see
     claudish:claudish-usage → "Every field of the preferences file is untrusted"
   → Name every dropped ID in your reply
   → DO NOT ask the user to re-pick while at least one saved ID survives
   → If NOTHING survives, say so and offer live alternatives

   IF EMPTY/MISSING (first time for this context):
   → Call: list_models (claudish MCP — current models, pricing, capabilities)
   → Ask user to select models (AskUserQuestion)
   → Save to contextPreferences[context]
   → Proceed with validation

3. User override triggers (explicit request to change):
   - "use different models"
   - "change models"
   - "update model preferences"
   → Ask user to select new models
   → Update contextPreferences[context]
```

**Example - Learning Flow:**

```
# First debug task ever:
Task: "Debug this authentication error"
→ Context: debug
→ contextPreferences.debug is empty
→ ASK: "Which models for debug tasks?"
→ User selects: grok, glm, minimax
→ SAVE to contextPreferences.debug
→ Run with those models

# Second debug task:
Task: "Debug the API timeout"
→ Context: debug
→ contextPreferences.debug = ["grok", "glm", "minimax"]
→ USE directly (no asking)
→ Run with saved models

# User wants to change:
Task: "Debug this error, use different models"
→ Detected: "different models" override trigger
→ ASK: "Which models for debug tasks?"
→ User selects: gemini, LATEST_GPT_MODEL
→ UPDATE contextPreferences.debug
→ Run with new models
```

---

## Related Skills

> **CRITICAL: Tracking Protocol Required**
>
> Before using any patterns in this skill, ensure you have completed the
> pre-launch setup from `multimodel:model-tracking-protocol`.
>
> Launching models without tracking setup = INCOMPLETE validation.

**Cross-References:**

- **multimodel:model-tracking-protocol** - MANDATORY tracking templates and protocols (NEW in v0.6.0)
  - Pre-launch checklist (8 required items)
  - Tracking table templates
  - Failure documentation format
  - Results presentation template
- **multimodel:quality-gates** - Approval gates and severity classification
- **multimodel:task-orchestration** - Progress tracking during execution
- **multimodel:error-recovery** - Handling failures and retries

**Skill Integration:**

This skill (`multi-model-validation`) defines **execution patterns** (how to run models in parallel).
The `model-tracking-protocol` skill defines **tracking infrastructure** (how to collect and present results).

**Use both together:**
```yaml
skills: multimodel:multi-model-validation, multimodel:model-tracking-protocol
```

---

## Core Patterns

### Pattern 0: Session Setup and Model Discovery (NEW v3.0)

**Purpose:** Create isolated session workspace and discover available models dynamically.

**Why Session-Based Workspaces:**

Using a fixed directory like `ai-docs/reviews/` causes problems:
- ❌ Multiple sessions overwrite each other's files
- ❌ Stale data from previous sessions pollutes results
- ❌ Hard to track which files belong to which session

Instead, create a **unique session directory** for each validation:

```bash
# Generate unique session ID
TARGET_SLUG=$(echo "${TASK_NAME:-review}" | tr '[:upper:] ' '[:lower:]-' | sed 's/[^a-z0-9-]//g' | head -c20)
SESSION_ID="review-${TARGET_SLUG}-$(date +%Y%m%d-%H%M%S)-$(head -c 4 /dev/urandom | xxd -p)"
SESSION_DIR="ai-docs/sessions/${SESSION_ID}"

# Create session workspace
mkdir -p "$SESSION_DIR"

echo "Session: $SESSION_ID"
echo "Directory: $SESSION_DIR"

# Example output:
# Session: review-auth-impl-20251212-143052-a3f2
# Directory: ai-docs/sessions/review-auth-impl-20251212-143052-a3f2
```

**Benefits:**
- ✅ Each session is isolated (no cross-contamination)
- ✅ Traceable - can associate files with a specific session
- ✅ Session ID can be used for tracking in statistics
- ✅ Parallel sessions don't conflict
- ✅ Aligned with the `dev:dev` session pattern
- ✅ Committed to git for audit trail (unlike `/tmp/`)

> **⚠️ Do NOT use `/tmp/` for session directories.** Files in `/tmp/` are not
> traceable, not committable, and parallel runs will overwrite each other.

---

**Dynamic Model Discovery:**

**NEVER hardcode model lists.** Models change frequently — new ones appear, old ones deprecate, pricing updates. Instead, read the live catalog (`list_models`) for current available models:

Call the `list_models` MCP tool (claudish). It returns the current recommended
set — model IDs, pricing, context window, capabilities, and the `provider@model`
access prefixes — served from claudish’s catalog with a 24-hour cache.

For every live variant in one family, call `search_models` with the family name.

**Recommended Free Models for Code Review:**

| Model | Provider | Context | Capabilities | Why Good |
|-------|----------|---------|--------------|----------|
| `qwen/LATEST_FREE_CODING_MODEL` | Qwen | 262K | Tools ✓ | Coding-specialized, large context |
| `mistralai/LATEST_FREE_CODING_MODEL` | Mistral | 262K | Tools ✓ | Dev-focused, excellent for code |
| `qwen/LATEST_FREE_REASONING_MODEL` | Qwen | 131K | Tools ✓ Reasoning ✓ | Massive 235B model, reasoning |

**Model Selection Flow (Learn and Reuse):**

```
1. Read Preferences File
   → cat .claude/multimodel-team.json
   → If file NOT exists → create empty one

2. Detect Task Context
   → Parse task for keywords (case-insensitive):
     - "debug", "error", "bug", "fix", "trace", "issue" → debug
     - "research", "investigate", "analyze", "explore", "find" → research
     - "implement", "build", "create", "code", "develop", "feature" → coding
     - "review", "audit", "check", "validate", "verify" → review
   → If no keywords match → context = "default"

3. Check for Override Triggers in User Message
   → "use different models", "change models", "update preferences"
   → If found → force_ask = true

4. Load or Learn Models
   → models = contextPreferences[context]

   IF models exist AND NOT force_ask:
     → USE models directly (no asking)
     → Go to step 6

   IF models empty OR force_ask:
     → Read: the live catalog (list_models)
     → AskUserQuestion with multiSelect
     → Save user selection to contextPreferences[context]
     → Go to step 6

5. Save Updated Preferences
   → Write .claude/multimodel-team.json
   → Update lastUpdated timestamp

6. Execute with Models
   → Launch parallel validation
   → No further confirmation needed
```

**Context Keywords:**

| Context | Keywords |
|---------|----------|
| debug | debug, error, bug, fix, trace, issue |
| research | research, investigate, analyze, explore, find |
| coding | implement, build, create, code, develop, feature |
| review | review, audit, check, validate, verify |

**Override Triggers (force re-selection):**
- "use different models"
- "change models"
- "update model preferences"
- "select new models"

### Routing is Claudish's

**Send the `id` from `list_models`. Never build an address.** Claudish owns backend
selection, credentials and fallback; this repo implements none of it.

A prefix/backend/key table used to sit here. It is deleted — it had drifted to the wrong
separator (`/` where claudish uses `@`), listed alias env vars as if canonical, covered a
third of the providers, and marked models "collision-free" that had since gained a direct
provider. A second copy in `claudish-usage` had drifted differently, which is the point:
restating claudish's routing anywhere in this repo guarantees two versions of the truth and
no way to tell which is stale.

If a model will not route, that is a claudish bug — report it with `report_error`. Do not
work around it by choosing a different prefix here.

**Interactive Model Selection (AskUserQuestion with multiSelect):**

**CRITICAL:** Use AskUserQuestion tool with `multiSelect: true` to let users choose models interactively. This provides a better UX than just showing recommendations.

```typescript
// Use AskUserQuestion to let user select models
AskUserQuestion({
  questions: [{
    question: "Which external models should validate your code? (Internal Claude reviewer always included)",
    header: "Models",
    multiSelect: true,
    options: [
      // Top paid (from the live catalog (list_models) + historical data)
      {
        label: "grok ⚡",
        description: "$0.85/1M | Quality: 87% | Avg: 42s | Fast + accurate"
      },
      {
        label: "gemini",
        description: "$7.00/1M | Quality: 91% | Avg: 55s | High accuracy"
      },
      // Free models — filter the list_models result by pricing
      {
        label: "qwen/LATEST_FREE_CODING_MODEL 🆓",
        description: "FREE | Quality: 82% | 262K context | Coding-specialized"
      },
      {
        label: "mistralai/LATEST_FREE_CODING_MODEL 🆓",
        description: "FREE | 262K context | Dev-focused, new model"
      }
    ]
  }]
})
```

**Remember Selection for Session:**

Store the user's model selection in the session directory so it persists throughout the validation:

```bash
# After user selects models, save to session
save_session_models() {
  local session_dir="$1"
  shift
  local models=("$@")

  # Always include internal reviewer
  echo "claude-embedded" > "$session_dir/selected-models.txt"

  # Add user-selected models
  for model in "${models[@]}"; do
    echo "$model" >> "$session_dir/selected-models.txt"
  done

  echo "Session models saved to $session_dir/selected-models.txt"
}

# Load session models for subsequent operations
load_session_models() {
  local session_dir="$1"
  cat "$session_dir/selected-models.txt"
}

# Usage:
# After AskUserQuestion returns selected models
save_session_models "$SESSION_DIR" "grok" "qwen/LATEST_FREE_CODING_MODEL"

# Later in the session, retrieve the selection
MODELS=$(load_session_models "$SESSION_DIR")
```

**Session Model Memory Structure:**

```
$SESSION_DIR/
├── selected-models.txt    # User's model selection (persists for session)
├── claude-review.md       # Internal review
├── grok-review.md         # External review (if selected)
├── qwen-coder-review.md   # External review (if selected)
└── consolidated-review.md # Final consolidated review
```

**Why Remember the Selection:**

1. **Re-runs**: If validation needs to be re-run, use same models
2. **Consistency**: All phases of validation use identical model set
3. **Audit trail**: Know which models produced which results
4. **Cost tracking**: Accurate cost attribution per session

**Always Include Internal Reviewer:**

```
BEST PRACTICE: Always run internal Claude reviewer alongside external models.

Why?
✓ FREE (embedded Claude, no API costs)
✓ Fast baseline (usually fastest)
✓ Provides comparison point
✓ Works even if ALL external models fail
✓ Consistent behavior (same model every time)

The internal reviewer should NEVER be optional - it's your safety net.

In the code-review panels `dev` dispatches — where claudish is optional — it is not a
claudish slot. Launch it as its own
`Agent(subagent_type: "dev:reviewer", run_in_background: false, …)` in the SAME message
as the `team` call, carrying the contract lines — `TARGET: BRANCH`, `FOCUS:`, its own
`OUTPUT:` path, and `MODELS:` naming the externals. Never seat it in that team's `models`
list. Its return is in your hands: list its OUTPUT file on REVIEWS: whether or not it
carries a `**Verdict**:` line — the synthesizer counts a file with no verdict as
no-verdict, never as one more approval. (`/team` itself, where claudish is a hard
dependency, seats it as the `internal` slot instead — Pattern 3 states the rule.)
```

---

### Pattern 1: The 4-Message Pattern (MANDATORY)

This pattern is **CRITICAL** for achieving true parallel execution with multiple AI models.

**Why This Pattern Exists:**

Claude Code executes tools **sequentially by default** when different tool types are mixed in the same message. To achieve true parallelism, you MUST:
1. Use ONLY one tool type per message
2. Ensure all Agent calls are in a single message
3. Separate preparation (Bash) from execution (Task) from presentation

**The Pattern:**

```
Message 1: Preparation (Bash Only)
  - Create workspace directories
  - Validate inputs (check if claudish installed)
  - Write the brief (input.md) — never a pre-computed diff: every reviewer is
    handed `TARGET: BRANCH` and captures its own surfaces through dev's
    `capture-review-surfaces.ts`
  - NO Agent calls
  - NO Tasks calls

Message 2: Parallel Execution (the internal Agent call and ONE team call, same message)
  - `Agent(subagent_type: "dev:reviewer", run_in_background: false, …)` — the
    internal reviewer, always present, carrying the contract lines
  - Every external model in a single `team` MCP call; the tool parallelises them
    internally. In a dev-dispatched panel the internal reviewer is never a
    `models` entry (Pattern 3 — `/team` itself seats it as the `internal` slot)
  - Pass require_pattern whenever the prompt mandates an output shape
  - (A pure-Agent fan-out with no external models still obeys the
     one-tool-type-per-message rule above)

Message 3: Auto-Consolidation (Task Only)
  - Automatically triggered when the panel settles — at N = 1 too, where the
    synthesizer passes the single review through with a `VERDICT:` line (Pattern 5)
  - Launch `dev:synthesizer` — the only consolidator; it reads reviews, never code
  - Pass every review file path on REVIEWS:, the three lines under 'Apply verdict
    thresholds' in `dev:reviewer`'s agent file on THRESHOLDS: — read at dispatch
    time, never recalled — and the consolidated file on OUTPUT: (Pattern 5)
  - Consensus analysis is the synthesizer's; never send the reviews to dev:reviewer

Message 4: Present Results
  - Show user prioritized issues
  - Include consensus levels (unanimous, strong, majority)
  - Link to detailed reports
  - Cost summary (if applicable)
```

**Example: 5-Model Parallel Code Review**

```
Message 1: Preparation (Session Setup + Model Discovery)
  # Create unique session workspace
  Bash: SESSION_ID="review-$(date +%Y%m%d-%H%M%S)-$(head -c 4 /dev/urandom | xxd -p)"
  Bash: SESSION_DIR="ai-docs/sessions/${SESSION_ID}" && mkdir -p "$SESSION_DIR"
  # No code capture here. Every reviewer is handed TARGET: BRANCH and runs dev's
  # capture-review-surfaces.ts itself, in BRANCH mode. A range computed here would
  # be one more hand-rolled diff, which is the one thing no dispatcher may do.

  # Discover available models
  MCP:  list_models   # current models, pricing, capabilities

  # User selects models via AskUserQuestion (see Pattern 0)

Message 2: Start the panel (the internal Agent call and ONE team call, same message)
  Bash: write the brief to "$SESSION_DIR/input.md" — the contract lines every
        external gets: TARGET: BRANCH / FOCUS: code / MODELS: none

  Agent(
    subagent_type: "dev:reviewer",
    run_in_background: false,
    description: "Internal code review",
    prompt: "TARGET: BRANCH
             FOCUS: code
             OUTPUT: $SESSION_DIR/claude-review.md
             MODELS: grok,LATEST_FREE_CODING_MODEL,gpt,LATEST_FREE_REASONING_MODEL"
  )
  ---
  claudish team(mode="run", path=$SESSION_DIR,
    models=["grok", "LATEST_FREE_CODING_MODEL", "gpt", "LATEST_FREE_REASONING_MODEL"],
    input_file="$SESSION_DIR/input.md",
    require_pattern="\*\*Verdict\*\*: (PASS|CONDITIONAL|FAIL)", agent="dev:reviewer")

  All 5 reviewers run at once: the Agent is the always-present internal reviewer,
  and the team tool parallelises the four externals internally. The internal
  reviewer is never a `models` entry — its return is in your hands, and its OUTPUT
  file goes on REVIEWS: whether or not it carries a `**Verdict**:` line; the
  synthesizer counts a file with no verdict as no-verdict, never as approval.

  The team call RETURNS IMMEDIATELY with a slots map. It does not carry the reviews.

Message 2b: Poll to completion
  claudish team(mode="status", path=$SESSION_DIR)
  # repeat until no slot in `models` has state === "RUNNING"
  # bound the loop; read idle_seconds_by_slot with activity_by_slot before
  # concluding a quiet slot is stuck

Message 3: Auto-Consolidation
  (Automatically triggered - don't wait for user to request)

  # The internal reviewer wrote $SESSION_DIR/claude-review.md. The team run wrote
  # one file per external slot, $SESSION_DIR/response-NN.md, named by
  # ANONYMOUS slot id rather than by model, because the vote is blind. Do not try
  # to attribute a file to a model before the verdict is in. The synthesizer is
  # the only consolidator: it is given the reviews and never the code, and no
  # reviewer ever sees another reviewer's output.

  Agent(
    subagent_type: "dev:synthesizer",
    run_in_background: false,
    description: "Consolidate code reviews",
    prompt: "REVIEWS: $SESSION_DIR/claude-review.md
             $SESSION_DIR/response-01.md
             $SESSION_DIR/response-02.md
             $SESSION_DIR/response-03.md
             $SESSION_DIR/response-04.md
             THRESHOLDS: <the three lines under 'Apply verdict thresholds' in
                          dev:reviewer's agent file, read at dispatch time, never
                          recalled>
             OUTPUT: $SESSION_DIR/consolidated-review.md
             Consolidate with consensus levels (unanimous / strong / majority / divergent).
             Compute the verdict line from your counts against THRESHOLDS.
             You are given reviews, never code. Do not review."
  )

Message 4: Present Results + Update Statistics
  # Track performance for each model (see Pattern 7)
  track_model_performance "claude-embedded" "success" 32 8 95
  track_model_performance "grok" "success" 45 6 87
  track_model_performance "qwen/LATEST_FREE_CODING_MODEL" "success" 52 5 82
  track_model_performance "gpt" "success" 68 7 89
  track_model_performance "mistralai/LATEST_FREE_CODING_MODEL" "success" 48 5 84

  # Record session summary
  record_session_stats 5 5 0 68 245 3.6

  "Multi-model code review complete! 5 AI models analyzed your code.
   Session: $SESSION_ID

   Top 5 Issues (Prioritized by Consensus):
   1. [UNANIMOUS] Missing input validation on POST /api/users
   2. [UNANIMOUS] SQL injection risk in search endpoint
   3. [STRONG] Weak password hashing (bcrypt rounds too low)
   4. [MAJORITY] Missing rate limiting on authentication endpoints
   5. [MAJORITY] Insufficient error handling in payment flow

   Model Performance (this session):
   | Model                          | Time | Issues | Quality | Cost   |
   |--------------------------------|------|--------|---------|--------|
   | claude-embedded                | 32s  | 8      | 95%     | FREE   |
   | grok          | 45s  | 6      | 87%     | $0.002 |
   | qwen/LATEST_FREE_CODING_MODEL          | 52s  | 5      | 82%     | FREE   |
   | gpt        | 68s  | 7      | 89%     | $0.015 |
   | mistralai/LATEST_FREE_CODING_MODEL   | 48s  | 5      | 84%     | FREE   |

   Parallel Speedup: 3.6x (245s sequential → 68s parallel)

   See $SESSION_DIR/consolidated-review.md for complete analysis.
   Performance logged to ai-docs/llm-performance.json"
```

**Performance Impact:**

- Sequential execution: 5 models × 3 min = 15 minutes
- Parallel execution: max(model times) ≈ 5 minutes
- **Speedup: 3x with perfect parallelism**

---

### Pattern 2: Parallel Execution Architecture

**Single Message, Multiple Tasks:**

The key to parallel execution is putting ALL Agent calls in a **single message** with the `---` delimiter:

```
✅ CORRECT - Parallel Execution:

Agent: <agent-1>
  Prompt: "Task 1 instructions"
---
Agent: <agent-2>
  Prompt: "Task 2 instructions"
---
Agent: <agent-3>
  Prompt: "Task 3 instructions"

All 3 execute simultaneously.
```

**Anti-Pattern: Sequential Execution**

```
❌ WRONG - Sequential Execution:

Message 1:
  Agent: <agent-1>
Message 2:
  Agent: <agent-2>
Message 3:
  Agent: <agent-3>
Each task waits for previous to complete (3x slower).
```

**Independent Tasks Requirement:**

Each Task must be **independent** (no dependencies):

```
✅ CORRECT - Independent:
  Task: review code for security
  Task: review code for performance
  Task: review code for style

  All can run simultaneously (same input, different perspectives).

❌ WRONG - Dependent:
  Task: implement feature
  Task: write tests for feature (depends on implementation)
  Task: review implementation (depends on tests)

  Must run sequentially (each needs previous output).
```

**Unique Output Files:**

Each Task MUST write to a **unique output file** within the session directory:

```
✅ CORRECT - Unique Files in Session Directory:
  Task: reviewer1 → $SESSION_DIR/claude-review.md
  Task: reviewer2 → $SESSION_DIR/grok-review.md
  Task: reviewer3 → $SESSION_DIR/qwen-coder-review.md

❌ WRONG - Shared File:
  Task: reviewer1 → $SESSION_DIR/review.md
  Task: reviewer2 → $SESSION_DIR/review.md (overwrites reviewer1!)
  Task: reviewer3 → $SESSION_DIR/review.md (overwrites reviewer2!)

❌ WRONG - Fixed Directory (not session-based):
  Task: reviewer1 → ai-docs/reviews/claude-review.md  # May conflict with other sessions!
```

**Wait for All Before Consolidation:**

Do NOT consolidate until ALL tasks complete:

```
✅ CORRECT - Wait for All:
  Launch: Task1, Task2, Task3, Task4 (parallel)
  Wait: All 4 complete
  Check: results.filter(r => r.status === 'fulfilled').length
  If >= 1: Dispatch dev:synthesizer (a passthrough with a verdict at N = 1);
           if any failed, also offer to retry them
  If 0:    Offer retry or abort

❌ WRONG - Premature Consolidation:
  Launch: Task1, Task2, Task3, Task4
  After 30s: Task1, Task2 done
  Consolidate: Only Task1 + Task2 (Task3, Task4 still running!)
```

---

### Pattern 3: Model Invocation via claudish MCP

**How models are invoked:**

External models are invoked via claudish MCP tools. The orchestrator calls the MCP tools
directly; no Bash invocation is needed. This is 100% reliable.

**Where the native reviewer sits — one rule.** It is a claudish `internal` slot when
claudish is a hard dependency of the dispatcher, and a separate `Agent(…)` call when
claudish is optional.

- **`/team` itself** — the `multimodel` plugin declares claudish as a dependency, so
  `internal` goes in the `models` list of the ONE `team` call and gets the same
  `require_pattern` shape check as every external. That procedure lives in this
  plugin's own `commands/team.md` (Step 2 onward) and is not restated here.
- **The code-review panels `dev` dispatches** (`/dev:dev` Phase 5, `/dev:audit`,
  `/dev:fix` Phase B) — claudish is optional there and the panel may be empty, so the
  internal reviewer is an always-present
  `Agent(subagent_type: "dev:reviewer", run_in_background: false, …)` issued in the
  same message as the `team` call, carrying the contract lines. Every code-review
  example in this skill is this case.

**For a dev-dispatched code-review panel:** write the brief to `input.md`, start the
internal Agent and the panel in one message, poll the panel to completion, then read
the externals' reviews off disk.
````
Agent(subagent_type: "dev:reviewer", run_in_background: false,
  description: "Internal code review",
  prompt: "TARGET: BRANCH\nFOCUS: code\nOUTPUT: ${SESSION_DIR}/claude-review.md\nMODELS: grok,gemini")
---
team(mode="run", path=SESSION_DIR, models=["grok", "gemini"],
  input_file=`${SESSION_DIR}/input.md`,
  require_pattern="\*\*Verdict\*\*: (PASS|CONDITIONAL|FAIL)", agent="dev:reviewer")
  // → { started: true, slots: { "grok": "01", "gemini": "02" }, ... }

team(mode="status", path=SESSION_DIR)   // until no slot has state === "RUNNING"
  // → read `${SESSION_DIR}/response-<slot>.md` for each slot
````

The `team` tool runs all models in parallel internally. **`run` does not wait and does not
return the answers** — it starts the slots and hands back the slot map. Per-model status
(`state`, `exitCode`, `outputSize`, `error.reason`) comes from a settled `status` response.
There is no `timeout` parameter any more, and passing one is silently ignored. Full procedure: `claudish:claudish-usage` → "The three-step lifecycle". Requires claudish >= 8.0.0.

**In a dev-dispatched panel the internal reviewer is never a `models` entry.** It runs as
the Agent above — on the host session, with the dev plugin loaded, so `TARGET: BRANCH`
resolves through dev's own capture script — and its return is in the dispatcher's hands.
Its file goes on REVIEWS: whether or not it carries a verdict line; the synthesizer counts
a file with no verdict as no-verdict, never as one more approval.

**`require_pattern` is what turns exit 0 into a real success check.** A slot that finished
without producing the required shape is reported FAILED (state EMPTY, reason
`shape_mismatch`) instead of counted as a success. Exit code 0 also occurs on API errors and
on a child that simply ignored the format, so without this the panel can report a verdict it
never actually received.

**For single-model delegation:**
```
create_session(model="grok", prompt=TASK_PROMPT, timeout_seconds=300)
→ channel events: session_started → tool_executing → completed/failed
→ get_output(session_id) to retrieve result
```

**Verification:**
- `team` tool: check `status.models[<slot>].state` on a SETTLED `status` response — never
  the `run` response, which returns before any model has answered
- `create_session`: The `completed` channel event confirms success; `failed` provides error details

### Correct Pattern Example

````
// ✅ CORRECT (dev-dispatched panel): the internal reviewer as its own Agent, every external in ONE team call — same message
Agent({ subagent_type: "dev:reviewer", run_in_background: false,
        description: "Internal code review",
        prompt: "TARGET: BRANCH\nFOCUS: code\nOUTPUT: ${SESSION_DIR}/claude-review.md\nMODELS: grok,gemini" })
team(mode="run", path=SESSION_DIR,
  models=["grok", "gemini"],
  input_file=`${SESSION_DIR}/input.md`,
  require_pattern="\*\*Verdict\*\*: (PASS|CONDITIONAL|FAIL)", agent="dev:reviewer")

// …then poll, then read the externals' reviews from response-<slot>.md
team(mode="status", path=SESSION_DIR)
````

```
// ❌ WRONG: the native reviewer fired into the background, its file read by nobody
Agent({ subagent_type: "dev:reviewer", run_in_background: true,
        prompt: "Review the change...\n\nWrite to: session/internal-result.md" })
```

The internal reviewer runs `run_in_background: false` so that its return — and whether its
OUTPUT file carries a `**Verdict**:` line — is in hand before the synthesizer is dispatched.
It goes on REVIEWS: either way; the synthesizer counts a file with no verdict as no-verdict,
never as approval. A background Agent's file is read by nobody until it is too late. (For
`/team` itself the correct form is the `internal` slot — `commands/team.md`, per the rule
above.)

---

### Pattern 4: Cost Estimation and Transparency

**Input/Output Token Separation:**

Provide separate estimates for input and output tokens:

```
Cost Estimation for Multi-Model Review:

Input Tokens (per model):
  - Code context: 500 lines × 1.5 = 750 tokens
  - Review instructions: 200 tokens
  - Total input per model: ~1000 tokens
  - Total input (5 models): 5,000 tokens

Output Tokens (per model):
  - Expected output: 2,000 - 4,000 tokens
  - Total output (5 models): 10,000 - 20,000 tokens

Cost Calculation (example rates):
  - Input: 5,000 tokens × $0.0001/1k = $0.0005
  - Output: 15,000 tokens × $0.0005/1k = $0.0075 (3-5x more expensive)
  - Total: $0.0080 (range: $0.0055 - $0.0105)

User Approval Gate:
  "Multi-model review will cost approximately $0.008 ($0.005 - $0.010).
   Proceed? (Yes/No)"
```

**Input Token Estimation Formula:**

```
Input Tokens = (Code Lines × 1.5) + Instruction Tokens

Why 1.5x multiplier?
  - Code lines: ~1 token per line (average)
  - Context overhead: +50% (imports, comments, whitespace)

Example:
  500 lines of code → 500 × 1.5 = 750 tokens
  + 200 instruction tokens = 950 tokens total input
```

**Output Token Estimation Formula:**

```
Output Tokens = Base Estimate + Complexity Factor

Base Estimates by Task Type:
  - Code review: 2,000 - 4,000 tokens
  - Design validation: 1,000 - 2,000 tokens
  - Architecture planning: 3,000 - 6,000 tokens
  - Bug investigation: 2,000 - 5,000 tokens

Complexity Factors:
  - Simple (< 100 lines code): Use low end of range
  - Medium (100-500 lines): Use mid-range
  - Complex (> 500 lines): Use high end of range

Example:
  400 lines of complex code → 4,000 tokens (high complexity)
  50 lines of simple code → 2,000 tokens (low complexity)
```

**Range-Based Estimates:**

Always provide a **range** (min-max), not a single number:

```
✅ CORRECT - Range:
  "Estimated cost: $0.005 - $0.010 (depends on review depth)"

❌ WRONG - Single Number:
  "Estimated cost: $0.0075"
  (User surprised when actual is $0.0095)
```

**Why Output Costs More:**

Output tokens are typically **3-5x more expensive** than input tokens:

```
Example Pricing (OpenRouter):
  - Grok: $0.50 / 1M input, $1.50 / 1M output (3x difference)
  - Gemini Flash: $0.10 / 1M input, $0.40 / 1M output (4x difference)
  - GPT-5 Codex: $1.00 / 1M input, $5.00 / 1M output (5x difference)

Impact:
  If input = 5,000 tokens, output = 15,000 tokens:
    Input cost: $0.0005
    Output cost: $0.0075 (15x higher despite only 3x more tokens)
    Total: $0.0080 (94% is output!)
```

**User Approval Before Execution:**

ALWAYS ask for user approval before expensive operations:

```
Present to user:
  "You selected 5 AI models for code review:
   - Claude Sonnet (embedded, free)
   - Grok Code Fast (external, $0.002)
   - Gemini 2.5 Flash (external, $0.001)
   - GPT-5 Codex (external, $0.004)
   - DeepSeek Coder (external, $0.001)

   Estimated total cost: $0.008 ($0.005 - $0.010)

   Proceed with multi-model review? (Yes/No)"

If user says NO:
  Offer alternatives:
    1. Use only free embedded Claude
    2. Select fewer models
    3. Cancel review

If user says YES:
  Proceed with Message 2 (parallel execution)
```

---

### Pattern 5: Auto-Consolidation Logic

**Automatic Trigger:**

Consolidation happens **automatically** as soon as the panel settles — at N = 1 as well
as N ≥ 2. `dev:synthesizer` is the only writer of the consolidated report; at N = 1 it
passes the single review through unchanged and appends the `VERDICT:` line, so the
output has one shape whatever N is:

```
✅ CORRECT - Auto-Trigger:

const results = await Promise.allSettled([task1, task2, task3, task4, task5]);
const successful = results.filter(r => r.status === 'fulfilled');
const failed = results.length - successful.length;

if (successful.length >= 1) {
  // Auto-trigger consolidation (DON'T wait for user to ask). N = 1 is a passthrough with a verdict.
  const reviewPaths = successful.map((r) => r.value.reviewFile); // one review file per slot
  const consolidated = await Agent({
    subagent_type: "dev:synthesizer",
    run_in_background: false,     // formatResults() consumes the return value
    description: "Consolidate code reviews",
    prompt: `REVIEWS: ${reviewPaths.join("\n")}
THRESHOLDS: <the three lines under 'Apply verdict thresholds' in dev:reviewer's agent file, read at dispatch time, never recalled>
OUTPUT: ${SESSION_DIR}/consolidated-review.md
Consolidate with consensus levels (unanimous / strong / majority / divergent).
Compute the verdict line from your counts against THRESHOLDS.
You are given reviews, never code. Do not review.`
  });

  if (failed > 0) {
    // An addition to the dispatch above, never a substitute for it
    notifyUser(`${failed} of ${results.length} models failed. Retry them and re-consolidate?`);
  }
  return formatResults(consolidated);
} else {
  // All failed — there is nothing to pass through
  notifyUser("All models failed. Check logs and retry?");
}

❌ WRONG - Wait for User:

const results = await Promise.allSettled([...]);
const successful = results.filter(r => r.status === 'fulfilled');

// Present results to user
notifyUser("3 reviews complete. Would you like me to consolidate them?");
// Waits for user to request consolidation...

❌ WRONG - Skip the synthesizer at N = 1:

if (successful.length >= 2) {
  await consolidate();
} else {
  notifyUser("Only 1 model succeeded. See single review or retry?");
  // The raw review carries no VERDICT: line and not the shape every other dispatcher reads
}
```

**Why Auto-Trigger:**

- Better UX (no extra user prompt needed)
- Faster workflow (no wait for user response)
- Expected behavior (user assumes consolidation is part of workflow)

**N = 1 is a passthrough, not a skip:**

Consensus levels need at least two reviews; the consolidated report does not. At N = 1
the synthesizer emits the single review unchanged — no `[CONSENSUS: …]` tags, nothing
reworded — followed by the `VERDICT:` line computed from that review's own counts against
THRESHOLDS, so the dispatcher still gets the one file its gate reads. The only dispatcher
that skips the synthesizer at N = 1 is `/dev:fix` Phase B, whose output is a vote tally,
and a single vote is its own tally:

```
if (successful.length >= 1) {
  // Dispatch dev:synthesizer: consolidation at N ≥ 2, passthrough with a verdict at N = 1
  if (successful.length < results.length) {
    // In addition, not instead
    notifyUser("Some models failed. Retry the failures and re-consolidate?");
  }
} else {
  // All failed
  notifyUser("All models failed. Check logs and retry?");
}
```

**Pass All Review File Paths:**

`dev:synthesizer` needs the path of EVERY review file, one per `REVIEWS:` line. It
reads the reviews and never the code, so the paths are all it gets:

```
Agent(
  subagent_type: "dev:synthesizer",
  run_in_background: false,
  description: "Consolidate code reviews",
  prompt: "REVIEWS: $SESSION_DIR/claude-review.md
           $SESSION_DIR/grok-review.md
           $SESSION_DIR/qwen-coder-review.md
           THRESHOLDS: <the three lines under 'Apply verdict thresholds' in
                        dev:reviewer's agent file, read at dispatch time, never
                        recalled>
           OUTPUT: $SESSION_DIR/consolidated-review.md
           Consolidate with consensus levels (unanimous / strong / majority / divergent).
           Compute the verdict line from your counts against THRESHOLDS.
           You are given reviews, never code. Do not review."
)
```

**Don't Inline Full Reviews:**

```
❌ WRONG - Inline Reviews (context pollution):
  Prompt: "Consolidate these reviews:

           Claude Review:
           [500 lines of review content]

           Grok Review:
           [500 lines of review content]

           Qwen Review:
           [500 lines of review content]"

✅ CORRECT - File Paths in Session Directory:
  prompt: "REVIEWS: $SESSION_DIR/claude-review.md
           $SESSION_DIR/grok-review.md
           $SESSION_DIR/qwen-coder-review.md
           THRESHOLDS: ...
           OUTPUT: $SESSION_DIR/consolidated-review.md
           ..."
```

---

### Pattern 6: Consensus Analysis

**Consensus Levels:**

Classify issues by how many models flagged them:

```
Consensus Levels (for N models):

UNANIMOUS (100% agreement):
  - All N models flagged this issue
  - VERY HIGH confidence
  - MUST FIX priority

STRONG CONSENSUS (67-99% agreement):
  - Most models flagged this issue (⌈2N/3⌉ to N-1)
  - HIGH confidence
  - RECOMMENDED priority

MAJORITY (50-66% agreement):
  - Half or more models flagged this issue (⌈N/2⌉ to ⌈2N/3⌉-1)
  - MEDIUM confidence
  - CONSIDER priority

DIVERGENT (< 50% agreement):
  - Only 1-2 models flagged this issue
  - LOW confidence
  - OPTIONAL priority (may be model-specific perspective)
```

**Example: 5 Models**

```
Issue Flagged By:              Consensus Level:    Priority:
─────────────────────────────────────────────────────────────
All 5 models                   UNANIMOUS (100%)    MUST FIX
4 models                       STRONG (80%)        RECOMMENDED
3 models                       MAJORITY (60%)      CONSIDER
2 models                       DIVERGENT (40%)     OPTIONAL
1 model                        DIVERGENT (20%)     OPTIONAL
```

**Keyword-Based Matching (v1.0):**

Simple consensus analysis using keyword matching:

```
Algorithm:

1. Extract issues from each review
2. For each unique issue:
   a. Identify keywords (e.g., "SQL injection", "input validation")
   b. Check which other reviews mention same keywords
   c. Count models that flagged this issue
   d. Assign consensus level

Example:

Claude Review: "Missing input validation on POST /api/users"
Grok Review: "Input validation absent in user creation endpoint"
Gemini Review: "No validation for user POST endpoint"

Keywords: ["input validation", "POST", "/api/users", "user"]
Match: All 3 reviews mention these keywords
Consensus: UNANIMOUS (3/3 = 100%)
```

**Model Agreement Matrix:**

Show which models agree on which issues:

```
Issue Matrix:

Issue                             Claude  Grok  Gemini  GPT-5  DeepSeek  Consensus
──────────────────────────────────────────────────────────────────────────────────
SQL injection in search              ✓      ✓     ✓       ✓       ✓      UNANIMOUS
Missing input validation

…(truncated)