Brainstorming

Brainstorming v2.0: Resilient Multi-Model Planning

Turn ideas into validated designs through collaborative AI dialogue with resilient model execution and confidence-based validation.

Overview

This skill improves upon v1.0 by addressing critical reliability gaps:

Key v2.0 Improvements:

• No AskUserQuestion dependency: Uses Task + Tasks for structured interaction
• Fallback chains: 3+ models per role ensures completion even if some fail
• Explicit parallelism: Documented Task call patterns for parallel execution
• Defined algorithms: Consensus matrix and confidence scoring are mathematically specified

When to Use

Use this skill BEFORE implementing any feature:

• "Design a user authentication system"
• "Brainstorm approaches for API rate limiting"
• "Plan architecture for a new dashboard feature"
• "Evaluate options for real-time data synchronization"

Prerequisites

Required Setup

# 1. Install required skills
/plugin marketplace add MadAppGang/magus
skill install superpowers:using-git-worktrees
skill install superpowers:writing-plans

# 2. Verify OpenRouter access (for multi-model)
export OPENROUTER_API_KEY=your-key

# 3. Configure models in ~/.claude/settings.json
# Use model IDs from the live catalog (list_models)
{
  "brainstorming": {
    "primary_model": "internal",
    "explorer_models": [
      "(model resolved from list_models)"
    ]
  }
}

Model Requirements

Workflow

Phase 0: Problem Analysis (200-300 words)

Objective: Capture problem scope, constraints, and success criteria

How to Ask Users (Without AskUserQuestion):

// Pattern: Use Tasks to track questions, Read/Write for presentation

// 1. Write question to temp file
await Write({
  file_path: "/tmp/brainstorm-q1.md",
  content: `## Question 1 of 3

**What are the main constraints or requirements for this feature?**

Please respond with:
- Functional requirements (what it must do)
- Non-functional requirements (performance, scale)
- Any existing dependencies or integrations
`
});

// 2. Present file and wait for user response
// User reads file, provides input via conversation

// 3. Summarize understanding
const problemSummary = await Write({
  file_path: "/tmp/brainstorm-problem.md",
  content: `## Problem Understanding

**Constraints identified:**
- [From user response]

**Success criteria:**
- [Measurable outcomes]

**Scope boundaries:**
- [What's in/out]

---

**Does this accurately capture the problem?** (Reply "yes" to proceed or clarify)
`
});

Gate Type: USER_GATE (requires confirmation)

Phase 1: Parallel Exploration

Objective: Generate diverse solutions via multi-model brainstorming

Fallback Chain Implementation:

interface ModelResult {
  model: string;
  success: boolean;
  output?: string;
  error?: string;
}

async function exploreWithFallback(
  prompt: string,
  role: "explorer"
): Promise<ModelResult> {
  // Use model IDs from `list_models` — pick current models for the task
  //
  const fallbackModels = role === "explorer"
    ? [/* models resolved from `list_models` for code */]
    : ["internal" /* plus models from list_models */];

  for (const model of fallbackModels) {
    try {
      const result = await Agent({
        model: model,
        prompt: prompt,
        timeout_ms: 120000  // 2 minute timeout
      });

      return { model, success: true, output: result };
    } catch (error) {
      console.warn(`Model ${model} failed:`, error.message);
      continue;  // Try next in chain
    }
  }

  throw new Error(`All models in fallback chain failed`);
}

Parallel Execution Pattern:

// WRONG: Sequential (slow)
// const result1 = await Agent({ model: "grok", ... });
// const result2 = await Agent({ model: "gemini", ... });
// const result3 = await Agent({ model: "sonnet", ... });

// CORRECT: Parallel (3-5x faster)
// Use model IDs from `list_models` (live catalog), filtered for code
const [result1, result2, result3] = await Promise.all([
  Agent({
    model: "(fast_coding role — resolve from list_models)",
    prompt: generateExplorerPrompt(problem, "fast_code")
  }),
  Agent({
    model: "(reasoning role — resolve from list_models)",
    prompt: generateExplorerPrompt(problem, "balanced")
  }),
  Agent({
    model: "(model resolved from list_models)",
    prompt: generateExplorerPrompt(problem, "thorough")
  })
]);

// Handle partial failures
const results = [result1, result2, result3].filter(r => r.success);
if (results.length === 0) {
  throw new Error("All exploration models failed");
}

Output Format:

## Approach: [Name]

**Model**: [Which model generated this]
**Approach Type**: [architecture/algorithm/pattern]
**Summary**: 2-3 sentences

**Key Components**:
1. Component A
2. Component B
3. Component C

**Trade-offs**:
- + Advantage
- - Disadvantage

**Confidence**: [Model's confidence 0-100]

Gate Type: AUTO_GATE (automatic consolidation)

Phase 2: Consensus Analysis

Objective: Identify strongest ideas using defined algorithms

Consensus Matrix Algorithm:

1. Clustering: Group approaches by semantic similarity (vector embedding + clustering)
2. Scoring: Count model agreement per cluster
3. Classification: UNANIMOUS (3/3), STRONG (2/3), DIVERGENT (1/3)
4. Confidence: Weighted average of model confidences + agreement bonus

Consensus Matrix Calculation:

interface Approach {
  id: string;
  name: string;
  summary: string;
  model: string;  // Which model proposed
  modelConfidence: number;  // 0-100
  embedding: number[];  // For clustering
}

interface Cluster {
  approaches: Approach[];
  representative: Approach;  // Most complete
  agreementScore: number;  // 0-1
  confidenceScore: number;  // 0-100
  consensusLevel: "UNANIMOUS" | "STRONG" | "DIVERGENT";
}

function calculateConsensus(approaches: Approach[]): Cluster[] {
  // Step 1: Cluster by semantic similarity
  const clusters = clusterByEmbedding(approaches, threshold: 0.85);

  // Step 2: Calculate metrics per cluster
  return clusters.map(cluster => {
    const models = cluster.map(a => a.model);
    const modelCount = new Set(models).size;
    const totalModels = approaches.length;

    // Agreement: proportion of models that have an approach in this cluster
    const agreementScore = modelCount / totalModels;

    // Confidence: weighted average + agreement bonus
    const baseConfidence = cluster
      .map(a => a.modelConfidence)
      .reduce((a, b) => a + b, 0) / cluster.length;

    const confidenceScore = Math.min(100,
      baseConfidence + (agreementScore * 20)  // +20% for agreement
    );

    // Consensus classification
    const consensusLevel = agreementScore >= 0.9 ? "UNANIMOUS" :
                          agreementScore >= 0.5 ? "STRONG" :
                          "DIVERGENT";

    return {
      approaches: cluster,
      representative: cluster.reduce((best, current) =>
        current.modelConfidence > best.modelConfidence ? current : best
      ),
      agreementScore,
      confidenceScore: Math.round(confidenceScore),
      consensusLevel
    };
  }).sort((a, b) => b.confidenceScore - a.confidenceScore);
}

Confidence Scoring Formula:

Confidence = Base + AgreementBonus - DiversityPenalty

Where:
  Base = average(model confidences in cluster)
  AgreementBonus = (unique_models / total_models) * 20
  DiversityPenalty = (1 - similarity_coefficient) * 10

Example:
  3 models propose similar approaches
  Base = (92 + 88 + 95) / 3 = 91.7
  AgreementBonus = (3/3) * 20 = 20
  DiversityPenalty = (1 - 0.9) * 10 = 1
  Confidence = 91.7 + 20 - 1 = 110.7 -> capped at 100
  Final: 97%

Consensus Matrix Example:

Gate Type: AUTO_GATE (automatic scoring)

Phase 3: User Selection

Objective: Present top approaches for user decision

Presentation Pattern:

async function presentApproaches(clusters: Cluster[]): Promise<string> {
  const topClusters = clusters.slice(0, 5);  // Top 5

  let presentation = `## Top Approaches\n\n`;

  for (const [index, cluster] of topClusters.entries()) {
    const approach = cluster.representative;

    presentation += `### ${String.fromCharCode(65 + index)}: ${approach.name} [${cluster.consensusLevel}]

**Summary**: ${approach.summary}

**Confidence**: ${cluster.confidenceScore}% (${cluster.approaches.length} model(s) agree)

**Pros**:
${cluster.approaches.map(a => `- ${a.summary}`).join("\n")}

**Cons**:
${cluster.approaches.map(a => `- Potential issue from ${a.model}`).join("\n")}

---
`;
  }

  presentation += `
## Your Choice

Which approach best fits your requirements?

- **A**: Select approach A
- **B**: Select approach B
- **C**: Select approach C
- **D**: Combine elements from multiple
- **E**: Explore alternatives (return to Phase 1)
`;

  // Save for user review
  await Write({
    file_path: "/tmp/brainstorm-approaches.md",
    content: presentation
  });

  return presentation;
}

Gate Type: USER_GATE (selection via conversation)

Phase 4: Detailed Planning

Objective: Elaborate selected approach into actionable sections

Confidence-Based Gating:

Section Template:

## [Section Name] (Confidence: XX%)

**Approach**: [Selected approach]

**Implementation Details**:
[200-300 words]

**Assumptions**:
- Assumption 1
- Assumption 2

**Confidence Calculation**:
- Technical feasibility: XX%
- Edge cases covered: XX%
- Team capability: XX%
- Overall: XX%

**Status**: [AUTO_GATE|PENDING_USER] - [Reason]

Gate Type: MIXED_GATE (adaptive)

Phase 5: Plan Validation

Objective: Final review before implementation

Validation Checklist:

## Plan Validation

**Problem**: [Summary]
**Approach**: [Selected]
**Confidence**: [Overall]

### Checklist

- [ ] Problem scope accurately captured
- [ ] Chosen approach matches expectations
- [ ] Module structure aligns with capabilities
- [ ] Technical constraints addressed
- [ ] Success criteria measurable

### Next Steps

**To proceed**:
1. Reply "approve" to finalize
2. Reply "revise [section]" to modify
3. Reply "restart" to begin fresh

**Final decision?**

Gate Type: USER_GATE (explicit approval)

Complete Parallel Execution Example

// Complete Phase 1 parallel exploration
async function runParallelExploration(problem: string): Promise<Approach[]> {
  // Use model IDs from `list_models` (live catalog), filtered for code
  //
  const explorerModels = [
    // fast_coding role — fast, code-focused
    // reasoning role — balanced, creative
    // another model from list_models — thorough
  ];

  const prompts = explorerModels.map(model =>
    `Generate 5 implementation approaches for: ${problem}

For each approach provide:
1. Name (2-3 words)
2. One-sentence summary
3. Key components (bullet points)
4. Trade-offs (+/-)
5. Your confidence (0-100)

Format as JSON array.`
  );

  // LAUNCH ALL MODELS IN PARALLEL
  const taskPromises = explorerModels.map((model, index) =>
    Agent({
      model: model,
      prompt: prompts[index],
      timeout_ms: 120000,
      max_turns: 1
    }).catch(error => ({
      model,
      success: false,
      error: error.message
    }))
  );

  // WAIT FOR ALL TO COMPLETE
  const results = await Promise.all(taskPromises);

  // CONSOLIDATE SUCCESSFUL RESULTS
  const approaches: Approach[] = results
    .filter(r => r.success)
    .flatMap(r => parseApproaches(r.output));

  // HANDLE PARTIAL FAILURES
  if (approaches.length < 5) {
    console.warn(`Only got ${approaches.length} approaches from ${explorerModels.length} models`);
    if (approaches.length === 0) {
      throw new Error("All models failed");
    }
  }

  return approaches;
}

Troubleshooting

Model Failures

Recovery Pattern:

async function resilientExploration(problem: string): Promise<Approach[]> {
  let attempts = 0;
  const maxAttempts = 3;

  while (attempts < maxAttempts) {
    try {
      return await runParallelExploration(problem);
    } catch (error) {
      attempts++;
      if (attempts === maxAttempts) throw error;

      // Exponential backoff
      await new Promise(r => setTimeout(r, Math.pow(2, attempts) * 1000));
    }
  }
}

Consensus Issues

User Interaction Issues

Configuration

Environment Variables

# Required for multi-model
OPENROUTER_API_KEY=...

# Optional
BRAINSTORM_TIMEOUT_MS=120000
BRAINSTORM_MAX_RETRIES=2
BRAINSTORM_MIN_MODELS=2  # Minimum models for valid consensus

Model Configuration

{
  "brainstorming": {
    "primary": ["internal"],
    "explorers": {
      // Use model IDs from the live catalog (list_models)
      "primary_chain": ["(another model from list_models)", "(another model from list_models)"],
      "fallback_chain": ["(model from list_models)", "(model from list_models)"]
    },
    "thresholds": {
      "auto_gate": 95,
      "confirm_gate": 60
    }
  }
}