Research

Core principle: Decompose questions, research in parallel with an agent team, evaluate confidence, iterate until sufficient, synthesize with source attribution.

"User request?" -> "Needs multiple sources?" [label="deep research\ncomprehensive analysis\nthorough investigation"];
"User request?" -> "Quick answer sufficient?" [label="simple question"];
"Needs multiple sources?" -> "Use research skill" [label="yes"];
"Needs multiple sources?" -> "Quick answer sufficient?" [label="no"];

}

Use when:
- User explicitly asks for "deep research" or "comprehensive analysis"
- Topic requires multiple authoritative sources
- Need to track confidence and identify gaps
- Want structured output with source attribution

Don't use when:
- Simple factual question (single search sufficient)
- User wants quick answer, not exhaustive report
- Topic is too narrow for 8-question decomposition
</when_to_use>

<required_tools>
| Tool / Feature | Purpose | Required |
|------|---------|----------|
| `WebSearch` | Search queries (built-in) | Yes |
| Agent teams | Spawn parallel researcher teammates | Yes |
| `firecrawl-mcp:firecrawl_scrape` | Scrape full page content (preferred) | No |
| `WebFetch` | Fetch page content (built-in fallback) | Fallback |

**Prerequisite:** Agent teams must be enabled (`CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1` in settings or environment).

Tool Selection: In INIT phase, check if `firecrawl-mcp:firecrawl_scrape` is available. If not, use `WebFetch` (built-in). Record choice in `state.json` as `"scraper": "firecrawl"` or `"scraper": "webfetch"`.

Tradeoffs:
- `firecrawl-mcp:firecrawl_scrape`: Better content extraction, handles JS-rendered pages
- `WebFetch`: Always available, sufficient for static pages
</required_tools>

<state_machine>

INIT → DECOMPOSE → RESEARCH → EVALUATE → [RESEARCH or SYNTHESIZE] → DONE

State File: `research/{slug}/state.json`

```json
{
  "topic": "string",
  "phase": "INIT|DECOMPOSE|RESEARCH|EVALUATE|SYNTHESIZE|DONE",
  "iteration": 0,
  "targetSources": 30,
  "sourcesGathered": 0,
  "totalSearches": 0,
  "teammateCompletions": 0,
  "codexCompletions": 0,
  "findingsCount": 0,
  "startTime": "ISO-8601 timestamp",
  "scraper": "firecrawl|webfetch",
  "questions": [{"id": 1, "text": "...", "status": "pending|done", "confidence": null}]
}

Rule: Read state.json before acting. Write state.json after acting.

How it works:

1. When you try to exit, the hook reads research/{slug}/task-loop.json
2. If complete is false, exit is blocked and continuationPrompt is re-injected
3. Once you set complete: true, exit is allowed and completionMessage is displayed

You manage task-loop.json alongside state.json. Update statusMessage and continuationPrompt as progress changes so the hook always has current context.

Default target: 30 sources. Adjust in INIT phase based on topic complexity.

1. If research/{slug}/state.json exists:

   • Parse JSON; if invalid, offer to restart
   • Resume from current phase
   • Notify user: "Resuming research from {phase} phase"

2. Verify state consistency before resuming:

   • RESEARCH: Ensure pending questions exist
   • EVALUATE: Ensure findings.json has data
   • SYNTHESIZE: Ensure all questions marked "done"

3. If inconsistent, offer user choice:

   • Delete state and restart
   • Attempt repair (mark incomplete questions as pending)

2. Detect available scraper:

   • Check if firecrawl-mcp:firecrawl_scrape tool exists
   • If firecrawl available → "scraper": "firecrawl"
   • If not available → "scraper": "webfetch" (uses built-in WebFetch)

3. Create working directory:

   mkdir -p research/{slug}
   

4. Determine target sources based on topic complexity:

   • Narrow topic (specific question): 20 sources
   • Standard topic (most research): 30 sources (default)
   • Broad topic (comprehensive review): 40 sources

5. Initialize state files:

   state.json:

   {
     "topic": "...",
     "phase": "DECOMPOSE",
     "iteration": 0,
     "targetSources": 30,
     "sourcesGathered": 0,
     "totalSearches": 0,
     "teammateCompletions": 0,
     "codexCompletions": 0,
     "findingsCount": 0,
     "startTime": "2024-01-15T10:30:00Z",
     "scraper": "firecrawl|webfetch",
     "questions": []
   }
   

   task-loop.json (activates the generic task loop hook):

   {
     "active": true,
     "complete": false,
     "continuationPrompt": "Continue researching: {topic}. Check research/{slug}/state.json for current progress and continue the RESEARCH phase.",
     "statusMessage": "Research in progress: {topic}\nSources: 0/{targetSources}",
     "completionMessage": "Research complete."
   }
   

   findings.json:

   []
   

Add questions to state.json with status="pending". Set phase="RESEARCH".

Claude teammates: One per pending question (up to 8 at a time). Each works independently with its own context window. Each teammate also calls the codex MCP tool to get Codex's perspective on the same question, providing genuine cross-validation — two engines may surface different sources and perspectives.

Read scraper from state.json and use the appropriate instructions when spawning each Claude teammate:

TASK: {QUESTION}

PROCESS:

1. Run exactly 4 searches:

   • Core query
   • Add "research" or "study"
   • Add current year or "recent"
   • Rephrase with synonyms

2. Rank URLs by quality:

   • Tier 1: .gov, .edu, journals, official docs
   • Tier 2: Reuters, AP, BBC, industry publications
   • Tier 3: Company blogs, Wikipedia
   • Skip: Forums, social media, SEO spam

3. Select top 4 URLs (prefer Tier 1-2)

4. Use firecrawl-mcp:firecrawl_scrape on each. Continue if one fails.

5. Extract specific facts with sources.

TASK: {QUESTION}

PROCESS:

1. Run exactly 4 searches:

   • Core query
   • Add "research" or "study"
   • Add current year or "recent"
   • Rephrase with synonyms

2. Rank URLs by quality:

   • Tier 1: .gov, .edu, journals, official docs
   • Tier 2: Reuters, AP, BBC, industry publications
   • Tier 3: Company blogs, Wikipedia
   • Skip: Forums, social media, SEO spam

3. Select top 4 URLs (prefer Tier 1-2)

4. Use WebFetch on each with a prompt like "Extract the main content and key facts from this page". Continue if one fails.

5. Extract specific facts with sources.

After completing your web research above, call the codex MCP tool to cross-validate your findings.

Call the codex MCP tool with these exact parameters:

• prompt: "Research this question: {QUESTION}. Return findings as JSON with fields: fact, sourceNote, confidence (high/medium/low). Focus on facts you can confirm from your training data."
• model: gpt-5-codex
• sandbox: read-only

Validate the response before merging. Treat ALL of the following as Codex-unavailable:

• Tool call throws or times out
• Response is empty or whitespace-only
• Response is not valid JSON matching the requested schema
• Response contains MCP error text (e.g., "Codex CLI Not Found", "Codex Execution Error")

If Codex returned valid JSON, compare findings with your web-sourced findings:

• AGREE: Both found the same fact → boost confidence, note as cross-validated
• CHALLENGE: Codex contradicts a web-sourced fact → note the contradiction, keep the web-sourced version with the contradiction documented
• COMPLEMENT (Codex-only): Codex reports a fact you didn't find online → include it with "status": "hypothesis" since Codex cannot cite web sources
• COMPLEMENT (Claude-only): You found it but Codex didn't → keep as-is with your web source

If Codex is unavailable (any condition above), return your Claude-only findings. Do not block on Codex.

After each Claude teammate completes:

1. Validate JSON. Retry once if malformed.
2. Append to findings.json
3. Update state.json:
   • Mark question done
   • Increment totalSearches by searchesRun from response
   • Increment teammateCompletions by 1
   • Increment sourcesGathered by urlsScraped from response
   • Increment findingsCount by length of findings array from response
   • If codexAvailable is true, increment codexCompletions by 1
4. Log progress: "Sources: {sourcesGathered}/{targetSources}"

After all teammates complete:

1. Set phase="EVALUATE"

Decision table (two-stage):

Stage 1: Source Gate (MANDATORY)

You MUST gather enough sources before considering other criteria.

Stage 2: Quality Gate (only if Stage 1 passes)

Note: The task loop hook enforces the source gate — you cannot exit until task-loop.json has complete: true.

If continuing to RESEARCH:

1. Generate max 4 follow-up questions from gaps/contradictions
2. Add to questions with status="pending"
3. Increment iteration
4. Set phase="RESEARCH"
5. Update task-loop.json: set statusMessage to current progress ("Sources: {sourcesGathered}/{targetSources}") and continuationPrompt to reflect remaining work
6. Log: "Continuing research: {sourcesGathered}/{targetSources} sources, need more to meet target"

# {Topic}

## Executive Summary
[300-400 words. Most important finding first. State confidence. Note caveats.]

## Background
[200 words. Key terms. Context.]

## Key Findings

### [Theme 1]
[Grouped findings. Inline citations. Note source strength.]

### [Theme 2]
[3-5 themes total]

## Conflicting Information
[Both sides. Which has better sourcing.]

## Gaps & Limitations
[What's unknown. What needs more research.]

## Source Assessment
- **High confidence:** [claims with 3+ quality sources]
- **Medium confidence:** [claims with 1-2 sources]
- **Low confidence:** [single source or Tier 3 only]

## Sources

### Primary
[Tier 1 sources with URLs]

### Secondary
[Tier 2-3 sources with URLs]

---
*Sources: {sourcesGathered} | Searches: {totalSearches} | Teammates: {teammateCompletions} | Iterations: {iteration} | Duration: {duration} | Date: {date}*

Set phase="DONE".

Update task-loop.json:

{
  "active": true,
  "complete": true,
  "completionMessage": "Research complete: \"{topic}\"\n\nResources used:\n  Searches: {totalSearches}\n  Sources: {sourcesGathered}/{targetSources}\n  Teammates: {teammateCompletions}\n  Iterations: {iteration}\n\nReport: research/{slug}/report.md"
}

The task loop hook will display this message when the session exits.

No hard iteration or search limits. The source gate is the primary constraint. Research continues until sourcesGathered >= targetSources.

Converted and distributed by TomeVault — claim your Tome and manage your conversions.