Iterative Retrieval

Iterative Retrieval Pattern

Solve the subagent context problem: agents don't know what they need until they start looking.

The Problem

When delegating to subagents, you face a dilemma:

• Send everything: Context overflow, wasted tokens, confused agent
• Send nothing: Agent lacks context, produces poor results
• Guess what's needed: Often wrong, misses critical files

The Solution: 4-Phase Iterative Loop

Phase 1: DISPATCH

Send a broad initial query to gather candidate context:

Search for files related to: [topic]
Keywords: [initial keywords]
File patterns: [likely patterns]
Return: File paths with relevance scores

Phase 2: EVALUATE

Score each result for relevance (0.0 to 1.0):

| File | Relevance | Reason |
|------|-----------|--------|
| src/auth/login.ts | 0.9 | Core auth logic |
| src/utils/hash.ts | 0.7 | Used by auth |
| src/config.ts | 0.3 | Has auth config section |
| src/routes.ts | 0.2 | Only references auth path |

Identify gaps:

• What terminology does this codebase use? (update search terms)
• What patterns are referenced but not found?
• What dependencies need exploration?

Phase 3: REFINE

Update search criteria based on evaluation:

Updated keywords: [refined keywords from codebase terminology]
New file patterns: [patterns discovered during evaluation]
Specific files to read: [high-relevance files]
Gaps to fill: [what's still missing]

Phase 4: LOOP

Repeat phases 1-3 with refined criteria.

Stopping Criteria:

• 3+ files with relevance >= 0.7
• No critical gaps identified
• OR max 3 iterations reached (proceed with best available)

Example: Finding Auth Implementation

Iteration 1:

• DISPATCH: Search for "auth", "login", "session"
• EVALUATE: Found middleware but not the auth service
• REFINE: Codebase uses "identity" not "auth"

Iteration 2:

• DISPATCH: Search for "identity", "credential", "token"
• EVALUATE: Found IdentityService, TokenManager, SessionStore
• Result: 5 files with relevance >= 0.8 — stop and proceed

Implementation Pattern

async function iterativeRetrieve(topic: string, maxIterations = 3) {
  let keywords = extractKeywords(topic);
  let context: FileContext[] = [];

  for (let i = 0; i < maxIterations; i++) {
    // DISPATCH
    const candidates = await searchCodebase(keywords);

    // EVALUATE
    const scored = await scoreRelevance(candidates, topic);
    context = mergeContext(context, scored.filter(s => s.relevance >= 0.5));

    // Check stopping criteria
    const highRelevance = context.filter(c => c.relevance >= 0.7);
    if (highRelevance.length >= 3) break;

    // REFINE
    keywords = refineKeywords(keywords, scored);
  }

  return context;
}

When to Use

• Delegating research to a subagent
• Exploring unfamiliar codebases
• Gathering context for code review
• Building context for architectural analysis

When NOT to Use

• You already know exactly what files are needed
• The task is in a single file
• The codebase is small enough to read entirely