Designing Agents

Designing Agents

Expert guidance for choosing architecture patterns and context strategies for agentic AI systems.

Before Designing

Gather this context (ask if not provided):

1. Task shape — Is the workflow predictable/structured or open-ended? How many steps? Can steps run in parallel?
2. Constraints — Latency budget? Cost sensitivity? Need for human approval at checkpoints?
3. Context pressure — How much information must the agent reason over? Will tasks exceed a single context window?
4. Existing infrastructure — What models, tools, and frameworks are already in play?

Core Principle: Context Is Everything

Effective agent design boils down to context management. Models degrade as context grows — every token depletes an "attention budget." The central design question is:

How do you fill the context window with just the right information for the next step?

All patterns below are strategies for managing this constraint.

Start Simple, Add Complexity Only When Measured

The most successful agent implementations use simple, composable patterns — not complex frameworks. Start with the simplest approach that could work:

1. Single LLM call with retrieval and examples — sufficient for many tasks
2. Prompt chaining — decompose into fixed sequential steps
3. Single agent with tools — when the task needs autonomous tool use
4. Multi-agent — only when a single agent demonstrably fails

Add complexity only when it demonstrably improves outcomes. Measure first.

Pattern Selection

Decision Framework

For detailed pattern descriptions and trade-offs: See references/design-patterns.md

Single Agent vs Multi-Agent

This is the field's biggest live disagreement, and the resolution is about task shape — not a universal winner.

• Multi-agent wins on parallelizable, breadth-first, read-heavy tasks (research, multi-source search) where subagents explore in isolated context windows and return compressed summaries. Anthropic reported an orchestrator-worker research system (Opus lead + Sonnet subagents) beat single-agent Opus by 90.2% on its internal research eval — but multi-agent systems burn ~15× the tokens of a chat (single agents ~4×), so reserve them for high-value work. In that analysis, token usage alone explained ~80% of performance variance (with tool-call count and model choice the other factors) — so spend the budget on giving each agent enough room and a capable model before adding more agents.
• Single-threaded wins on tasks needing shared context and tightly coupled writes (most coding). Cognition's "Don't Build Multi-Agents" argues parallel agents fragment context and make conflicting decisions; their rule is to share context everywhere and keep writes single-threaded. Anthropic agrees coding has fewer truly parallelizable subtasks and that agents aren't yet great at real-time coordination.

Rule of thumb: let extra agents add intelligence (parallel reading, diverse perspectives), not conflicting actions. Keep writes on one thread.

Dynamic Workflow Composition

The patterns above are architectures — how to structure the system. When agents orchestrate other agents at runtime, deterministic control flow (fan-out, conditionals, loops) decides what runs while each spawned agent does the open-ended reasoning. These composable recipes cover that layer:

• Classify-and-act — a classifier routes each input (or labels each output) to different handlers
• Fan-out-and-synthesize — split into independent units, run an agent on each clean context, merge at a barrier
• Adversarial verification — a separate agent refutes each output against a rubric to kill false positives
• Generate-and-filter — overproduce candidates, then prune by rubric/verification and dedupe
• Tournament — N agents attempt the same task with different approaches; a judge picks a winner
• Loop-until-done — spawn until a measured stop condition (no new findings, errors cleared), not a fixed count

They combine — e.g. fan out → adversarially verify each → loop until dry → synthesize. For descriptions, barrier semantics, examples, and how to trigger them in Claude Code: See references/workflow-patterns.md

Context Engineering Strategies

These strategies apply across all patterns. Mix and match based on your context pressure.

Give Agents a Computer

Agents benefit from OS-level primitives — filesystem and shell. The filesystem gives persistent context. The shell lets agents run utilities, CLIs, scripts, or code they write.

The fundamental agent abstraction is the CLI. Think of it as "AI for your operating system."

Multi-Layer Action Space

Keep the set of tools loaded up front small (a handful up to ~20) — this is what the agent sees at any moment, not a cap on total tools. With Tool Search and Programmatic Tool Calling you can expose dozens of tools (75-tool agents work well) while loading only what the current step needs. Push actions to the computer:

Layer 1: Tool calls (small set of atomic tools — bash, file read/write, search)
Layer 2: Shell utilities and CLIs (progressively disclosed via --help)
Layer 3: Code the agent writes and executes (chains many actions, saves tokens)

Writing and executing code chains many actions without processing intermediate tool results — significant token savings.

Progressive Disclosure

Show only essential information upfront, reveal details on demand:

• Tool definitions — index and retrieve on demand, not all upfront. The Tool Search Tool is the shipped mechanism: mark tools with defer_loading so their definitions stay out of the prompt, then the agent retrieves them by query at runtime — preserving the prompt cache while exposing dozens of tools. Code-as-filesystem and Programmatic Tool Calling are complementary mechanisms for the same goal.
• CLI tools — list available utilities in instructions, agent reads --help when needed
• MCP servers — sync descriptions to a folder, agent reads full spec only if needed
• Skills — YAML frontmatter loaded into context, full SKILL.md read only when triggered

Offload Context to Filesystem

Write information out of the context window into files:

• Old tool results → files (read back only if needed)
• Agent plans → plan file (read periodically to reinforce objectives)
• Agent trajectories → log files (available for review without consuming context)

Apply summarization only after offloading has diminishing returns. Summarization loses information; offloading preserves it.

Cache Context

Prompt caching lets agents resume from a prefix without re-processing. Cache hit rate is the most important metric for production agents — a higher-capacity model with caching can be cheaper than a lower-cost model without it.

Design context mutations (adding/removing blocks) with cache-friendliness in mind.

Isolate Context with Sub-Agents

Delegate tasks to sub-agents with isolated context windows, tools, and instructions:

• Parallelizable tasks — fan out to independent sub-agents (e.g., code review checking different issues)
• Long-running tasks — context lives in files, progress communicated via git history
• Map-reduce — split work, combine results (e.g., lint rule updates across many files)

The Ralph Loop

For tasks that exceed a single context window, use two specialized agent types:

Initializer agent (runs once):

• Sets up the environment (dev server scripts, git repo)
• Creates a structured feature/task list (JSON, not markdown — less prone to overwrites)
• Writes progress tracking file
• Commits initial state to git

Worker agent (loops until done):

1. Reads git log + progress file to understand current state
2. Runs setup scripts (e.g., init.sh) to restore environment
3. Verifies existing work still passes before starting new work
4. Tackles one task from the plan
5. Self-verifies with tests before marking complete
6. Commits progress with descriptive messages
7. Updates progress file

Critical rules:

• Agents may only update the passes field on tasks — never remove or edit tests
• Use JSON for state files (structured, machine-parseable, less corruption risk)
• Each session starts by reading state, not by asking the user what to do
• End each session in a production-ready state

For implementation details and failure modes: See references/long-running-harness.md

Evolve Context Over Time

Agents can learn from experience by reflecting over past sessions:

• Task-specific prompts — score trajectories, reflect on failures, propose prompt variants
• Memory — distill sessions into entries, reflect across entries, update persistent context
• Skills — reflect over trajectories to distill reusable procedures, save as new skills

For detailed context strategies: See references/context-engineering.md

Agent Design Workflow

Copy this checklist and track progress:

Agent Design Progress:
- [ ] Step 1: Define task shape (structured vs open-ended, step count, parallelism)
- [ ] Step 2: Start with simplest viable pattern
- [ ] Step 3: Build single-agent prototype with core tools
- [ ] Step 4: Test on real prompts — identify where it fails
- [ ] Step 5: Apply context strategies (offloading, progressive disclosure, caching)
- [ ] Step 6: If single agent insufficient, decompose into sub-agents
- [ ] Step 7: Add human-in-the-loop checkpoints for high-stakes decisions
- [ ] Step 8: Measure: latency, cost, accuracy, context utilization
- [ ] Step 9: Iterate — add complexity only where measurements justify it

Anti-Patterns

• Premature multi-agent — Adding agents before proving a single agent can't handle it. Multi-agent fits parallelizable, read-heavy work with compressed summaries — not tasks needing shared context and coupled writes (see Single Agent vs Multi-Agent)
• Framework over understanding — Using complex frameworks without understanding the underlying prompts and responses
• Monolithic prompt — One agent with 50 tools and a giant system prompt instead of decomposing
• Context stuffing — Loading everything into context upfront instead of progressive disclosure
• Summarize-first — Compressing context before trying to offload it to files
• No caching strategy — Running agents without prompt caching, making them cost-prohibitive
• Unverified loops — Long-running agent loops without checkpoints or exit conditions

Output Format

When designing an agent system, deliver:

1. Architecture diagram — which agents, their responsibilities, how they communicate
2. Pattern justification — why this pattern fits the task shape and constraints
3. Context strategy — how context is managed (disclosure, offloading, caching, isolation)
4. Tool inventory — what tools each agent needs (keep it small)
5. Human checkpoints — where human review is needed
6. Exit conditions — how loops terminate, how the system knows it's done

Related Skills

• designing-agent-tools: For tool-level design (schemas, descriptions, responses, evaluation)

Sources

• Building effective agents
• Programmatic tool calling
• Advanced tool use
• Orchestrate subagents at scale with dynamic workflows
• Introducing dynamic workflows in Claude Code
• How we built our multi-agent research system
• Don't Build Multi-Agents — Cognition
• Reasoning-loop papers: ReAct (arXiv:2210.03629), ReWOO (arXiv:2305.18323), Reflexion (arXiv:2303.11366), Tree-of-Thoughts (arXiv:2305.10601)