Long Context
A large context window lets you send a lot; it does not make sending a lot free or accurate. Count first, structure the input, cache what repeats, and actively manage context as it grows.
When to Use
- Cross-document reasoning over a corpus that fits in the window
- Whole-repository review or refactoring where relevance is hard to predict
- Long-running agents whose history keeps growing
- Deciding between long context, retrieval (RAG), or both
Workflow
Step 1: Check the window for your model
Context window size depends on the model. The docs list which current models have a 1M-token window and which have 200K, and state that for models with a 1M window, 1M is the default with no beta header. Older guidance that required a context-1m-... beta header applies only to the models it was written for.
Do not hard-code the number. Read it at runtime:
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic();
const MODEL = process.env.ANTHROPIC_MODEL!; // a current model ID
const info = await client.models.retrieve(MODEL);
console.log(info.max_input_tokens, info.max_tokens); // context window, output cap
Pricing for very long prompts also varies by model; check the pricing page's long-context section rather than assuming a flat rate.
Step 2: Count tokens before you send
const { input_tokens } = await client.messages.countTokens({
model: MODEL,
system,
messages,
});
if (input_tokens > info.max_input_tokens! * 0.9) {
// trim, retrieve instead, or compact first
}
Everything counts: system prompt, every message (tool results, images, documents), tool definitions, and the output including thinking. If the input alone exceeds the window, the API returns 400 invalid_request_error ("prompt is too long"). On recent models, if input plus max_tokens would exceed the window, the request is accepted and generation can stop with stop_reason: "model_context_window_exceeded"; handle that stop reason.
Step 3: Structure the input for recall
The docs warn that accuracy and recall degrade as token count grows ("context rot"). Structure helps the model find things:
const prompt = `
<documents>
<document index="1" source="src/auth.ts">...</document>
<document index="2" source="docs/architecture.md">...</document>
</documents>
Using only the documents above, explain how a session token is validated.
Quote the relevant lines before answering.
`;
- Put long material first and the question at the end
- Wrap each source in tags with an identifier the answer can cite
- Ask for supporting quotes before the answer on retrieval-style questions
- Remove what is clearly irrelevant: less noise beats more coverage
Step 4: Cache the stable part
If several questions hit the same corpus, put the corpus before a cache breakpoint so later requests read it from cache (see the prompt-caching skill). Caching changes what you pay for those tokens, not whether they count toward the window.
Step 5: Manage growing conversations
For agents and long chats, choose a strategy before you hit the limit:
| Strategy | What it does | Where it runs |
|---|---|---|
| Compaction (beta) | Replaces older turns with a summary Claude writes | Server-side; on demand or at a token threshold |
| Context editing (beta) | Clears old tool results or thinking blocks by rule | Server-side, configured per request |
| Memory tool | Moves durable facts into files Claude reads on demand | Your storage (see memory-tool) |
| Retrieval | Fetches only relevant chunks per question | Your pipeline |
Compaction and context editing are beta features with their own beta headers and parameters, and both have changed over time. Follow the current compaction and context-editing pages for exact request shapes. Whichever you use, append the full response.content back to history, not just the text, so server-generated blocks are preserved.
Long Context vs Retrieval
| Long context fits better | Retrieval fits better |
|---|---|
| Corpus fits comfortably in the window | Corpus is unbounded or much larger than the window |
| Relevance is hard to predict in advance | Queries target known, narrow facts |
| Few questions per corpus load (or cached) | Many cheap lookups across many users |
| Synthesis across many sources | Data changes frequently or needs per-user isolation |
A hybrid is common: retrieve a generous candidate set, then let the model reason over it in one long prompt.
Latency
Large inputs take longer to process. Stream responses so users see progress, and cache stable prefixes so repeat requests skip most of the input processing.
Anti-Patterns
- Hard-coding a window size or beta header copied from an old example
- Sending an entire repository when a directory would do
- Undelimited blobs of text with the question at the top
- Letting agent history grow until the API rejects the request
- Appending only the text of responses when using compaction, which drops the compaction block
Best Practices
- Read the window from the Models API and count tokens before sending
- Structure long inputs with tagged sources and put the question last
- Cache the reusable prefix
- Plan context management (compaction, editing, memory, retrieval) from the start for agents
- Handle
model_context_window_exceededand "prompt is too long" explicitly - Measure answer quality at your real input sizes; do not assume recall is uniform
References
- Context windows: https://platform.claude.com/docs/en/build-with-claude/context-windows
- Token counting: https://platform.claude.com/docs/en/build-with-claude/token-counting
- Compaction: https://platform.claude.com/docs/en/build-with-claude/compaction
- Context editing: https://platform.claude.com/docs/en/build-with-claude/context-editing
- Models overview: https://platform.claude.com/docs/en/about-claude/models/overview
- Effective context engineering: https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents