Context Engineering

Context Engineering

Concept of the skill

Context engineering is the discipline of compiling the smallest sufficient, highest-signal working set for an LLM at each step: instructions, memory, retrieved facts, tool outputs, examples, conversation history, and task metadata.

Coverage

• Core principle: the model is a reasoning engine that reasons over whatever is in its context window — wrong context produces correct reasoning over false premises
• The five-layer context stack: system prompt, persistent memory, always-loaded rules, injected skills, agent prompt — what each layer does and how each can fail
• The four context failure modes: missing, stale, wrong, overwhelming — diagnostic questions for each, table of symptoms, and prevention strategies
• Four context quality metrics: injection precision, injection recall, context utilization, freshness score — definitions, healthy ranges, and how to measure each
• Context-compilation levers: selection, structuring, sequencing, compaction, memory integration, retrieval, provenance, and tool-result clearing
• Frequent Intentional Compaction (FIC): proactive compaction at task boundaries, target utilization range, and the difference between planned and forced compaction
• Subagent delegation pattern: when to delegate context-heavy investigation to a subagent so the main agent receives a summary instead of raw evidence
• Debugging decision tree: how to diagnose any agent failure by walking from missing-context through overwhelming-context before blaming the model
• The verification checklist: gates a context-engineering review must pass before declaring the pipeline healthy

Philosophy of the skill

The model is a reasoning engine that reasons over whatever is in its context window. If the context is wrong, the reasoning is correct but the conclusion is wrong. This means most agent failures are context failures, not model failures.

Without this discipline, teams blame the model for mistakes caused by missing keywords, stale skill content, or an overwhelmed window. Context engineering provides the diagnostic framework to identify why an agent produced a wrong answer and the design principles to prevent recurrence. It treats the context window as a deliberate design surface — not a dumping ground — so that the model's native reasoning produces the correct output without heroic prompting.

Current agent literature converges on the same lesson: context is finite working memory, not a warehouse. Anthropic frames context engineering as curating the optimal set of tokens across instructions, tools, MCP, external data, and message history; Anthropic's API docs warn that more context is not automatically better because recall can degrade as token count grows; IBM describes context engineering as selecting, structuring, compressing, sequencing, and integrating tool or memory information so agents can reason over the right facts. Those sources all point to the same practical rule: every context item needs a job.

Most agent failures are context failures, not model failures. Context engineering is the discipline of designing what information the model sees, when it sees it, and in what form — so that the model's native reasoning produces the correct output without heroic prompting.

Core Principle

The model is a reasoning engine. It reasons over whatever context is in its window. If the context is wrong, the reasoning is correct but the conclusion is wrong. You cannot prompt your way out of a context problem.

Three implications:

1. Garbage in, garbage out — a perfectly written prompt cannot compensate for missing domain knowledge. The model will hallucinate confidently.
2. Signal-to-noise ratio matters — flooding the context window with irrelevant information degrades performance just as surely as omitting critical information. The model attends to everything.
3. Context is a design surface — the information in the context window is as intentional as a database schema or an API contract. It should be designed, measured, and iterated.

The Five-Layer Context Stack

Layer 5 — Agent prompt          (the task-specific instruction from user or orchestrator)
Layer 4 — Injected skills       (domain knowledge selected per task by a router or injector)
Layer 3 — Always-loaded rules   (universal guardrails: security, naming, GDPR, etc.)
Layer 2 — Persistent memory     (cross-session knowledge: user preferences, prior decisions)
Layer 1 — System prompt         (foundational identity, non-negotiable rules, reading order)

Each layer adds context. Each layer can introduce failure. The context engineer's job is to ensure the right information reaches the right layer at the right time.

The five layers are the stable scaffolding. Around them sits the runtime payload: retrieved documents, tool results, examples, summaries, temporary notes, and conversation history. Treat that payload as a compiled artifact. It should be rebuilt for the current step, not treated as a permanent archive.

The exact mechanism varies by harness — Claude Code uses CLAUDE.md + .claude/rules/ + MEMORY.md; Cursor uses .cursorrules; OpenCode uses AGENTS.md; Aider uses CONVENTIONS.md. The abstraction holds across all of them.

The Four Context Failure Modes

Every agent mistake can be traced to one of four context failures. Diagnosing which failure mode caused the mistake is the first step to fixing it.

Missing Context

The agent does not have the information it needs. Most common, easiest to diagnose.

Diagnostic question: "Did the agent have access to the information that would have prevented this mistake?"

Stale Context

The agent has the information, but it is outdated. The skill says X, but the codebase has moved to Y.

Diagnostic question: "Was this information correct when it was written? Has the source of truth changed since?"

Prevention: every skill carries a drift_check.last_verified date. Skills whose verification is older than the lifecycle policy (e.g. 90 days for portable, 30 days for integration skills) should be reviewed before use in a high-stakes task.

Wrong Context

The agent has information, but it is incorrect. The most dangerous failure mode because the agent acts confidently on false premises.

Diagnostic question: "Was the information the agent acted on actually correct?"

Prevention: skill content cites specific files and line numbers. Generic "best practice" advice without grounded evidence is a wrong-context risk.

Overwhelming Context

The agent has too much information. The signal is diluted by noise; the model's attention is spread too thin.

Diagnostic question: "Did the agent have so much context that it couldn't focus on the right information?"

Prevention: measure injection precision. Treat "more than roughly 30% irrelevant injections" as a local tuning alarm, not a universal law; calibrate the threshold against the workspace's router, task mix, and cost of false positives.

Context Quality Metrics

Four metrics measure the health of a context engineering system. The bands below are starting heuristics for a local trend dashboard, not externally standardized thresholds. Calibrate them against known-good and known-bad task runs before using them as gates.

Injection Precision

Of all skills injected for a task, what percentage were actually needed?

Injection Precision = (skills used / skills injected) × 100

How to measure: after completing a task, review which injected skills were actually referenced in the agent's work. Skills injected but never consulted are false positives.

Injection Recall

Of all skills that would have been useful for a task, what percentage were actually injected?

Injection Recall = (relevant skills injected / relevant skills total) × 100

How to measure: after a context failure, check whether the skill that would have prevented it was indexed and whether its keywords matched the prompt.

Context Utilization

What percentage of the context window is used productively (contributes to correct output)?

Context Utilization = (productive context / total context used) × 100

Productive context: relevant skill content, necessary file reads, on-topic conversation history. Unproductive context: irrelevant skills, stale memory, redundant file reads, verbose tool output.

Freshness Score

What percentage of injected skill content is current (drift_check within the lifecycle window)?

Freshness Score = (skills with drift_check inside window / total injected skills) × 100

Frequent Intentional Compaction (FIC)

FIC is a proactive context-management strategy. Instead of waiting for the window to fill and reacting with an emergency compact, plan compaction points into the workflow.

Target utilization: use 40-60% of the context window during steady-state work as an initial heuristic, then adjust based on observed answer quality, tool-output volume, and compaction loss.

When to compact: at natural breakpoints, not when forced by pressure.

The forced compact is dangerous because it is uncontrolled — you lose whatever the compaction algorithm decides is least important, which may include critical task context.

Compaction is only one lever. Long-running agents also need memory and tool-result clearing. Memory stores durable notes outside the context window and retrieves them later when relevant. Tool-result clearing removes raw outputs after their useful summary or citation has been captured. The rule is simple: preserve decisions, evidence, and open constraints; clear raw material whose job is done.

FIC Anti-Patterns

Subagent Delegation for Context-Heavy Work

Some tasks consume enormous context (reading 20 files, searching across the codebase, analysing dependencies). Doing this in the main session pollutes its context for every subsequent step. Delegate to a subagent instead:

Main agent:    "I need to understand the data pipeline that feeds the dashboard"
                ↓ spawn subagent with narrow scope
Subagent:      reads 15 files, traces the pipeline, builds a mental model
                ↓ returns a 200-word structured summary
Main agent:    receives summary (≈200 tokens, not ≈15,000)

The subagent's context is disposable. The main agent gets the conclusion without the investigation cost.

This pattern is especially valuable for: codebase exploration, audit work, dependency tracing, multi-file refactor planning, and any "look at everything before deciding" investigation.

Debugging Agent Failures

When an agent produces wrong output, walk this decision tree before blaming the model:

Agent produced wrong output
    │
    ▼
Was the right skill (or rule) loaded?
    │
   NO ──▶  MISSING CONTEXT
    │       Fix: add keywords; promote rule to always-loaded layer; save the decision
    │
   YES
    │
    ▼
Was the loaded content correct and current?
    │
   NO ──▶  STALE or WRONG CONTEXT
    │       Fix: update the skill; verify against source; cite line numbers
    │
   YES
    │
    ▼
Was the context window > 80% full?
    │
   YES ──▶ OVERWHELMING CONTEXT
    │       Fix: apply FIC; delegate investigation to a subagent
    │
   NO
    │
    ▼
Was the prompt itself ambiguous or underspecified?
    │
   YES ──▶ Prompt problem (use prompt-craft)
    │
   NO
    │
    ▼
Genuine model reasoning failure (rare; try a different model or add observable reasoning scaffolds)

Prefer observable scaffolds when you reach the final branch: smaller tasks, explicit intermediate artifacts, checks, tests, or a stronger model. Do not mask a context failure by asking for more private reasoning.

Failure analysis template

When recording a context failure for retrospective:

## Context Failure Report

Date:            YYYY-MM-DD
Task:            [what the agent was asked to do]
Failure:         [what went wrong]
Failure mode:    Missing | Stale | Wrong | Overwhelming
Root cause:      [why the context was bad]
Fix applied:     [what was changed to prevent recurrence]
Layer affected:  Layer 1 / 2 / 3 / 4 / 5 (system prompt / memory / rules / skills / prompt)

Verification

Use this checklist when designing a new skill, debugging a failure, or auditing the context pipeline:

• Every agent failure was diagnosed to one of the four failure modes (Missing, Stale, Wrong, Overwhelming) before any fix landed
• Skills involved in any failure now have keywords that match how users actually phrased the request
• Skill content cites specific files and line numbers, not generic advice
• drift_check.last_verified is inside the lifecycle window for every active skill
• No two skills give contradictory advice on the same topic (relations.suppresses is honest about ownership)
• Context utilization is tracked at task breakpoints; FIC fires at natural boundaries
• Injection precision is above 80% (most injected skills are actually consulted)
• Injection recall is above 90% (most needed skills are injected)
• Context-heavy investigations are delegated to subagents so the main agent receives summaries, not raw evidence
• Always-loaded rules contain only universal guardrails; domain-specific guidance lives in skills
• Raw tool results have been summarized, cited, or cleared once they are no longer needed
• Retrieved context records source, date, and relevance so stale or wrong inputs can be traced

Source Notes

• Anthropic's effective context engineering guide frames context engineering as curating the token set around agents, including tools, MCP, external data, and history.
• Anthropic's context-window docs describe the context window as working memory and warn that larger context does not automatically improve recall.
• Anthropic's context-engineering cookbook distinguishes compaction, memory, and tool-result clearing for long-running agents.
• IBM's context engineering overview names selection, structuring, prompt design, compression, sequencing, and tool or memory integration as core steps.
• Context Engineering 2.0 situates the concept historically and argues for systematic context-engineering practice in AI systems.

Do NOT Use When