Prompt Cache Control

Explicit Prompt Cache Control Protocol (Breakpoint Architecture)

Overview

Large multi-turn agent sessions re-send the entire conversation history, tool definitions, and system guidelines on every turn. In a 40-turn task with a 5,000-token prefix, the client transmits 200,000 input tokens.

Modern LLM caching engines (Anthropic Prompt Caching, OpenAI Automatic Prefix Caching, DeepSeek Context Caching) store transformer KV-cache states in GPU VRAM across requests. When a prompt matches a cached prefix:

• Input Token Cost Drops by 90% (e.g. from $3.00/M tokens down to $0.30/M tokens on Claude 3.5 Sonnet).
• Time-to-First-Token (TTFT) Drops by 75% to 85%.
• Cache TTL: Automatically refreshes for 5 minutes after every read.

The Explicit Cache Control Protocol places strategic cache breakpoints (cache_control: {"type": "ephemeral"}) at stability boundaries to maximize cache hit rates.

Uncached Re-Ingestion vs. Strategic Breakpoint Caching

┌─────────────────────────────────────────────────────────────┐
│                 Prompt Caching Economics                    │
│                                                             │
│  Uncached Execution (40 Turns $\times$ 5,000 tokens):             │
│  • Total Input Tokens Billed: 200,000 tokens @ $3.00/M      │
│  • Input Bill: $0.60 per task                               │
│  • Average TTFT Latency: 2.8 seconds / turn                 │
│                                                             │
│  Strategic Breakpoint Caching (97.5% Cache Hit Rate):       │
│  • Turn 1 (Cache Write): 5,000 tokens @ $3.75/M = $0.018    │
│  • Turns 2..40 (Cache Read): 195,000 tokens @ $0.30/M=$0.058│
│  ↳ Total Input Bill: $0.076 (87.3% Net Financial Savings!)  │
│  ↳ Average TTFT Latency: 0.35 seconds / turn (8x Faster!)   │
└─────────────────────────────────────────────────────────────┘

The 4-Breakpoint Anthropic Architecture

Anthropic allows up to 4 discrete cache_control breakpoints per API request. Place them at descending layers of stability:

┌───────────────────────────────────────────────────────────────────────────┐
│ BREAKPOINT 1: Core System Identity & Rules (100% Static) ──► CACHE_POINT 1│
│ BREAKPOINT 2: MCP Tool Definitions & Schemas             ──► CACHE_POINT 2│
│ BREAKPOINT 3: Repository Architecture Blueprint (Session)──► CACHE_POINT 3│
│ BREAKPOINT 4: Historical Conversation History (Turn N-2) ──► CACHE_POINT 4│
│ ───────────────────────────────────────────────────────────────────────── │
│ DYNAMIC TAIL: Latest User Prompt & Tool Execution Result ──► (Dynamic)    │
└───────────────────────────────────────────────────────────────────────────┘

Production Anthropic SDK Implementation

import anthropic
from typing import List, Dict, Any

client = anthropic.Anthropic()

def create_cached_agent_request(
    system_prompt: str,
    tool_schemas: List[Dict[str, Any]],
    conversation_history: List[Dict[str, Any]],
    latest_user_prompt: str
) -> anthropic.types.Message:
    """Dispatches Anthropic request with multi-layer cache breakpoints."""
    
    # Format messages array with cache breakpoint on the older conversation history
    messages_payload = []
    
    # 1. Add older history turns
    for idx, msg in enumerate(conversation_history):
        # Set cache breakpoint on the penultimate turn to cache conversation prefix
        if idx == len(conversation_history) - 1:
            messages_payload.append({
                **msg,
                "content": [
                    {
                        "type": "text",
                        "text": msg["content"] if isinstance(msg["content"], str) else msg["content"][0]["text"],
                        "cache_control": {"type": "ephemeral"}
                    }
                ]
            })
        else:
            messages_payload.append(msg)
            
    # 2. Add latest dynamic turn (uncached)
    messages_payload.append({"role": "user", "content": latest_user_prompt})
    
    response = client.messages.create(
        model="claude-3-5-sonnet-20241022",
        max_tokens=2048,
        system=[
            {
                "type": "text",
                "text": system_prompt,
                "cache_control": {"type": "ephemeral"}  # Breakpoint 1: System prompt
            }
        ],
        tools=[
            # Breakpoint 2: Tool definitions cache
            {**tool_schemas[0], "cache_control": {"type": "ephemeral"}},
            *tool_schemas[1:]
        ],
        messages=messages_payload
    )
    
    # Telemetry inspection
    usage = response.usage
    print(f"Cache Read (90% discount): {getattr(usage, 'cache_read_input_tokens', 0)}")
    return response

The 3 Golden Rules to Prevent Cache Invalidation

1. Keep Prefix Bytes 100% Identical: Even a single whitespace change, reordered JSON key, or modified punctuation in the prefix invalidates everything downstream of that byte.
2. Sort Tool Schemas Deterministically: Always serialize tool JSON with sort_keys=True so tool ordering never shifts between turns.
3. Minimum Token Threshold: Breakpoints only trigger on blocks $\ge 1,024$ tokens on Sonnet ($\ge 2,048$ tokens on Haiku). Group smaller instructions into a single block to clear the threshold.

Benchmark Comparison

Evaluation across 50 multi-turn coding sessions:

Agent Operational Directive

MANDATORY: Agent runtimes communicating with Anthropic or OpenAI models must enable explicit prompt caching on system prompts, tool schemas, and conversation history anchors. Keep static headers immutable across the entire session.