viktorbezdek Context Compression Strategies
When agent sessions generate millions of tokens of conversation history, compression becomes mandatory. The naive approach is aggressive compression to minimize tokens per request. The correct optimization target is tokens per task: total tokens consumed to complete a task, including re-fetching costs when compression loses critical information.
When to Use / Not Use
Use when:
- Agent sessions exceed context window limits
- Codebases exceed context windows (5M+ token systems)
- Designing conversation summarization strategies
- Debugging cases where agents "forget" what files they modified
- Building evaluation frameworks for compression quality
Do NOT use when:
- Diagnosing why context is degrading -> use
context-degradation - KV-cache optimization or context partitioning -> use
context-optimization - Learning foundational context theory -> use
context-fundamentals - File-based offloading or scratch pads -> use
filesystem-context
Decision Tree
Why do you need compression?
├── Agent sessions hitting context limits
│ ├── What matters most?
│ │ ├── File tracking + decision history (long sessions) -> Anchored Iterative Summarization
│ │ ├── Maximum token savings (short sessions, low re-fetch cost) -> Opaque Compression
│ │ └── Readability + phase boundaries -> Regenerative Full Summary
│ └── Not sure? -> Start with Anchored Iterative (best quality trade-off)
├── Need to measure if compression is working
│ └── Probe-based evaluation (§Probe-Based Evaluation)
├── When to trigger compression?
│ └── See §Compression Trigger Strategies
└── Not about reducing size? -> See related skills
Core Compression Approaches
| Method | Ratio | Quality | Best For |
|---|---|---|---|
| Anchored Iterative | 98.6% | 3.70 | Long sessions where file tracking matters (coding, debugging) |
| Opaque | 99.3% | 3.35 | Short sessions, maximum token savings, low re-fetch costs |
| Regenerative Full | 98.7% | 3.44 | Sessions with clear phase boundaries, readability critical |
Key insight: Structure forces preservation. Dedicated summary sections act as checklists the summarizer must populate, preventing silent information drift.
Anchored Iterative Summarization
- Define explicit summary sections matching your agent's needs
- On first compression trigger, summarize truncated history into sections
- On subsequent compressions, summarize only new truncated content
- Merge new summary into existing sections (not full regeneration)
- Track which information came from which compression cycle
When to Use Each Approach
| Condition | Recommended Approach |
|---|---|
| Sessions 100+ messages, file tracking critical | Anchored Iterative |
| Maximum token savings, short sessions, low re-fetch cost | Opaque |
| Clear phase boundaries, readability critical | Regenerative |
| Need to audit what was preserved | Anchored Iterative |
Structured Summary Sections
## Session Intent
[What the user is trying to accomplish]
## Files Modified
- auth.controller.ts: Fixed JWT token generation
- config/redis.ts: Updated connection pooling
- tests/auth.test.ts: Added mock setup for new config
## Decisions Made
- Using Redis connection pool instead of per-request connections
- Retry logic with exponential backoff for transient failures
## Current State
- 14 tests passing, 2 failing
- Remaining: mock setup for session service tests
## Next Steps
1. Fix remaining test failures
2. Run full test suite
3. Update documentation
Compression Trigger Strategies
| Strategy | Trigger Point | Trade-off |
|---|---|---|
| Fixed threshold | 70-80% context utilization | Simple but may compress too early |
| Sliding window | Keep last N turns + summary | Predictable context size |
| Importance-based | Compress low-relevance sections first | Complex but preserves signal |
| Task-boundary | Compress at logical task completions | Clean summaries but unpredictable timing |
Recommendation: Sliding window + structured summaries for most coding agent use cases.
The Artifact Trail Problem
Artifact trail integrity is the weakest dimension across all compression methods (2.2-2.5/5.0 in evaluations). Coding agents need to track:
- Which files were created, modified, or read-only
- Function names, variable names, error messages
Solution: Supplement structured summaries with a separate artifact index or dedicated file-state tracking in agent scaffolding.
Tokens-Per-Task vs Tokens-Per-Request
Traditional metrics target tokens-per-request. This is the wrong optimization. When compression loses critical details, the agent must re-fetch information, re-explore approaches, and waste tokens recovering context.
The right metric is tokens-per-task: total tokens consumed from task start to completion. A compression strategy saving 0.5% more tokens per request but causing 20% more re-fetching costs more overall.
Probe-Based Evaluation
Traditional metrics (ROUGE, embedding similarity) fail for functional quality. Probe-based evaluation directly measures functional quality by asking questions after compression:
| Probe Type | What It Tests | Example Question |
|---|---|---|
| Recall | Factual retention | "What was the original error message?" |
| Artifact | File tracking | "Which files have we modified?" |
| Continuation | Task planning | "What should we do next?" |
| Decision | Reasoning chain | "What did we decide about the Redis issue?" |
Six Evaluation Dimensions
- Accuracy: Are technical details correct? (largest variation between methods)
- Context Awareness: Does response reflect current conversation state?
- Artifact Trail: Does agent know which files were read or modified? (universally weak)
- Completeness: Does response address all parts of the question?
- Continuity: Can work continue without re-fetching?
- Instruction Following: Does response respect stated constraints?
Anti-Patterns
| Anti-Pattern | Problem | Solution |
|---|---|---|
| Optimizing tokens-per-request | Re-fetch costs exceed savings; agent re-explores already-tried approaches | Optimize for tokens-per-task instead; measure total cost including recovery |
| Generic summarization | "Summarize this conversation" loses structured information | Use anchored iterative summarization with explicit sections for files, decisions, next steps |
| Full regeneration each cycle | Repeated regeneration introduces subtle drift; each cycle loses a few details | Use incremental merging: summarize only new truncated content, merge into existing sections |
| Ignoring artifact trail | File tracking degrades across long sessions (2.2-2.5/5.0 quality) | Implement separate artifact index outside the summary; dedicated file-state tracking |
| Compressing during active reasoning | Agent loses thread of current investigation mid-compression | Switch from fixed-threshold triggers to task-boundary triggers; compress only at logical completion points |
| No quality measurement | Cannot distinguish good compression from bad | Implement probe-based evaluation; run recall/artifact/continuation/decision probes after each compression |
| Over-aggressive compression ratio | 99.3% compression (opaque) loses 0.35 quality points vs 98.6% (anchored) | Accept slightly lower compression ratios for better quality; the 0.7% tokens retained buy 40% fewer re-fetch cycles |
Guidelines
- Optimize for tokens-per-task, not tokens-per-request
- Use structured summaries with explicit sections for file tracking
- Trigger compression at 70-80% context utilization
- Implement incremental merging rather than full regeneration
- Test compression quality with probe-based evaluation
- Track artifact trail separately if file tracking is critical
- Accept slightly lower compression ratios for better quality retention
- Monitor re-fetching frequency as a compression quality signal
Integration
- context-degradation - Compression is a mitigation strategy for degradation
- context-optimization - Compression is one optimization technique among many
- memory-systems - Compression relates to scratchpad and summary memory patterns
References
Internal reference:
- Evaluation Framework Reference - Detailed probe types and scoring rubrics
External resources:
- Factory Research: Evaluating Context Compression for AI Agents (December 2025)
- Research on LLM-as-judge evaluation methodology (Zheng et al., 2023)