eyadsibai Context Compression Strategies
When agent sessions generate millions of tokens, compression becomes mandatory. Optimize for tokens-per-task (total tokens to complete a task), not tokens-per-request.
Compression Approaches
1. Anchored Iterative Summarization (Recommended)
- Maintain structured summaries with explicit sections
- On compression, summarize only newly-truncated content
- Merge with existing summary instead of regenerating
- Structure forces preservation of critical info
2. Opaque Compression
- Highest compression ratios (99%+)
- Sacrifices interpretability
- Cannot verify what was preserved
3. Regenerative Full Summary
- Generate detailed summary on each compression
- Readable but may lose details across cycles
- Full regeneration rather than merging
Structured Summary Format
## Session Intent
[What the user is trying to accomplish]
## Files Modified
- auth.controller.ts: Fixed JWT token generation
- config/redis.ts: Updated connection pooling
## Decisions Made
- Using Redis connection pool instead of per-request
- Retry logic with exponential backoff
## Current State
- 14 tests passing, 2 failing
- Remaining: mock setup for session service tests
## Next Steps
1. Fix remaining test failures
2. Run full test suite
3. Update documentation
Compression Triggers
| Strategy | Trigger | Trade-off |
|---|---|---|
| Fixed threshold | 70-80% context | Simple but may compress early |
| Sliding window | Last N turns + summary | Predictable size |
| Importance-based | Low-relevance first | Complex but preserves signal |
| Task-boundary | At task completions | Clean but unpredictable |
The Artifact Trail Problem
File tracking is the weakest dimension (2.2-2.5/5.0 in evaluations). Coding agents need:
- Which files were created
- Which files were modified and what changed
- Which files were read but not changed
- Function names, variable names, error messages
Solution: Separate artifact index or explicit file-state tracking.
Probe-Based Evaluation
Test compression quality with probes:
| Probe Type | Tests | Example |
|---|---|---|
| Recall | Factual retention | "What was the original error?" |
| Artifact | File tracking | "Which files have we modified?" |
| Continuation | Task planning | "What should we do next?" |
| Decision | Reasoning chain | "What did we decide about Redis?" |
Compression Ratios
| Method | Compression | Quality | Trade-off |
|---|---|---|---|
| Anchored Iterative | 98.6% | 3.70 | Best quality |
| Regenerative | 98.7% | 3.44 | Moderate |
| Opaque | 99.3% | 3.35 | Best compression |
The 0.7% extra tokens buys 0.35 quality points—worth it when re-fetching costs matter.
Three-Phase Workflow (Large Codebases)
- Research Phase: Explore and compress into structured analysis
- Planning Phase: Convert to implementation spec (~2,000 words for 5M tokens)
- Implementation Phase: Execute against the spec
Best Practices
- Optimize for tokens-per-task, not tokens-per-request
- Use structured summaries with explicit file sections
- Trigger compression at 70-80% utilization
- Implement incremental merging over regeneration
- Test with probe-based evaluation
- Track artifact trail separately if critical
- Monitor re-fetching frequency as quality signal