Research: Compaction System Mapping for TunaCode

Research -- Compaction System Mapping for TunaCode

Date: 2026-02-10 Owner: claude Phase: Research git_commit: a454860be6cd2c8c3d1a1a35f9fb9beb7edc5685

Goal

Map the current tunacode codebase against the pi-mono compaction system architecture (documented in memory-bank/research/2026-02-07_18-06-58_compaction-system.md) to identify what exists, what is missing, and what needs to be built to implement context compaction.

Summary

TunaCode has no compaction system. Messages grow unboundedly, token estimation is cosmetic (display only), and context overflow errors are unhandled. However, the architecture has clean integration points: tinyagent exposes a transform_context hook designed exactly for compaction, and tunacode already has token estimation, session persistence, and a command system ready to host compaction.

Findings

Current Architecture vs Pi-Mono Compaction Features

Key Files That Already Exist

Deleted Files (Ghost .pyc Only)

These files once existed but were removed during the pydantic-ai migration. They may have contained early compaction logic:

• core/agents/resume/summary.py -- summarization logic (deleted)
• core/agents/resume/prune.py -- pruning logic (deleted)
• core/agents/agent_components/truncation_checker.py -- truncation checking (deleted)
• core/agents/history_preparer.py -- history preparation (deleted)
• core/agents/agent_components/message_handler.py -- message handling (deleted)

Key Patterns / Solutions Found

1. TinyAgent transform_context Hook -- The Primary Integration Point

TinyAgent exposes transform_context in AgentOptions -- an async callback that receives the full message list before each LLM call and returns a (potentially modified) list. This is the designed injection point for compaction.

Location: /home/tuna/tunacode/tinyAgent/tinyagent/agent.py (AgentOptions dataclass) Called at: /home/tuna/tunacode/tinyAgent/tinyagent/agent_loop.py:107

if config.transform_context:
    messages = await config.transform_context(messages, signal)

TunaCode constructs AgentOptions at src/tunacode/core/agents/agent_components/agent_config.py:286-290 but does NOT set transform_context. Wiring a compaction function here would intercept every LLM call with zero changes to tinyagent.

2. Message Flow: How Messages Accumulate

User types message
    |
    v
app.py -> process_request(message)
    |
    v
RequestOrchestrator._run_impl()
    |
    +-- conversation.messages -> agent.replace_messages(history)  [L312-316]
    +-- agent.stream(user_message)
    |       |  (inside tinyagent agent_loop.py)
    |       +-- Appends UserMessage
    |       +-- Calls LLM -> Appends AssistantMessage
    |       +-- If tool calls: Appends ToolResultMessages
    |       +-- Loops until no more tool calls
    |
    +-- _persist_agent_messages() [L362-373]
    |     conversation.messages = agent.state["messages"] + external
    |
    v
state_manager.save_session()  [JSON to disk]

Each user request can generate many messages (user + assistant + N tool calls + N tool results + follow-up assistant). Growth is bounded per-request only by max_iterations (default 15), but accumulates unboundedly across requests.

3. Token Estimation Infrastructure

src/tunacode/utils/messaging/token_counter.py:

• estimate_tokens(text: str) -> int -- len(text) // 4
• estimate_message_tokens(message) -> int -- converts to CanonicalMessage, counts all content
• estimate_messages_tokens(messages) -> int -- sums across all messages

Identical to pi-mono's chars/4 heuristic. Used only in app.py:_update_resource_bar().

4. Context Window Configuration

ConversationState.max_tokens is set from the model registry's context window (default 200,000). The resource bar displays estimated_tokens / max_tokens. No code compares these values for enforcement.

5. Message Types (TinyAgent Format)

Three message roles in tinyagent:

• UserMessage -- role: "user", content: text/image
• AssistantMessage -- role: "assistant", content: text/thinking/tool_call, has usage and stop_reason
• ToolResultMessage -- role: "tool_result", has tool_call_id, tool_name, content, is_error

Valid cut points for compaction: after UserMessage or after AssistantMessage (without pending tool calls). Never cut between a tool_call and its tool_result.

6. Session Persistence Format

Sessions are JSON files at ~/.local/share/tunacode/sessions/{project_id}_{session_id}.json:

{
  "version": 1,
  "session_id": "...",
  "project_id": "...",
  "created_at": "...",
  "last_modified": "...",
  "working_directory": "...",
  "current_model": "...",
  "session_total_usage": {...},
  "thoughts": [...],
  "messages": [...]
}

Compaction state will be stored as a new top-level "compaction" field (see Architecture section). Old sessions without this field default to None -- no migration needed.

Architecture for Compaction Implementation

Council Review (2026-02-10): Reviewed by Codex (structural integrity), Gemini (conceptual clarity), and Qwen (pragmatic longevity). All recommendations accepted. Decisions locked below.

Design Decisions (Locked)

Trigger Architecture: CompactionController

A single CompactionController owns the policy: "should we compact?" It is invoked from two sites only:

1. Pre-request (primary): Called from _run_impl() before agent.stream(). Estimates tokens, compacts if above threshold, shows "Compacting..." in UI.
2. Overflow retry (fallback): Catches context overflow API errors, triggers compaction via the same controller, retries the request once.

The transform_context hook delegates to the controller. It does NOT independently decide to compact. This prevents double-compaction and state divergence.

User types message
    |
    v
RequestOrchestrator._run_impl()
    |
    +-- CompactionController.check_and_compact(conversation)
    |     |
    |     +-- estimate_tokens(messages)
    |     +-- if above threshold:
    |     |     +-- calculate retention window
    |     |     +-- serialize compactable messages
    |     |     +-- call summarizer LLM (same model)
    |     |     +-- store CompactionRecord in session metadata
    |     |     +-- replace old messages with summary UserMessage
    |     |     +-- show "Compacting..." UI status
    |     +-- if summarization fails:
    |           +-- log error, proceed with full context (fail-safe)
    |
    +-- conversation.messages -> agent.replace_messages(history)
    +-- agent.stream(user_message)
    |       |
    |       +-- transform_context hook:
    |       |     delegates to CompactionController (no-op if already compacted)
    |       +-- LLM call
    |       +-- tool calls / results loop
    |
    +-- if ContextOverflowError:
    |     +-- CompactionController.force_compact(conversation)
    |     +-- retry agent.stream() once
    |
    +-- _persist_agent_messages()
    v
state_manager.save_session()

Compaction State: Session-Level Metadata (NOT Message List)

Compaction bookkeeping lives in a CompactionRecord persisted as a top-level field in the session JSON. The message list stays pure -- only user, assistant, tool_result roles.

Session format after compaction:

{
  "version": 1,
  "session_id": "...",
  "messages": [
    {"role": "user", "content": [{"type": "text", "text": "...summary injected at runtime..."}]},
    {"role": "user", "content": [{"type": "text", "text": "Now update the tests"}]},
    {"role": "assistant", "content": [{"type": "text", "text": "I'll update..."}]}
  ],
  "compaction": {
    "summary": "## Goal\nUser is refactoring auth module...",
    "compacted_message_count": 47,
    "tokens_before": 150000,
    "tokens_after": 25000,
    "compaction_count": 1,
    "previous_summary": null,
    "last_compacted_at": "2026-02-10T21:30:00Z"
  }
}

How the summary reaches the LLM:

1. CompactionRecord is stored in session metadata (not in messages)
2. transform_context reads the record and injects a synthetic UserMessage containing the summary text at the front of the message list
3. This injected message is transient -- never persisted back to the message list
4. The LLM sees: [summary UserMessage] + [recent real messages]
5. No convert_to_llm override needed. No custom roles. Standard message types only.

Migration: Old sessions without a "compaction" field get None default. No migration logic needed.

Iterative updates: On subsequent compactions, the existing record's summary becomes previous_summary. The summarizer receives both the old summary and new messages to produce an updated summary. File tracking comes from ConversationState.files_in_context, not from the compaction record.

Components to Build

Configuration Parameters

Note: compaction_model removed -- always uses the current conversation model.

Retention Window Calculation

The algorithm determines what to keep, not where to cut. Walk backwards from the newest message, accumulating estimated tokens. Stop when accumulated >= keep_recent_tokens. Snap to the nearest valid boundary.

Valid boundaries:

• After a UserMessage
• After an AssistantMessage with stop_reason != None (completed response)

Never split:

• Between a tool_call (in AssistantMessage.content) and its ToolResultMessage
• At an aborted message (stop_reason == "aborted")

Everything before the retention window boundary is serialized and summarized. Everything after it is kept as-is.

Message Serialization for Summarizer

Convert messages outside the retention window to text format for the summarizer LLM:

[User]: What they said
[Assistant]: Response text
[Tool Call]: tool_name(arg1="val1", arg2="val2")
[Tool Result]: Output text (truncated to 500 chars)
[Assistant]: Follow-up response

Tool results should be truncated aggressively -- the summarizer only needs the gist, not the full output.

Structured Summary Format (Adapted for TunaCode)

## Goal
[What the user is trying to accomplish]

## Constraints & Preferences
- [Requirements gathered from conversation]

## Progress
### Done
- [x] Completed actions with file paths
### In Progress
- [ ] Current work

## Key Decisions
- **[Decision]**: [Rationale]

## Next Steps
1. [What should happen next]

## Files Touched
### Read
- path/to/file.py
### Modified
- path/to/changed.py

## Critical Context
- [Data the assistant needs to continue effectively]

Fail-Safe Behavior

When summarization fails (LLM error, timeout, rate limit):

1. Log the failure with full error context
2. Proceed with uncompacted context -- send the full message list to the LLM
3. If the LLM call also fails with overflow: surface ContextOverflowError to the user via _show_system_notice() with message: "Context too large. Compaction failed. Use /compact to retry manually."
4. Never silently swallow a compaction failure
5. Never retry summarization automatically -- one attempt per trigger

When the user cancels (abort signal during transform_context):

1. Abort the summarization LLM call
2. Proceed with uncompacted context (fail-safe)
3. Do not surface an error -- the user chose to cancel

Knowledge Gaps

• tinyagent transform_context behavior under abort: Does the abort signal in transform_context(messages, signal) properly cancel an in-flight compaction LLM call? Need to verify signal propagation.
• OpenRouter error format for context overflow: Need to verify the exact error response format from OpenRouter when context is exceeded (likely context_length_exceeded in the error type or code field).
• Performance of chars/4 for compaction threshold: The heuristic overestimates by ~20-30%. For compaction threshold, this is acceptable (triggers early rather than late), but actual token counts from usage would be more precise.

References

• Pi-mono compaction reference: memory-bank/research/2026-02-07_18-06-58_compaction-system.md
• Token counter: src/tunacode/utils/messaging/token_counter.py
• Agent loop: src/tunacode/core/agents/main.py
• Agent config (transform_context wiring): src/tunacode/core/agents/agent_components/agent_config.py
• Session state: src/tunacode/core/session/state.py
• Conversation state: src/tunacode/core/types/state_structures.py
• TinyAgent agent loop: tinyAgent/tinyagent/agent_loop.py
• TinyAgent agent: tinyAgent/tinyagent/agent.py
• Resume sanitizer: src/tunacode/core/agents/resume/sanitize.py
• UI commands: src/tunacode/ui/commands/__init__.py
• Exceptions: src/tunacode/exceptions.py
• Resource bar: src/tunacode/ui/widgets/resource_bar.py
• App: src/tunacode/ui/app.py