Pi Tmux Smoke

pi tmux smoke test

pi --print is single-shot and the SDK scripts every turn - neither runs the interactive TUI, so neither can exercise a real /compact, a slash command, an overlay, or a keybinding. To observe those you drive the actual pi binary inside a detached tmux pane: type into it with tmux send-keys, read the rendered screen with tmux capture-pane. This skill is the mechanics and the sharp edges.

Prefer the SDK or pi --print whenever the behavior is scriptable (see ../feature-eval-author/SKILL.md for the driver-choice table). tmux is the fallback for interactive-only behavior; it is flakier because you assert on a screen scrape, not structured events.

When to use this skill

Reach for a tmux-driven run when the thing under test only manifests interactively:

• A real /compact (and anything that arms on session_before_compact, e.g. the memory capture nudge).
• A slash command or overlay: /todos, /scratchpad, /memory, /skill:<name>.
• A keybinding, the editor, or any TUI affordance.
• An end-to-end "does the session even come up clean with these extensions" smoke check.

Stay on the SDK / --print for: save / recall / dedup / secret-gating (success = a file or a toolResult event), READ / ACT / SELECT probes, and anything you want to run at high N with exact scoring.

Workflow

1. Source the env, in the pane

Loading your provider's credentials in the pane is mandatory: a fresh tmux shell does not inherit them - not even a login shell. Load them in the pane's command line, not just the outer shell that calls tmux.

2. Boot, send, capture

source ~/.pi/agent/env 2>/dev/null || true   # outer shell: load creds if your setup keeps them there
S=pi-smoke-$$
MODEL=provider/model                         # e.g. anthropic/claude-haiku-4-5
tmux new-session -d -s "$S" -x 200 -y 50     # detached; give it a wide pane so output isn't truncated
tmux send-keys -t "$S" "{ source ~/.pi/agent/env 2>/dev/null || true; } && pi --model $MODEL" Enter
sleep 8                                       # let the TUI boot before typing the first turn
tmux send-keys -t "$S" 'first turn that establishes the fact' Enter
# Poll instead of guessing the turn length (see step 4):
tmux send-keys -t "$S" '/compact' Enter
sleep 12
# Assert, don't dump the screen: surface a one-word verdict, not 50 rows of TUI chrome (see "Token cost" below).
tmux capture-pane -t "$S" -p | grep -q "$TOKEN" && echo PASS || echo FAIL
tmux kill-session -t "$S"

To send a literal Enter as its own keypress (submit), pass Enter as a separate send-keys arg. To send text that contains characters tmux treats specially, use send-keys -l (literal) for the text and a separate Enter.

3. Force a /compact on a short session

/compact throws "Nothing to compact (session too small)" before it emits session_before_compact whenever there's nothing older than the kept window (keepRecentTokens defaults to 20000). To exercise compaction on a short smoke run:

• Point PI_CODING_AGENT_DIR at a temp dir of symlinks to ~/.pi/agent/* minus settings.json, and write a settings.json there with "compaction": { "enabled": true, "keepRecentTokens": 1 }.
• Run ≥2 short turns before /compact so there is a turn to summarize. (Use ≥3 if your check mines the compaction summary: a 1-turn forced compaction yields a split-turn summary - ## Original Request / ## Early Progress / ## Context for Suffix - rather than the normal ## Goal / ## Constraints & Preferences / ## Key Decisions shape.)

4. Assert on the pane - and beat the flake

capture-pane -p is a screen scrape, so timing is the main flake source. sleep is a guess; if trials are inconsistent, poll until a sentinel appears instead of sleeping a fixed time:

for _ in $(seq 1 30); do
  tmux capture-pane -t "$S" -p | grep -q 'SENTINEL' && break   # prompt glyph, a tool-call line, a known token
  sleep 2
done

For state that the TUI does not render (e.g. a cache-safe <system-reminder> that is never persisted to the transcript), screen-scraping won't see it. Observe it with temporary env-gated instrumentation inside the extension's handler - appendFileSync to a debug path right after the inject call - run the smoke test, read the marker file, then revert the instrumentation. Commit real code first: a stray git checkout to strip debug will wipe uncommitted work on the same file.

Keep the caller's token cost down

Every capture-pane you read flows into the driving agent's context, not the model-under-test. A wide, tall pane is mostly blank rows and TUI chrome (borders, the input box, the status line, spinner frames) - pure token waste. Two facts to calibrate against, both measured:

• capture-pane -p already strips trailing whitespace per line, so trailing-space bloat is a non-issue - no need for a sed 's/ *$//'.
• It emits one line per pane row including blanks, so a -y 50 pane is ~49 mostly-empty lines even when the content is two lines. Pane height and width, not trailing spaces, are the bloat.

Levers, most effective first:

1. Assert, don't read. Pipe to grep -q and emit a one-word verdict (PASS/FAIL), or grep -o "$TOKEN" to surface only the match. The agent reads one token, not the screen. In a poll loop, grep -q produces zero output per iteration - only the final verdict reaches context.
2. Capture only the region you need. | tail -n 8 (the recent output sits at the bottom), or bound the range with capture-pane -p -S -12 (last 12 lines). Never read the full scrollback.
3. Drop blank rows and chrome before it reaches context. | grep -vE '^\s*$' removes the empty rows; add a grep -vE 'box-drawing or status-bar pattern' if your assertion still drags in chrome.
4. Right-size the pane. Use the smallest -x/-y that still fits the assertion without wrapping or truncating the target line. A 120x24 pane is far cheaper than 200x50; only go wide when a long line would otherwise wrap.
5. Never pipe escape sequences in. Omit -e (it's off by default) - escape codes multiply tokens and read as noise.
6. Go out-of-band for long runs. capture-pane -p > /tmp/pane.txt (or tmux pipe-pane) and then grep/grep -A2 the FILE, surfacing only matches - so the raw screen never enters context at all.

Anti-patterns

• Forgetting to source env in the pane. The session boots, the first turn hangs on a 401/timeout, and the pane looks "stuck" for no obvious reason.
• Fixed sleeps as the only synchronization. Works once, flakes the next run. Poll capture-pane for a sentinel.
• Dumping the whole pane into context. A raw capture-pane -p of a tall pane is mostly blank rows and chrome - grep/tail to a verdict instead (see "Keep the caller's token cost down").
• Scraping for something the TUI never renders. Cache-safe reminders, internal state - instrument the handler instead.
• A too-narrow pane. capture-pane returns the wrapped/truncated view; set -x/-y wide enough for your assertion.
• Leaking sessions. Always tmux kill-session -t "$S" (and clean the temp PI_CODING_AGENT_DIR) so a failed run doesn't leave a model-backed pane running.
• Using tmux for scriptable behavior. If success is a file on disk or a structured event, the SDK / pi --print --mode json is faster and far less flaky.

References

• ../feature-eval-author/SKILL.md - the eval playbook; driver-choice table (SDK vs --print vs tmux) and the READ-vs-ACT framing. This skill is its interactive fallback.
• extensions/memory.md - the capture nudge (PI_MEMORY_CAPTURE_TURN) and what arms on session_before_compact, the canonical reason to need a real /compact.