Genai Io Gen Code Gen Code

QA — Feature Regression Testing

Run automated and interactive regression tests for a San feature by name.

Arguments

• <feature> — Feature name, e.g. agent, cli-startup, loop, session, hook.
• --interactive — Also run interactive tmux tests (default: automated only).
• --pane — For interactive tests, split a pane in the current tmux window instead of creating a separate session. Lets the user watch the test live.

Workflow

1. Resolve the feature doc

Look for the feature in this order:

1. docs/reference/<feature>.md — reference docs, may have Automated Tests and Interactive Tests (tmux) sections
2. docs/packages/2-feature/<feature>.md — feature package docs, has a ## Tests section

Read the file. Extract:

• Automated Tests section — the go test commands and known test cases (from reference docs).
• Interactive Tests (tmux) section — the step-by-step tmux script (from reference docs).
• Tests section — package-level test info from feature docs.

If the feature matches an integration test directory name (tests/integration/<feature>/), add go test ./tests/integration/<feature>/... as a default test command even if the doc doesn't list one.

If the feature cannot be found in any doc path, list all available features (integration test dirs + doc stems from both locations) and ask the user to pick one.

2. Build the binary

Always build first so tests run against the latest code:

make build

If the build fails, stop and report the error — no point running tests against stale code.

3. Run automated tests

Execute every go test command extracted from the feature doc's Automated Tests section, plus the default integration test command if applicable. Capture stdout/stderr and exit codes. Each command is a separate test group.

Record results as:

• PASS — exit code 0
• FAIL — exit code non-zero; include the failure output

4. Run interactive tests (only with --interactive or --pane)

Skip this phase unless the user explicitly opts in.

Pane mode (--pane)

Split a pane in the current tmux window for the test:

# Detect current session and window
TMUX_TARGET=$(tmux display-message -p '#{session_name}:#{window_index}')
# Split horizontally, 50% width
tmux split-window -h -t "$TMUX_TARGET" -l '50%'

Run all interactive test steps in that pane via tmux send-keys. Use tmux capture-pane -p to read the pane output after each step and verify the Expected comments from the doc.

Session mode (default when --interactive without --pane)

Create a dedicated tmux session:

tmux new-session -d -s qa_<feature> -x 220 -y 60

Run the interactive test steps there. Kill the session when done.

Interpreting interactive results

For each test step:

1. Send the command via tmux send-keys.
2. Wait the prescribed sleep duration (or a reasonable default).
3. Capture the pane with tmux capture-pane -t <target> -p.
4. Check the captured output against the Expected comment.
5. Record PASS/FAIL with the captured output as evidence.

After all steps, clean up (kill session/pane, remove temp files) as specified in the doc's cleanup section.

5. Generate the report

Print a summary table:

## QA Report: Feature — <Name>

### Automated Tests
| Test Group               | Result |
|--------------------------|--------|
| go test ./internal/x/... | PASS   |
| go test ./tests/int/...  | FAIL   |

### Interactive Tests
| Step                         | Result | Notes                    |
|------------------------------|--------|--------------------------|
| Test 1: Basic TUI startup    | PASS   | TUI with input box shown |
| Test 2: Print mode           | FAIL   | No output after 15s      |

### Summary
Automated: 2/3 passed
Interactive: 3/4 passed
Overall: PARTIAL PASS

Use PASS, FAIL, or SKIP for each item. Include failure details inline so the user can act on them immediately.

Important notes

• The binary path is ./bin/san (built by make build), not the installed san.
• Interactive tests that require an LLM response need a configured provider. If no provider is set, mark those steps as SKIP with a note.
• For interactive tests, use Ctrl+C (C-c) to exit san, not q (which gets interpreted as user input).
• Always clean up temp files and tmux sessions/panes after testing, even on failure.
• If a test step times out (no expected output after the sleep), mark it FAIL and move on — don't hang.

Source: genai-io/gen-code — distributed by TomeVault.