E2e Test Thinker

E2E Test Thinker

Analyze the current branch's changes and determine whether — and how — they can be covered by CLI-based end-to-end tests using the uc-e2e-tests harness.

When to Use

• After completing a feature or bug fix, before opening a PR
• When the user asks "should I write e2e tests for this?" or "can this be tested via CLI?"
• When reviewing a branch for test coverage gaps
• Triggered by: "e2e test", "端到端测试", "think about e2e", "e2e coverage", "CLI test plan"

Background: The E2E Harness

The harness lives at tests/e2e/ and provides three building blocks:

Test pattern: every test is #[tokio::test] #[ignore] and runs with cargo test -p uc-e2e-tests -- --ignored. Tests are black-box: they only interact via the CLI binary and HTTP health endpoint.

Existing test files (for reference and dedup):

Procedure

Phase 1: Gather the Diff

Run these commands to understand what changed on the current branch:

# What files changed?
git diff main...HEAD --stat

# Full diff for analysis
git diff main...HEAD

# Commit messages for intent
git log main..HEAD --oneline

If the branch IS main (no divergence), fall back to git diff HEAD~5..HEAD or ask the user which range to analyze.

Phase 2: Classify Each Change

For every changed file/module, classify it into one of these buckets:

Phase 3: Check for Existing Coverage

For each ✅/⚠️ item, grep existing test files to see if it's already covered:

grep -r "relevant_command_or_keyword" tests/e2e/tests/

Report what's covered and what's not.

Phase 4: Design Test Cases

For each uncovered testable change, produce a test case sketch following these rules:

Structure rules:

1. One #[tokio::test] #[ignore] async fn test_<descriptive_name>() per scenario
2. Use TestProfile::new("unique-slug") for isolation
3. Use TestDaemon::start(profile).await to spawn daemon
4. Use TestCli::new(&daemon.profile) for CLI commands
5. Always init before testing business commands (unless testing pre-init behavior)
6. Use run_capture() for assertions on exit code, stdout, stderr
7. Use --json flag when testing structured output
8. Parse JSON with serde_json::from_str::<serde_json::Value>() for schema assertions

Assertion patterns:

• Exit code: assert!(output.success()) or assert!(!output.success())
• JSON field: json.get("field").and_then(|v| v.as_str())
• Contains: combined.contains("expected")
• Valid JSON: serde_json::from_str(output.stdout.trim()).is_ok()

Decide which test file the new case belongs in:

• Fits an existing file's theme → add to that file
• New feature area → propose a new file name

Phase 5: Output the Plan

Present the results as:

## E2E Test Analysis for branch `<branch-name>`

### Changes Summary
- <one-line per changed area>

### Not Testable via CLI
- <item>: <reason>

### Already Covered
- <item>: covered by `<test_file>::<test_fn>`

### Proposed New Tests

#### 1. `test_<name>` → `<target_file>.rs`

**What it tests:** <one sentence>

**Sketch:**
```rust
#[tokio::test]
#[ignore]
async fn test_<name>() {
    // ... concrete test code ...
}
```text

**Confidence:** High / Medium / Low
**Why:** <why this confidence level>

Phase 6: Ask Before Writing

After presenting the plan, ask the user:

1. Which tests to actually write (all / subset / none)
2. Whether to create new test files or append to existing ones
3. Any edge cases they want added

Do NOT write test code until the user confirms.

Decision Criteria: When NOT to Propose E2E Tests

Skip proposing e2e tests when:

• The change is purely GUI (React/Tauri commands) with no CLI surface
• The change is a docs/CI/build-only change
• The change is an internal refactor where existing tests provide regression coverage
• The CLI binary doesn't expose the changed behavior (e.g., internal daemon-to-daemon protocol change with no CLI observability)

In these cases, explain WHY and suggest the appropriate test layer (unit test, integration test, manual test, GUI test).

Common Pitfalls

• Don't test daemon internals — e2e is black-box. If you need to assert internal state, that's a unit/integration test.
• Don't parse human-readable CLI output with exact string matching — use --json mode and parse structured output.
• Don't forget profile isolation — every test MUST use a unique TestProfile. Shared profiles cause flaky parallel test runs.
• Don't assume network — single-node tests don't need network. Dual-node tests use localhost loopback only.
• Don't test timing — avoid sleep-based assertions. Poll with a deadline loop if you need to wait for async behavior.