Project Testing

Testing

Core Principle

Untested code is broken code. The operator does not manually test for the assistant. Every behavior the project ships has at least one automated test exercising it. Manual UI walkthroughs are diagnostics for the assistant; they are not proof of correctness. If a behavior cannot be tested, the design is wrong — refactor until it can be.

Tests are written before the implementation (see project-workflow). A failing test is the executable contract; the implementation makes it pass without weakening it.

Test Commands

# All Go tests (run during regular dev)
go test ./...

# With race detector (required in CI; also run before commits touching concurrency)
go test -race ./...

# One adapter
go test ./internal/adapters/aiagent_v3

# Update golden files for a single adapter
go test ./internal/adapters/aiagent_v3 -update-golden

# Coverage
go test -cover ./...
go test -coverprofile=coverage.out ./... && go tool cover -html=coverage.out

# Benchmarks
go test ./internal/adapters/... -bench=. -benchmem

# Frontend
cd frontend && npm test                  # vitest watch mode
cd frontend && npm test -- --run         # one-shot
cd frontend && npm run e2e               # playwright headless
AI_VIEWER_E2E_PORT=17710 npm run e2e     # use a different local E2E port
cd frontend && npm run e2e -- --headed   # playwright debug
cd frontend && npm run e2e -- --ui       # playwright UI mode

# Full pre-commit check
./scripts/lint.sh && go test -race ./... && cd frontend && npm test -- --run && npm run lint

Test Pyramid

Coverage Thresholds (enforced)

scripts/check-coverage.sh enforces statement coverage on the gated internal/* set (/cmd/ excluded) against coverage.out. Branch coverage and new-code-in-PR ≥ 90% are deferred (see quality-gates.md "Go — Coverage"). Lowering a threshold to land a PR is a contract breach; either add tests or split the PR.

Mandatory Test Kinds

• Every adapter exposes at least one FuzzXxx target covering its parse path.
• internal/canonical exposes no FuzzXxx target — it owns no decoders; all untrusted-bytes parsing (and thus all fuzzing) lives in the adapters.
• Performance-critical paths have benchmarks (bench/baseline.txt baseline). scripts/check-bench.sh runs -p=1 -count=6 -cpu=1 (package benchmark binaries serialized, benchstat's 0.95-CI minimum, serial hot-path CPU list) and gates a statistically-significant > 20% sec/op regression per benchmark. It is a local/workstation gate (the workstation baseline is not comparable to CI-runner hardware), so CI runs only the bench compile-smoke + the gate's hardware-independent self-test. Real-mode benchmark runs fail closed with exit 2 before sampling when host load is too high for valid wall-time evidence or when load/CPU preflight evidence is unavailable.
• Concurrency-touching code is stress-tested with scripts/test.sh --stress 10 (-race -count=10) locally; CI runs -count=1 per push and -count=10 race stress nightly.
• Frontend E2E covers golden paths AND error states (network failure, empty list, malformed SSE event). axe runs on every route.
• Every UI change has at least one Playwright assertion that proves the behavior end-to-end. Component tests alone are not sufficient for user-visible behavior.

Fixture Management

Adapter fixtures live under testdata/<adapter>/<scenario>/INPUT/ with expected canonical events at testdata/<adapter>/<scenario>/expected.jsonl.

Sanitization is mandatory (see .agents/sow/specs/security.md and testing-strategy.md):

• Replace originId/sessionId with stable test UUIDs.
• Strip user messages → [REDACTED_USER_MESSAGE].
• Strip tool I/O → [REDACTED_TOOL_OUTPUT].
• Replace API URLs with https://api.example.invalid/....
• Replace API keys with [REDACTED_SECRET].

Use scripts/sanitize-fixture.sh <input-file> <output-dir> (built during Phase 1).

CI grep-scans testdata/ for common secret patterns and fails on hits.

Writing a New Test

For Go:

func TestX(t *testing.T) {
    t.Parallel()
    // arrange
    // act
    // assert with cmp.Diff for structs/slices
}

For frontend:

import { render, screen } from '@testing-library/react';
import { describe, it, expect } from 'vitest';

describe('SessionRow', () => {
  it('renders status badge', () => {
    render(<SessionRow session={mockFailedSession} />);
    expect(screen.getByRole('status', { name: /failed/i })).toBeInTheDocument();
  });
});

CI Gates (zero tolerance)

The full catalog lives in project-quality-gates. A PR cannot land if any gate fails:

• Any Go test fails (race detector enabled).
• Any frontend test or Playwright spec fails.
• Any lint warning exists (Go or frontend).
• Coverage threshold missed on changed code.
• Any fuzz target crashed during its run.
• Any benchmark regresses > 20% from baseline.
• Any committed fixture trips the secret scanner.
• Any spec is stale relative to code (scripts/spec-drift.sh catches the structural indicators; manual audit covers prose).
• Axe accessibility violations at serious/critical level.
• Frontend bundle exceeds size budget.

Never add t.Skip, // nolint, it.skip, or test.skip to land a PR. Either fix the underlying issue or open a SOW that explicitly justifies the suppression with an expiry condition.

Debugging Failing Tests

1. Run the single failing test with -v: go test -v -run TestName ./pkg.
2. For race conditions: re-run with -count=10 -race.
3. For flaky frontend E2E: re-run with --headed --debug to step through.
4. Check the CI log for the full output; many local-pass / CI-fail issues are platform-specific (path separators, time zones).

Adding a New Test Scenario for an Adapter

1. Capture a real session (in the operator's local environment) demonstrating the scenario.
2. Sanitize via scripts/sanitize-fixture.sh.
3. Place under testdata/<adapter>/<scenario>/INPUT/.
4. Run go test ./internal/adapters/<adapter> -update-golden to generate expected.jsonl.
5. Review the golden file manually. Does every event look right? Are timestamps reasonable? Are sub-agent links correct?
6. Commit fixture + golden + a test case in adapter_test.go if not auto-discovered.

Reporting Honest Test Status

When reporting to the operator, never say "code works" without:

• The test file paths covering the new behavior.
• The gate command outputs (./scripts/gates.sh summary plus any focused reruns used for diagnosis).
• The external review status.

Honest phrasings:

• "Code written, tests passing, gates green, review pending." — work is mid-flight.
• "Code written, tests passing, gates green, review converged — ready." — work is done.
• "Code written, behavior X not yet covered by a test — not ready." — work is incomplete.

Cross-References

• Workflow: .agents/skills/project-workflow/SKILL.md
• Gates: .agents/skills/project-quality-gates/SKILL.md
• Spec: .agents/sow/specs/testing-strategy.md