# Three Layer Testing

> Run unit + integration tests after every meaningful code change, and add system/E2E only when the change has broad architectural impact. After each coding task the skill executes the unit and integration layers from fast to slow, fixes every failure with a test-first regression, and produces a PASS/FAIL summary plus a bug-triage table. The system/E2E layer is **conditionally** triggered — only when the diff touches architecture, schema, auth, middleware stack, public routes, or multi-module flows; small/localized changes stop at integration. Use after the user finishes implementing a feature, bug fix, refactor, or dependency bump; when the user explicitly asks 跑测试 / 回归 / 全量测试 / run tests / regression / test cycle; or as the final verification step before concluding any coding task. The skill auto-discovers the project's test framework (pytest, npm test, go test, cargo test, etc.), skips E2E gracefully when runtime deps are missing, and enforces "no new failures and no pass-rate regression" as the exit criteri

- Skill: `playerrch/three-layer-testing` (Agent Skill)
- Install (CLI): `npx skillmds@latest add playerrch/three-layer-testing`
- Raw SKILL.md: https://api.skillmd.com/api/skills/playerrch/three-layer-testing/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Integrations & APIs
- Author: playerrch (https://skillmd.com/u/playerrch)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/playerrch/three-layer-testing

---


# Three-Layer Testing Cycle

Every non-trivial code change ends with one pass of this cycle. No exceptions.

## When to run

Run this skill when **any** of the following is true:

- The agent just finished writing or modifying code (new feature, bug fix,
  refactor, dep upgrade, config change that affects runtime).
- The user says 跑测试 / 回归 / 全量测试 / run tests / regression / test cycle /
  check if nothing is broken.
- Closing a task or PR-preparation moment.

Skip only if the change is **pure docs / pure comments**, or the user explicitly
says "don't run tests".

## Philosophy

Three concentric rings, fast → slow, cheap → expensive:

| Layer | Scope | Speed | External deps | Run frequency |
|-------|-------|-------|---------------|---------------|
| **Unit** | Pure functions/classes, isolated | seconds | none (all I/O mocked) | **every change** |
| **Integration** | Wired components (route + middleware + DB) but in-process | <5 min | temp DB only; external HTTP mocked | **every change** |
| **System (E2E)** | Real running service + real browser/CLI | minutes | real backend + (optional) browser | **conditional** (see "E2E trigger matrix" below) |

Unit catches most regressions for 1% of the time. Integration pins down the
wiring. E2E certifies the user-visible experience — but it is expensive and
flaky, so it only runs when the change is broad enough to warrant it.

## Mandatory workflow

Copy this checklist and tick as you go. Step 4 is conditional — if not
triggered, explicitly mark it `skipped (out of trigger scope)` in the report.

```
Testing cycle:
- [ ] Step 1  Discover framework + existing tests
- [ ] Step 2  Unit layer          (always; must pass)
- [ ] Step 3  Integration layer   (always; must pass)
- [ ] Step 4  System / E2E        (only when trigger matrix matches)
- [ ] Step 5  Triage + fix (test-first regression for every bug)
- [ ] Step 6  Report PASS/FAIL summary + bug table
```

Do not mark a step "done" without real tool output.

## E2E trigger matrix

Before Step 4, evaluate the change against this matrix. Run E2E **only if at
least one row matches**.

| Trigger | Examples |
|---------|----------|
| **Architecture change** | Middleware chain modified, router registration refactored, DI/container swap, deploy-mode branching, new top-level lifecycle hooks |
| **Data-model / schema change** | New table, column add/remove/rename, migration added, foreign-key topology change |
| **Auth / authz change** | JWT claim shape, RBAC permission code changes, new role, ownership check modification, password policy, session/token lifecycle |
| **New public endpoint or page** | A route a user or browser can reach that didn't exist before |
| **Cross-module refactor** | A single change touches 3+ modules, or renames a type/symbol used across layers |
| **External-gateway change** | LLM client, payment, email, MCP server, HTTP client swap |
| **Security boundary** | CORS list, CSP, rate-limit rules, IP-block rules, input-sanitization policy |
| **Dependency major bump** | Framework, ORM, auth library major version jump |
| **User explicitly asks** | "跑完整 E2E" / "系统测试" / "run full regression including E2E" |

**Skip E2E** when the change is localized and none of the above triggers apply:

- Bug fix inside a single function with unit regression
- Adding a pure helper / utility
- Tightening a validator pattern
- Docstring / comment / rename inside one file
- Adding tests only

When skipping, the report's System row reads:

> `skipped (no trigger matched: change is localized to <module/file>)`

If the decision is ambiguous, **default to running E2E** — the cost of a
missed regression at that layer is higher than the cost of a 5-minute run.

## Step 1 · Discover the project

Identify test framework + command. Use these heuristics, in order:

| Signal | Command |
|--------|---------|
| `pytest.ini` / `pyproject.toml` with `[tool.pytest.ini_options]` / `tests/` dir | `pytest -q --tb=line` |
| `package.json` with a `"test"` script | `npm test --silent` (or `pnpm test` / `yarn test`) |
| `Cargo.toml` | `cargo test --quiet` |
| `go.mod` + files ending in `_test.go` | `go test ./...` |
| `pom.xml` | `mvn -q test` |
| `build.gradle` | `./gradlew test --quiet` |
| Makefile with `test:` target | `make test` |

If nothing matches: ask the user for the test command **once**; don't guess.

For monorepos: run every sub-project's tests. Do not pick favourites.

Also inspect:

- `tests/` layout — is there a `unit/ integration/ system/` split? If yes, you
  can run each layer separately (e.g. `pytest tests/unit` then
  `pytest tests/integration`). If not, the layer distinction lives inside
  marker annotations (`@pytest.mark.unit`, etc.) or filename prefixes.
- CI config (`.github/workflows/*.yml`, `.gitlab-ci.yml`, `Jenkinsfile`) —
  the CI command is the canonical one; mirror it.

## Step 2 · Unit layer

- Must run every cycle.
- If missing (no unit tests exist): flag it, write at least minimum unit tests
  for the code you just touched, then run.
- A passing unit suite that ignores the code you changed is not a pass — verify
  at least one assertion touches your change (coverage or grep).

Typical commands:

```bash
# pytest
pytest tests/unit -q --tb=line

# jest
npx jest --testPathPattern=unit --silent

# go
go test -run Unit ./... -count=1
```

## Step 3 · Integration layer

- Uses an in-process client against the full app (TestClient / supertest /
  httptest / etc.) with an **isolated temp DB** — never the dev/production DB.
- All outbound HTTP to third-party services is mocked (`respx`, `nock`,
  `wiremock`, `httpmock`…).
- If integration tests don't exist yet: this is the highest-value layer to
  invest in; write at least one end-to-end happy-path plus the RBAC/error
  branches relevant to your change.

Typical commands:

```bash
pytest tests/integration -q --tb=line

npx jest --testPathPattern=integration --silent
```

## Step 4 · System / E2E (conditional)

First check the **E2E trigger matrix** above. If no trigger matches, **skip
this step** and record the skip reason in the report. Do not run E2E out of
habit on localized changes.

When triggered, run only if **all** of the following are true:

1. A running service is reachable (check health endpoint / listening port).
2. A browser automation entry point is available — Cursor browser MCP
   (`cursor-ide-browser`), Playwright, Selenium, or a CLI equivalent.
3. Test fixtures/seed for known accounts exist (or the user authorises direct
   DB manipulation via a seed script).

If any runtime dep is missing, mark the system layer **skipped** in the report
with a one-line reason. Do not block on it.

When running:

- Use a dedicated seed script to provision test accounts (`seed` command with
  idempotent reset); never log in with the user's personal credentials.
- Clean up afterwards (`cleanup` / fixture teardown) unless the account is
  intended to be long-lived.
- Focus journeys on the area the change affects (RBAC, auth flow, schema-bound
  page, etc.) — not necessarily every page. Full-site smoke is optional.
- Capture at least one screenshot per journey for the report.

## Step 5 · Triage and fix

### R1. Test-first regression (non-negotiable for any bug found)

For every failure / bug discovered:

1. Write a minimal test that **reproduces the failure** (a new test, or tighten
   an existing one). Run it — confirm RED.
2. Edit the production code to make that test GREEN.
3. Re-run the **narrowest** layer that contains the fix. If GREEN, run the
   wider suite.
4. Add a short comment on the test linking it to the bug ID
   (`# Regression: BUG-P0-01 ...`).

Never modify an existing passing test to accept the buggy behaviour. Never add
an exception that silences a failure.

### R2. Severity triage

| P0 | Functional or security breakage. **Stop everything.** Fix now + regression test + re-run full cycle. |
| P1 | Consistency/UX/stability issue that has a workaround. Fix if < 15 min, else record in the bug table. |
| P2 | Cosmetic, dead code, deprecation noise. Record only. |

### R3. Classification matrix for failures

| Failure cause | Action |
|---------------|--------|
| Real bug in production code | Apply R1 (test-first fix) |
| Flaky test (non-deterministic) | Repeat run 3×; if flaky, mark `xfail(strict=False)` with TODO, don't mask with retry |
| Test is wrong (outdated assertion) | Fix the test, add a short note why |
| Environment / setup issue | Fix setup (fixture, CI config, seed script) |
| External dep broken | Mock it; if already mocked, fix the mock |

### R4. Pass-rate ratchet

After your changes, total passed count must be `>=` the pre-change count. If it
regresses (e.g. a previously passing test now fails and you can't figure out
why), stop and investigate. Do not commit with lower pass count than before.

## Step 6 · Report

Produce a concise report in the chat. Copy the template below:

```markdown
## Test run summary (three-layer)

| Layer | Tests | Passed | Failed | XFail | Skipped | Time |
|-------|-------|--------|--------|-------|---------|------|
| Unit        | … | … | … | … | … | … s |
| Integration | … | … | … | … | … | … s |
| System      | … | … | … | … | … | … s *or* `skipped (out of trigger scope: <reason>)` |
| **Total**   | … | … | … | … | … | … s |

When System is skipped by the trigger matrix, also print one line explaining
which rule applied, e.g.:
`System E2E skipped — change is a 1-function fix in utils/sanitize.py, no trigger matched.`

### Bugs discovered & fixed this cycle

| ID | Severity | Symptom (1 line) | Root cause (1 line) | Fix file(s) | Regression test |
|----|----------|------------------|----------------------|-------------|-----------------|
| BUG-P0-01 | P0 | … | … | `foo/bar.py` | `tests/unit/test_bar.py::test_repro` |

### Still failing / known issues

- …  (or: none)

### Next suggested action

-   e.g. "run cycle on CI to confirm", or "no further action"
```

If everything PASSes and nothing was fixed, the table collapses to a single
line — that is fine, still produce it.

## Hard rules (apply always)

1. **No fake PASS.** Never claim 测试通过 / tests pass without the actual
   command output in the same turn. Cite exit code / summary line.
2. **Mock all external I/O** in unit + integration. If a test hits a real
   network endpoint, it belongs in E2E.
3. **Never touch production / user data.** Use a temp DB (`tempfile`, env
   `DATABASE_URL=sqlite:///<tmp>`) or a dedicated test schema.
4. **No `-x` / fail-fast** on the full suite. You want the full failure set
   for triage, not the first one.
5. **Deterministic time** — if a test depends on `now()`, mock the clock
   (`freezegun`, `sinon.useFakeTimers`, `time.freeze`).
6. **Idempotent seed/cleanup** — seed scripts should be safe to run 2× in a
   row without errors.
7. **No emojis in code or test output files** unless the user asked. Keep
   terminal output and commit messages ASCII-friendly.

## Common anti-patterns to catch

- Tests that `sleep()` instead of waiting on a signal → replace with
  `pytest-asyncio`/`waitUntil`/signal.
- `try: ... except Exception: pass` inside a test — silences real failures.
- Shared mutable module state between tests — add an autouse reset fixture.
- Rate-limiter / timer fixtures that leak into the next test — reset in
  `autouse` fixture.
- E2E that logs in with the user's real credentials — forbidden; always use a
  dedicated `test_*` account managed by a seed script.
- Running E2E against production URLs — forbidden.

## Degraded modes

| Situation | Degraded behaviour |
|-----------|---------------------|
| No browser MCP, no Playwright installed | Skip system layer; report `skipped (no e2e runtime)` |
| Service not running on expected port | Attempt one start; if fails, skip system; report reason |
| External service (LLM, payment, email) unreachable | Confirm mocks are in place; if test was a "live" E2E probe, mark skipped |
| Flaky test after 3 repeats | Mark `xfail(strict=False)` with `# TODO: investigate flake, see BUG-…` |
| New failure you cannot diagnose in 10 min | Report as open BUG with repro; do NOT mask |

## Summary

**Every code change → one full cycle → fix-with-regression-test → report.**

The skill is done when:

- Unit PASS.
- Integration PASS.
- System PASS **or** explicitly skipped with a reason — out-of-trigger-scope
  for localized changes is a valid reason; missing runtime deps is also valid.
- Every discovered bug has both a code fix and a failing-then-passing
  regression test.
- A written report in the chat quoting real command output.