Write Vibe Tests

Write Vibe Tests

Use this skill to make Vibe architecture testable, not just well-shaped. Tests should protect observable behavior while allowing internals to move.

Core Principle

Test behavior through stable boundaries. Do not couple tests to private methods, internal call choreography, or temporary structure.

If a refactor changes no observable behavior but breaks many tests, the tests are probably coupled to implementation details.

Vibe Test Boundaries

Prefer Fakes Over Mocks

Prefer in-memory implementations and fake adapters that implement real contracts.

• Put reusable test doubles in tests/stubs/ and name them Fake*.
• Make fakes small and behavior-oriented.
• Mock only at hard process, network, time, or third-party boundaries when a fake would be more complex than the behavior under test.
• Avoid assertions like "method X was called with Y" unless the call itself is the observable contract.

Legacy Or Refactor Workflow

When code is hard to test:

1. Find the smallest seam: function boundary, constructor dependency, protocol/port, wrapper, composition root, feature flag, or module boundary.
2. Add characterization tests through the nearest public entry point.
3. Capture current observable behavior, even if awkward.
4. Refactor behind the seam in small steps.
5. Replace broad characterization checks with clearer behavior/spec tests as the design improves.

Prefer "make it testable" refactors first: isolate I/O, extract pure functions, introduce ports/adapters where useful, or move construction out of business logic.

Test Shape

• Stack: pytest + pytest-asyncio + pytest-textual-snapshot + respx.
• Use descriptive test names; do not add test docstrings. Pytest displays docstrings instead of node IDs when present, which hurts.
• Arrange, act, and assert clearly, but optimize for readability over ceremony.
• Keep tests deterministic, fast, and explicit about failure.
• Use autouse fixtures from tests/conftest.py (config_dir, tmp_working_directory) for config/home/working-directory isolation.
• Mark async tests with @pytest.mark.asyncio.
• Mock outbound HTTP with respx.
• Use the narrowest relevant test first, then broaden when shared contracts are touched.
• Tests are exempt from the ANN and PLR ruff rules (see per-file-ignores).
• Changes to managed shell tools, managed shell backends, shell prompts, or shell dependencies require focused POSIX/common validation and passing Windows shell CI. If Windows shell CI is unavailable, the PR notes must include manual native Windows commands and results.
• After app-server session behavior, backend adapters, or shared contract-fixture changes, run both backend contract commands. A failing Unified frontier is expected before parity; include its complete pytest summary in parity-related PRs.

Avoid

• Testing private methods as the primary coverage for behavior.
• Asserting intermediate internal state when user-visible output, emitted events, files, or return values can be asserted.
• Building mocks that mirror the implementation.
• Adding abstractions only to satisfy a test.
• Writing tests that require a specific internal file split or call order when the domain behavior is unchanged.

Verification

After Python test/code changes, run:

uv run ruff format .
uv run ruff check --fix .

Then run the targeted tests:

uv run pytest <test-path-or-node-id>

Run uv run pyright when signatures, models, protocols, or shared contracts changed.