Test Architecture

Test Architecture — Advanced Testing Patterns

Use this skill for advanced testing beyond unit tests. Covers E2E, load testing, mutation testing, contract testing, and visual regression. Complements tdd-with-agents (which covers unit tests).

Relationship with tdd-with-agents

1. E2E Testing (Playwright)

Setup

# tests/e2e/conftest.py
import pytest
from playwright.async_api import async_playwright

@pytest.fixture
async def browser():
    async with async_playwright() as p:
        browser = await p.chromium.launch(headless=True)
        yield browser
        await browser.close()

@pytest.fixture
async def page(browser):
    context = await browser.new_context()
    page = await context.new_page()
    yield page
    await context.close()

Test Pattern

# tests/e2e/test_user_flow.py
import pytest

@pytest.mark.asyncio
async def test_user_login_flow(page):
    """E2E: User can log in and see dashboard."""
    await page.goto("http://localhost:8000/login")
    await page.fill('input[name="email"]', "test@example.com")
    await page.fill('input[name="password"]', "password123")
    await page.click('button[type="submit"]')
    
    # Verify redirect to dashboard
    await page.wait_for_url("**/dashboard")
    assert await page.title() == "Dashboard"
    
    # Verify user info displayed
    await page.wait_for_selector(".user-name")
    user_name = await page.text_content(".user-name")
    assert "Test User" in user_name

Running E2E Tests

# Start app first
uvicorn app.main:app --reload &

# Run E2E tests
pytest tests/e2e/ -v --asyncio-mode=auto

2. Load Testing (k6)

k6 Script

// tests/load/api_load_test.js
import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  stages: [
    { duration: '30s', target: 20 },   // Ramp up to 20 users
    { duration: '1m', target: 20 },    // Stay at 20 users
    { duration: '30s', target: 50 },   // Spike to 50 users
    { duration: '1m', target: 50 },    // Stay at 50 users
    { duration: '30s', target: 0 },    // Ramp down
  ],
  thresholds: {
    http_req_duration: ['p(95)<500'],  // 95% of requests < 500ms
    http_req_failed: ['rate<0.01'],    // Error rate < 1%
  },
};

export default function () {
  const res = http.get('http://localhost:8000/api/products');
  check(res, {
    'status is 200': (r) => r.status === 200,
    'response time < 500ms': (r) => r.timings.duration < 500,
  });
  sleep(1);
}

Running Load Tests

# Install k6
# See https://k6.io/docs/getting-started/installation/

# Run load test
k6 run tests/load/api_load_test.js

# Run with cloud reporting
k6 cloud tests/load/api_load_test.js

Locust Alternative (Python)

# tests/load/locustfile.py
from locust import HttpUser, task, between

class APIUser(HttpUser):
    wait_time = between(1, 3)
    
    @task(3)
    def get_products(self):
        self.client.get("/api/products?page=1&limit=20")
    
    @task(1)
    def create_product(self):
        self.client.post("/api/products", json={
            "name": "Test Product",
            "price": 9.99
        })

# Run Locust
locust -f tests/load/locustfile.py --host=http://localhost:8000

3. Mutation Testing

Mutation testing verifies that your tests actually test something by mutating the code and checking if tests catch the mutations.

Setup (Python)

pip install mutmut

Running

# Run mutation testing
mutmut run --paths-to-mutate=src/services/

# Show mutations that survived (tests didn't catch)
mutmut results

Interpreting Results

mutmut results

- Killed mutants: 85 (tests caught the mutation)
- Survived mutants: 15 (tests didn't catch the mutation)
- Total mutants: 100

Survived mutants indicate gaps in test coverage.
Review each survived mutant and add tests.

4. Contract Testing (Pact)

Contract testing ensures that API consumers and providers agree on the contract.

Consumer Test (Python)

# tests/contract/test_review_consumer.py
import pytest
from pact import Consumer, Provider

pact = Consumer('ReviewFrontend').has_pact_with(Provider('ReviewAPI'))

def test_get_reviews_contract():
    expected = {
        'reviews': each_like({
            'id': like(1),
            'product_id': like(100),
            'rating': like(5),
            'comment': like('Great product!')
        })
    }
    
    (pact.given("reviews exist")
     .upon_receiving("a request for reviews")
     .with_request("GET", "/api/reviews", query={"product_id": "100"})
     .will_respond_with(200, body=expected))
    
    with pact:
        result = requests.get(f"{pact.uri}/api/reviews", params={"product_id": "100"})
    
    assert result.json()['reviews'][0]['rating'] == 5

Running Contract Tests

# Generate pact file
pytest tests/contract/ -v

# Verify pact file against provider
pact-verifier --provider-base-url=http://localhost:8000 \
  --pact-url=pacts/ReviewFrontend-ReviewAPI.json

5. Visual Regression Testing

Setup (Playwright)

# tests/visual/test_review_card.py
import pytest

@pytest.mark.asyncio
async def test_review_card_visual(page):
    """Visual regression: ReviewCard renders correctly."""
    await page.goto("http://localhost:3000/reviews")
    
    # Take screenshot and compare with baseline
    await page.screenshot(
        path="tests/visual/baseline/review-card.png",
        full_page=True
    )
    
    # Compare with baseline (using pixelmatch or similar)
    # This is typically done in CI with a visual diff tool

Running Visual Tests

# Run visual tests
pytest tests/visual/ -v

# In CI, use Percy, Chromatic, or Playwright's built-in comparison
npx playwright test --update-snapshots  # Update baselines

Test Strategy Matrix

Integration with Hermes and Themis

Hermes uses this skill during implementation to write advanced tests. Themis uses this skill during review to validate test quality beyond unit coverage.