Pytest Patterns

Pytest Patterns

Modern pytest testing patterns, anti-patterns, and quality rules for Python.

pytest_smell.py detects structural test smells deterministically. The prose sections below cover judgment-level guidance that the script cannot evaluate. Never re-derive smell detection by eye when the script can do it; always run the script for consistent, auditable findings.

Core Principles

Patterns vs Anti-Patterns

Fixtures

# GOOD: Fixtures for test dependencies
@pytest.fixture
def db_session():
    """Provide a clean database session."""
    session = create_session()
    yield session
    session.rollback()
    session.close()

@pytest.fixture
def sample_user(db_session):
    """Create a sample user for tests."""
    user = User(name="John", email="john@example.com")
    db_session.add(user)
    db_session.commit()
    return user

def test_get_user(db_session, sample_user):
    result = get_user_by_id(db_session, sample_user.id)
    assert result.name == "John"

# BAD: Setup in each test
def test_get_user():
    session = create_session()
    user = User(name="John", email="john@example.com")
    session.add(user)
    session.commit()
    # ... test code
    session.rollback()  # Easy to forget cleanup!

Fixture Scopes

# GOOD: Appropriate fixture scopes
@pytest.fixture(scope="session")
def docker_db():
    """Start database container once per test session."""
    container = start_postgres_container()
    yield container
    container.stop()

@pytest.fixture(scope="module")
def api_client():
    """Create API client once per test module."""
    return APIClient(base_url="http://test.local")

@pytest.fixture  # Default: function scope
def clean_data(db_session):
    """Fresh data for each test."""
    yield
    db_session.query(User).delete()

# BAD: Wrong scope causes test pollution
@pytest.fixture(scope="session")  # DANGER!
def user():
    return User(name="John")  # Shared across all tests!

Parametrization

# GOOD: Parametrize for multiple cases
@pytest.mark.parametrize("email,valid", [
    ("user@example.com", True),
    ("user.name@example.co.uk", True),
    ("invalid-email", False),
    ("@missing.local", False),
    ("", False),
])
def test_email_validation(email, valid):
    assert validate_email(email) == valid

# GOOD: Multiple parameters with IDs
@pytest.mark.parametrize(
    "input_data,expected",
    [
        pytest.param({"name": "John"}, True, id="valid-name"),
        pytest.param({"name": ""}, False, id="empty-name"),
        pytest.param({}, False, id="missing-name"),
    ]
)
def test_user_validation(input_data, expected):
    assert is_valid_user(input_data) == expected

# BAD: Repeated test code
def test_valid_email():
    assert validate_email("user@example.com") is True

def test_valid_email_with_subdomain():
    assert validate_email("user@sub.example.com") is True

def test_invalid_email():
    assert validate_email("invalid") is False
# ... 10 more nearly identical tests

Mocking

A mock-call assertion (.assert_called_*) is only a valid substitute for a real assertion at an OUTBOUND boundary the test owns — an HTTP request the code constructs, an event published to an external system — where the call itself IS the observable. It is never a substitute for observing a write path's actual effect; see "Overuse of mocking" below and the write-path rule in qa.md.

# GOOD: Outbound HTTP call — the request IS the observable
def test_api_call(mocker):
    mock_get = mocker.patch("requests.get")
    mock_get.return_value.json.return_value = {"data": "test"}

    result = fetch_data("http://api.example.com")

    assert result["data"] == "test"
    mock_get.assert_called_once_with("http://api.example.com")

# GOOD: Context manager for scoped patches
def test_with_timeout(mocker):
    with mocker.patch("time.sleep"):
        result = operation_with_retry()
        assert result is not None

# GOOD: Spy on an outbound side effect (logging) — not a stand-in for verifying a persisted write
def test_logs_error(mocker):
    spy = mocker.spy(logger, "error")

    process_invalid_data()

    spy.assert_called_once()

# BAD: Overuse of mocking — note module.db.save mocked on what should be a write-path test;
# that write should hit a real or in-memory database instead (see qa.md Test Double Taxonomy)
def test_save_user(mocker):
    mocker.patch("module.validate")
    mocker.patch("module.normalize")
    mocker.patch("module.db.save")
    mocker.patch("module.cache.invalidate")
    mocker.patch("module.events.publish")
    # Testing nothing but mocks!

Assertions

# GOOD: Clear, specific assertions
def test_user_creation():
    user = create_user("John", "john@example.com")

    assert user.name == "John"
    assert user.email == "john@example.com"
    assert user.id is not None

# GOOD: Use pytest.raises for exceptions
def test_invalid_email_raises():
    with pytest.raises(ValidationError) as exc_info:
        create_user("John", "invalid-email")

    assert "email" in str(exc_info.value)
    assert exc_info.value.field == "email"

# GOOD: Approximate comparisons
def test_calculation():
    result = complex_calculation()
    assert result == pytest.approx(3.14159, rel=1e-5)

# BAD: Vague assertions
def test_result():
    result = process()
    assert result  # What should it be?
    assert result is not None  # Still unclear

# BAD: Exception testing without context
def test_raises():
    try:
        risky_operation()
        assert False, "Should have raised"
    except Exception:
        pass  # What exception? What message?

Test Organization

# GOOD: Class-based grouping for related tests
class TestUserService:
    """Tests for UserService class."""

    @pytest.fixture(autouse=True)
    def setup(self, db_session):
        self.service = UserService(db_session)
        self.db = db_session

    def test_create_user(self):
        user = self.service.create("John", "john@example.com")
        assert user.id is not None

    def test_create_duplicate_raises(self, sample_user):
        with pytest.raises(DuplicateError):
            self.service.create(sample_user.name, sample_user.email)

# GOOD: conftest.py for shared fixtures
# conftest.py
@pytest.fixture
def app():
    """Create test application."""
    return create_app(testing=True)

@pytest.fixture
def client(app):
    """Create test client."""
    return app.test_client()

Anti-Patterns to Avoid

Test Pollution

# BAD: Shared mutable state
users = []  # Module-level! Shared across tests!

def test_add_user():
    users.append(User("John"))
    assert len(users) == 1  # May fail if other test ran first!

# GOOD: Fresh state per test
@pytest.fixture
def users():
    return []

def test_add_user(users):
    users.append(User("John"))
    assert len(users) == 1  # Always passes

Slow Tests

# BAD: Real network calls
def test_fetch_data():
    result = requests.get("https://api.example.com/data")  # Slow, flaky
    assert result.status_code == 200

# GOOD: Mock external services
def test_fetch_data(mocker):
    mocker.patch("requests.get").return_value.status_code = 200
    result = fetch_external_data()
    assert result is not None

# BAD: Unnecessary sleep
def test_async_operation():
    start_operation()
    time.sleep(5)  # Why?
    assert is_complete()

# GOOD: Poll or use async waiting
def test_async_operation():
    start_operation()
    wait_for(is_complete, timeout=5)
    assert is_complete()

Assertion in Loop

# BAD: Loop hides failures
def test_all_users_valid():
    users = get_all_users()
    for user in users:
        assert user.is_valid()  # Which one failed?

# GOOD: Clear failure messages
def test_all_users_valid():
    users = get_all_users()
    invalid = [u for u in users if not u.is_valid()]
    assert not invalid, f"Invalid users: {invalid}"

# GOOD: Parametrize instead
@pytest.mark.parametrize("user_id", [1, 2, 3, 4, 5])
def test_user_valid(user_id, db_session):
    user = get_user(db_session, user_id)
    assert user.is_valid()

Testing Implementation

# BAD: Testing internal state
def test_cache_internals():
    cache = Cache()
    cache.set("key", "value")
    assert cache._internal_dict["key"] == "value"  # Private!

# GOOD: Test public behavior
def test_cache_retrieval():
    cache = Cache()
    cache.set("key", "value")
    assert cache.get("key") == "value"

# BAD: Testing order of operations
def test_save_calls_validate(mocker):
    validate = mocker.patch("module.validate")
    save("data")
    validate.assert_called_before(save)  # Implementation detail!

# GOOD: Test the outcome
def test_save_validates_data():
    with pytest.raises(ValidationError):
        save(invalid_data)

Configuration Best Practices

# pytest.ini or pyproject.toml [tool.pytest.ini_options]
[pytest]
testpaths = tests
python_files = test_*.py
python_functions = test_*
addopts = -v --tb=short --strict-markers
markers =
    slow: marks tests as slow (deselect with '-m "not slow"')
    integration: marks tests as integration tests
filterwarnings =
    error
    ignore::DeprecationWarning

# pyproject.toml
[tool.pytest.ini_options]
minversion = "7.0"
addopts = "-ra -q --strict-markers"
testpaths = ["tests"]

[tool.coverage.run]
source = ["src"]
branch = true

[tool.coverage.report]
fail_under = 80

Quality Checklist

Invocation — Test Smell Detector (L3 Script)

After reviewing or writing test files, run the smell detector to get a structured, auditable finding list. Consume the script's output only — the script source never enters context.

Smells detected:

| SMELL-09 | bare_mock_truthiness | WARN | assert <chain>.called or assert <chain>.call_count used as a bare (uncompared, non-negated) truthiness check on any mock — added specifically because this exact form is the evasion QA found: it evades SMELL-08 too, since a bare .called/.call_count read is not an ast.Compare node and is classified as a real assertion by SMELL-08's classifier. .called/.call_count do correctly track whether the mock was invoked (they are not always-true) — the defect is evidentiary weakness: no argument-shape or exact-count verification, easy to misread as equivalent to .assert_called_once(). assert not x.called (the negative form, legitimate equivalent of .assert_not_called()) is not flagged. Independent of SMELL-08 — applies to any mock, not just DB sessions |

SMELL-08 recall limitation — read before trusting a clean run. "Under-fires, by design" above understates how narrow this rule's recall actually is; the honest reading of a clean SMELL-08 run is "none of the most blatant fully-vacuous DB-mock tests were found," NOT "no vacuous write-path tests exist here." Measured facts, not estimates: against fully-vacuous tests (a test whose only assertions are positive DB-session mock-call verifications and/or tautological read-backs — the exact class this rule targets), detection is effectively complete. Against the broader population of tests that contain a positive DB-session mock assertion at all, the rule flags roughly 11% of them — not because the other ~89% are missed defects, but because most of those tests also carry a genuine assertion on a production-computed value alongside the mock-call check, and a single real assertion correctly blocks firing (see E1/E2 above). The 11% is the rule correctly declining to fire on legitimate tests, not a coverage gap. Evasion is trivial and requires no adversarial intent: in a hand-written adversarial check, 5 realistic vacuous write-path tests were constructed and SMELL-08 caught 0 of 5 — evasions included (1) a fixture-provided session bound to a variable name off the heuristic list, (2) the mock-call assertion wrapped inside a helper function rather than inlined, (3) a parametrized test using an unconventional session variable name, (4) a session obtained through an async context manager, and (5) assert mock_db.commit.called — bare attribute-access truthiness on a mock instead of .assert_called_once() — which evaded every smell rule in this file silently, not just SMELL-08. Ordinary naming variance and normal helper-extraction refactors defeat this rule; treat SMELL-08 as advisory only. Never use a clean run as a blocking CI gate, and never treat it as evidence that a test file has no vacuous write-path tests.

Run via Bash (single file):

python .claude/skills/testing/pytest-patterns/scripts/pytest_smell.py path/to/test_file.py

*Run via Bash (directory — walks all test_.py / _test.py recursively):*

python .claude/skills/testing/pytest-patterns/scripts/pytest_smell.py tests/

Run via Bash (stdin):

cat test_example.py | python .claude/skills/testing/pytest-patterns/scripts/pytest_smell.py -

Output fields (JSON):

{
  "findings": [
    {
      "smell_id": "SMELL-01",
      "name": "no_assert",
      "severity": "ERROR",
      "message": "Test 'test_foo' has no assertion — it can never fail",
      "file": "tests/test_foo.py",
      "line": 10,
      "function": "test_foo"
    }
  ],
  "summary": {
    "total": 3,
    "error": 2,
    "warn": 1,
    "files_analyzed": 1
  }
}

What the agent does with the output:

1. Address all ERROR severity findings first — these are broken/useless tests.
2. Review WARN severity findings — these are flaky/noisy patterns; fix unless there is a documented reason.
3. Use summary.error count in any CI gate decision.

Error handling: Invalid file path → exits 1 with ERROR: message on stderr. Syntax error in a test file → reported as a PARSE-ERROR finding (not a crash). Empty input → exits 0 with zero findings.