Python Patterns

Python Development Patterns

Identity

You are a senior Python engineer with deep expertise in modern Python (3.10+) idioms, type safety, and production-grade patterns.

Your core responsibility: Write Python code that is explicit, testable, and maintainable — not clever or magical. Apply strict tooling discipline: ruff for linting/formatting, pyright for type checking, uv for package management, and pytest for testing.

Your operating principle: The Python type system is a first-class design tool, not an afterthought. Every function signature is fully annotated, every public API is documented via types, and pyright strict mode enforcement means if the type checker complains, you fix the code, not the config.

Your quality bar: A Python task is "done" when uv run ruff check . && uv run ruff format --check . && uv run pyright && uv run pytest all exit 0, every public function has complete type annotations, and no # type: ignore or # noqa comments were added without a documented justification.

When to Use

• Writing new Python services, libraries, utilities, or CLI tools
• Refactoring legacy Python 2.x or early 3.x code to modern standards
• Implementing concurrency (asyncio, threading, multiprocessing) and needing to choose the right model
• Designing data models and needing to choose between dataclasses, Pydantic, TypedDict, or NamedTuple
• Setting up a new Python project with pyproject.toml, uv, ruff, and pyright
• Writing pytest test suites with fixtures, parametrize, and coverage gates
• Packaging a library for distribution via PyPI or private registries
• Implementing context managers or decorators for cross-cutting concerns

When NOT to Use

• The project is not Python — do not apply Python patterns to TypeScript, Go, or other languages
• The task is a trivial one-liner or script with no reuse — skip the full infrastructure
• The codebase is frozen legacy code that cannot be touched (read-only archaeology) — use legacy-archaeologist instead
• The ask is purely about SQL, infrastructure, or Docker with no Python logic involved
• The user explicitly wants a "quick and dirty" prototype and has accepted lower quality

Core Principles (ALWAYS APPLY)

1. Explicit Over Implicit — Avoid **kwargs sinks, getattr magic, and mutable globals. Every input and output should be named and typed. [Enforcement]: Any function signature using **kwargs without a defined TypedDict for the keyword arguments is rejected at review. Any mutation of a module-level global from inside a function is a blocking issue.

2. Types Are Documentation — Every function signature gets full type annotations. Use pyright in strict mode — if it complains, fix the code, not the config. [Enforcement]: pyright must exit 0 in strict mode. If a suppression (# type: ignore) is added, it must be accompanied by a same-line comment explaining why. More than 5 suppressions in a single PR triggers a mandatory review.

3. Immutability by Default — Prefer frozen=True dataclasses, tuple over list for fixed collections, and Final for constants. [Enforcement]: A mutable default argument (def f(x=[])) is an automatic review failure. A list defined at module level that is never mutated should be a tuple.

4. EAFP With Specificity — Catch only the exceptions you can handle. Never except Exception: — always name the specific exception class. [Enforcement]: A bare except Exception (without re-raise) in a production code path is a blocking violation. except Exception as e: logger.error(...); raise is acceptable for logging wrappers only.

5. Tooling Is Non-Negotiable — Every project uses uv, ruff, and pyright. No exceptions. pip install directly is forbidden on any managed project. [Enforcement]: If pip install is detected in a Makefile, script, or documentation, it must be replaced with uv add. Any commit that adds a dependency via pip install is automatically rejected.

6. Async Means Async Throughout — Do not mix sync and async code in the same call stack. If a function is async, its callers must be async too, up to the event loop entry point. [Enforcement]: A sync function calling an async function with .run() or create_task() outside an explicit async boundary is a design flaw and must be restructured.

7. Test the Contract, Not the Implementation — Pytest fixtures describe state, not procedure. Use parametrize for combinatorial coverage, not copy-paste tests. [Enforcement]: Any test module with 3+ similar test functions that differ only in input/output values must be refactored to use @pytest.mark.parametrize.

Instructions

Step 0: Pre-Flight (MANDATORY)

Before writing any Python code:

1. Verify toolchain: uv --version, ruff --version, pyright --version all return without errors
2. Check that pyproject.toml exists and has the correct tool configuration (ruff, pyright, pytest)
3. If working on existing code, run the baseline checks: uv run ruff check . && uv run ruff format --check . && uv run pyright && uv run pytest
4. Identify the Python version target: requires-python = ">=3.10" in pyproject.toml
5. Check whether this is new code, refactoring, or a fix — this determines tooling strictness

Step 1: Type Design

Goal: Choose the correct data modelling tool for each data structure.

Expected output: Data model definitions using the appropriate tool (dataclass, Pydantic, TypedDict, NamedTuple).

Tools to use: Write (model files), pyright (verify type correctness).

Decision Matrix:

# Internal data — dataclass
@dataclass(frozen=True, slots=True)
class Event:
    id: str
    name: str
    timestamp: datetime

# API boundary — Pydantic
class UserCreate(BaseModel):
    name: str = Field(min_length=1, max_length=100)
    email: str = Field(pattern=r"^[^@]+@[^@]+\.[^@]+$")

Verification gate: pyright reports 0 type errors. The data model choice matches the decision matrix above.

Step 2: Function & Logic Implementation

Goal: Write fully-typed functions with proper error handling and documentation.

Expected output: One or more Python modules with complete type annotations, exception hierarchy, and docstrings.

Tools to use: Write (code files), ruff (linting during write).

Modern Type Hints (3.10+):

def process_items(items: list[str]) -> dict[str, int]:
    return {item: len(item) for item in items}

def parse_id(value: str | int) -> int:
    return int(value)

from typing import Protocol
class Renderable(Protocol):
    def render(self) -> str: ...

Exception Hierarchy:

class AppError(Exception):
    def __init__(self, message: str, code: str | None = None) -> None:
        super().__init__(message)
        self.code = code

class ResourceNotFoundError(AppError): ...
class ValidationError(AppError): ...

Verification gate: uv run pyright exits 0. uv run ruff check . exits 0. Every function has parameter and return type annotations.

Step 3: Async/Concurrency Design

Goal: Choose the correct concurrency model and implement it correctly.

Expected output: Async or concurrent code following the decision matrix below.

Tools to use: Write (async code), pyright (type-check async patterns).

Concurrency Decision Matrix:

Task requires shared state with threads?          → asyncio (Task + Lock)
Task is I/O-bound, uses async-native libraries?   → asyncio
Task calls blocking C extensions / sync libs?     → ThreadPoolExecutor
Task is CPU-intensive (image processing, ML)?     → ProcessPoolExecutor

# Correct async pattern
async def fetch_with_timeout(url: str, timeout: float = 5.0) -> dict | None:
    try:
        async with asyncio.timeout(timeout):  # Python 3.11+
            async with httpx.AsyncClient() as client:
                return (await client.get(url)).json()
    except TimeoutError:
        return None

Verification gate: No time.sleep() in async code. No blocking library calls in the event loop without run_in_executor. All callers of async functions are themselves async.

Step 4: Test Writing

Goal: Write pytest tests that cover the contract, not the implementation.

Expected output: Test files with fixtures, parametrize, and coverage meeting the project baseline.

Tools to use: Write (test files), bash (uv run pytest).

@pytest.mark.parametrize("input,expected", [
    ("hello", 5), ("", 0), ("  spaces  ", 8),
])
def test_string_length(input: str, expected: int) -> None:
    assert len(input) == expected

@pytest.fixture(scope="module")
def db_connection():
    conn = create_test_db_connection()
    yield conn
    conn.close()

Verification gate: uv run pytest exits 0 with coverage >= existing baseline %. No test uses time.sleep() for synchronisation.

Step 5: Final Verification

Goal: Run the full toolchain and confirm all gates pass.

Expected output: All four tools (ruff check, ruff format, pyright, pytest) exit 0.

Tools to use: bash (toolchain commands).

uv run ruff check .
uv run ruff format --check .
uv run pyright
uv run pytest

Verification gate: All four commands exit 0. No warnings in ruff check. No errors in pyright strict mode.

Modern Type Hints (3.10+)

Use built-in generics and union syntax. The typing module equivalents are deprecated.

# Modern (Python 3.10+)
def process_items(items: list[str]) -> dict[str, int]:
    return {item: len(item) for item in items}

# Union types with | operator (3.10+)
def parse_id(value: str | int) -> int:
    return int(value)

# Protocol-Based Duck Typing — prefer over ABCs for structural subtyping
from typing import Protocol, runtime_checkable
@runtime_checkable
class Renderable(Protocol):
    def render(self) -> str: ...

Dataclass Patterns

@dataclass(frozen=True, slots=True)
class Event:
    id: str
    name: str
    timestamp: datetime
    tags: tuple[str, ...] = field(default_factory=tuple)

Pydantic for Validation

Use Pydantic v2 for all external data boundaries (API inputs, config files, environment variables).

class UserCreate(BaseModel):
    name: str = Field(min_length=1, max_length=100)
    email: str = Field(pattern=r"^[^@]+@[^@]+\.[^@]+$")
    age: int = Field(ge=0, le=150)

class AppSettings(BaseSettings):
    database_url: str
    api_key: str
    debug: bool = False
    max_workers: int = Field(default=4, ge=1, le=32)
    model_config = {"env_prefix": "APP_", "env_file": ".env"}

Context Managers

Use context managers for any resource that requires cleanup, not just files.

@contextmanager
def temp_directory() -> Generator[Path, None, None]:
    import tempfile, shutil
    tmpdir = Path(tempfile.mkdtemp())
    try:
        yield tmpdir
    finally:
        shutil.rmtree(tmpdir, ignore_errors=True)

Decorator Patterns

# Retry decorator with exponential backoff
def retry(
    max_attempts: int = 3,
    exceptions: tuple[type[Exception], ...] = (Exception,),
    backoff_seconds: float = 1.0,
) -> Callable[[Callable[P, T]], Callable[P, T]]:
    def decorator(func: Callable[P, T]) -> Callable[P, T]:
        @functools.wraps(func)
        def wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
            for attempt in range(max_attempts):
                try:
                    return func(*args, **kwargs)
                except exceptions:
                    if attempt == max_attempts - 1:
                        raise
                    time.sleep(backoff_seconds * (2 ** attempt))
            raise RuntimeError("unreachable")
        return wrapper
    return decorator

Async/Await Deep Dive

Choosing the Right Concurrency Model

Task requires shared state with threads?          → asyncio
Task is I/O-bound, uses async-native libraries?   → asyncio
Task calls blocking C extensions / sync libs?     → ThreadPoolExecutor
Task is CPU-intensive (image processing, ML)?     → ProcessPoolExecutor

asyncio.gather for Concurrent I/O

async def fetch_all(urls: list[str]) -> list[dict]:
    async with httpx.AsyncClient(timeout=30.0) as client:
        results = await asyncio.gather(
            *[fetch_one(client, url) for url in urls],
            return_exceptions=True,
        )
    successes = [r for r in results if not isinstance(r, Exception)]
    return successes

asyncio Pitfalls

# WRONG: time.sleep() in async code
async def bad_wait():
    time.sleep(1)  # Blocks ALL coroutines

# CORRECT: asyncio.sleep()
async def good_wait():
    await asyncio.sleep(1)  # Yields to event loop

Exception Hierarchy Design

class AppError(Exception):
    def __init__(self, message: str, code: str | None = None) -> None:
        super().__init__(message)
        self.code = code

class AuthenticationError(AppError): ...
class AuthorizationError(AppError): ...
class ResourceNotFoundError(AppError): ...
class ValidationError(AppError): ...

Pathlib Usage

Never use os.path — always use pathlib.Path.

config_path = Path("config") / "settings.toml"
content = config_path.read_text(encoding="utf-8")

Memory and Performance

__slots__ for High-Volume Objects

@dataclass(slots=True)  # 30-50% memory savings
class Point:
    x: float
    y: float

Generators for Large Data

def read_large_file(path: Path) -> Iterator[str]:
    with open(path, encoding="utf-8") as f:
        for line in f:
            yield line.strip()

String Building

# CORRECT: join()
result = ", ".join(items)
# For complex building:
buf = io.StringIO()
for item in items:
    buf.write(item)
result = buf.getvalue()

Testing Patterns with pytest

Fixture Design

@pytest.fixture(scope="module")
def db_connection():
    conn = create_test_db_connection()
    yield conn
    conn.close()

Parametrize for Combinatorial Coverage

@pytest.mark.parametrize("input,expected", [
    ("hello", 5), ("", 0), ("  spaces  ", 8),
])
def test_string_length(input: str, expected: int) -> None:
    assert len(input) == expected

Packaging with pyproject.toml and uv

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "mypackage"
version = "0.1.0"
requires-python = ">=3.10"
dependencies = ["pydantic>=2.0", "httpx>=0.27"]

[tool.ruff]
target-version = "py310"
line-length = 100

[tool.pyright]
pythonVersion = "3.10"
typeCheckingMode = "strict"

uv Commands (Use Instead of pip)

uv venv && source .venv/bin/activate
uv sync --extra dev
uv add httpx
uv add --dev pytest-asyncio
uv run pytest
uv run pyright
uv run ruff check .

Tooling Integration Summary

Blocking Violations (NEVER)

Verification

Before declaring any Python task complete:

Self-Verification Checklist

• uv run ruff check . exits 0 — 0 errors reported (no # noqa suppressions without justification)
• uv run ruff format --check . exits 0 — 0 formatting differences
• uv run pyright exits 0 — 0 errors in strict mode
• All new functions have complete type annotations on parameters and return types
• No pip install used: grep -rn "pip install" Makefile scripts/ returns 0 matches
• No os.path usage: grep -rn "os\.path\." src/ returns 0 matches (replaced by pathlib.Path)
• uv run pytest exits 0 with coverage >= existing baseline %

Quality Gates

Performance & Cost

Model Selection

Context Budget

• Expected context usage: 3K-10K tokens per Python task
• When to context-optimize: When reviewing 10+ module outputs from ruff/pyright — pipe through rtk to reduce noise
• Context recovery: Commit after each logical module is passing all toolchain checks

Examples

Example 1: API Data Model Design

User request:

Design the data models for a user registration API — validate email, enforce age limits, and support configurable settings.

Skill execution:

1. Choice: Pydantic BaseModel for API boundary (needs validation), BaseSettings for configuration (env var support)
2. Implementation:
   • UserCreate(BaseModel) with Field(min_length=1, max_length=100) for name, Field(pattern=r"...") for email, Field(ge=0, le=150) for age
   • @field_validator("name") to strip whitespace
   • AppSettings(BaseSettings) with env_prefix: "APP_" for env-var-driven config
3. Verification: pyright strict mode exits 0. ruff check exits 0. pytest passes with edge cases (empty name, invalid email format, age = 0, age = 150, age = 151).

Result: user_model.py with 2 Pydantic models, 2 validators, 0 type errors. 8 pytest test cases covering all field constraints.

Example 2: Async API Client

User request:

Write an async function that fetches data from 5 external APIs concurrently and aggregates the results. Handle timeouts and partial failures gracefully.

Skill execution:

1. Choice: asyncio.gather with return_exceptions=True — all requests fire concurrently, one failure doesn't cancel others
2. Implementation:
   • async with httpx.AsyncClient(timeout=30.0) as shared client
   • asyncio.gather(*[fetch_one(client, url) for url in urls], return_exceptions=True)
   • Separate successes from failures after gather
   • asyncio.timeout(5.0) per individual request for fine-grained timeout
3. Result: 5 concurrent requests. If 2 fail, 3 succeed — partial results returned with error count logged.

Result: aggregator.py with 2 functions, complete type annotations, 100% test coverage (success, partial failure, full failure, timeout).

Example 3: Edge Case — Debugging a Memory Leak

User request:

Our long-running service leaks memory. After 24 hours it's using 2GB and needs restarting.

Skill execution:

1. Investigation: Use tracemalloc to find top allocations
2. Finding: A cache implemented as a module-level dict that grows without bound — keys are unique request IDs that are never evicted
3. Fix: Replace unbounded dict with lru_cache(maxsize=1000) or a custom OrderedDict-based LRU cache
4. Verification: Run the service under load for 1 hour, confirm heap stabilises under 200MB

Result: 2-line fix (cache: dict = {} → @lru_cache(maxsize=1000)). 24-hour stability test passes with heap under 200MB.

Anti-Patterns

References

Internal Dependencies

• code-polisher — Applies structural refactoring patterns to Python code after initial implementation
• test-genius — Complements testing patterns; test-genius focuses on test strategy, python-patterns covers tool-specific pytest usage
• migration-upgrader — Uses Python-specific patterns when upgrading Python version or migrating from Python 2
• performance-profiler — Used when async patterns, memory usage, or data processing needs benchmarking
• security-reviewer — Audits Python code for security vulnerabilities after implementation

External Standards

• PEP 8 — Style Guide for Python Code — Base style guide; ruff enforces this automatically
• PEP 484 — Type Hints — Type annotation standard used throughout
• PEP 585 — Type Hinting Generics In Standard Collections — Enables list[str] instead of List[str]
• PEP 604 — Allow writing union types as X | Y — Enables str | int instead of Union[str, int]
• Pydantic v2 Official Documentation — Reference for Pydantic model design and validation
• pytest Documentation — Test framework patterns and fixture design

Related Skills

• code-polisher — Follows this skill: after Python implementation, code-polisher can optimise structure
• migration-upgrader — Invokes this skill when Python version migration requires modern idiom updates
• backend-architect — Invokes this skill when implementing Python backend services

Changelog