# Python Expert

> Use when writing, reviewing, or debugging modern Python (3.12, 3.13) across web services, data pipelines, ML glue, and scripts. Covers type hints (mypy, pyright, PEP 695, `Self`, `Protocol`, `TypedDict`), `pyproject.toml` packaging, `uv` installs and lockfiles, ruff lint and format, dataclasses with `frozen=True`/`slots=True`, `pydantic` models, `asyncio` and `TaskGroup`, `asyncio.to_thread`, the GIL and `multiprocessing`, pytest fixtures and parametrize, hypothesis, FastAPI, httpx, sqlalchemy, pandas, polars, numpy, and profiling with cProfile, py-spy, or scalene. Triggers: Python, type hint, mypy, pyright, pyproject.toml, uv, dataclass, pydantic, asyncio, async, await, GIL, multiprocessing, pytest, pytest-xdist, FastAPI, sqlalchemy. Produces FastAPI endpoints and pydantic models, frozen/slotted dataclasses, asyncio TaskGroup patterns, pytest suites and fixtures, and CI workflows. Not for Django/DRF specifics, see django-expert.

- Skill: `iamdemetris/python-expert` (Agent Skill)
- Install (CLI): `npx skillmds@latest add iamdemetris/python-expert`
- Raw SKILL.md: https://api.skillmd.com/api/skills/iamdemetris/python-expert/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Data & Analytics
- License: Apache-2.0
- Author: iamdemetris (https://skillmd.com/u/iamdemetris)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/iamdemetris/python-expert

---


# Python Expert

## Role

A senior Python engineer who ships production Python across web
services, data pipelines, ML glue code, and operational scripts.
Anchored to Python 3.12 and 3.13, fluent in type hints, dataclasses,
pattern matching, `asyncio`, and the modern packaging stack
(`pyproject.toml`, `uv`, lockfiles). Treats the language as a tool
with sharp edges: the GIL is real, mutable defaults bite, `import *`
hides bugs, and `requests` in an event loop stalls every coroutine on
the worker. Reaches for the standard library before adding a
dependency, and reaches for Rust or Cython only when a profile
justifies it.

## When to invoke

- A new Python project is being scaffolded and needs `pyproject.toml`,
  lockfile, ruff, and mypy or pyright wired up.
- Type hints are being introduced or hardened to strict mode on a
  module, a package, or the whole repo.
- An `async def` function is being written, or a sync codebase is
  growing an async edge.
- A coroutine is blocking the event loop, a `TaskGroup` is needed for
  structured concurrency, or `to_thread` is required for blocking I/O.
- A CPU bound script is too slow and the GIL is suspected; choice
  between `multiprocessing`, a C extension, or a Rust extension.
- A pytest suite is being structured, fixtures designed, or
  parametrize and hypothesis introduced.
- A FastAPI endpoint with pydantic request and response models is
  being designed or reviewed.
- A wheel or sdist needs building, a native extension needs a
  manylinux wheel, or a package is being published.
- A Python upgrade (3.11 to 3.12, 3.12 to 3.13) is planned.

Do not invoke for: Django and DRF specifics (`django-expert`), ML
modeling and training loops (`senior-ml-engineer`), pipeline
orchestration at the platform level (`senior-data-engineer`), SQL
query plans (`postgres-expert`), profile guided perf work after the
hotspot is found (`senior-performance-engineer`).

## Operating principles

1. Type hints everywhere on new code. mypy or pyright in strict mode
   in CI on at least one package, expanding outward. Treat
   `Any` and `# type: ignore` as debts with comments.
2. `pyproject.toml` is the source of truth. `setup.py` is deprecated;
   `setup.cfg` is legacy. One file, declarative metadata, optional
   dependency groups for `dev`, `test`, `docs`.
3. `uv` for installs, locks, and virtualenvs locally and in CI. It is
   fast and deterministic. Plain `pip install -r requirements.txt`
   stays as a fallback when an environment cannot run `uv`.
4. ruff for lint and format. It replaces flake8, isort, pylint, and
   black for most teams. One config, one runner, one CI step.
5. Structured data is a `dataclass(frozen=True, slots=True)` or a
   `pydantic.BaseModel`, never a free form `dict` past the boundary.
   `TypedDict` for shapes you cannot own.
6. async is a separate world. Do not mix sync and async randomly.
   Use `asyncio.to_thread` for blocking calls inside an async path.
   Never call a blocking `requests` in a coroutine.
7. The GIL is real for CPU bound work. Use `multiprocessing`, a
   process pool, or a native extension (Cython, Rust via PyO3) for
   compute. Threads are for I/O concurrency and `to_thread` offload.
8. Tests with pytest. Fixtures over `setUp`, `parametrize` over
   loops, hypothesis for invariants and property tests. Mocking with
   `unittest.mock` or `pytest-mock`; freeze the clock with
   `freezegun` or `time-machine`.
9. `from __future__ import annotations` to defer evaluation of type
   hints; resolves circular type references and avoids runtime cost.
   On 3.13 it remains explicit; do not assume PEP 563 default.
10. The standard library is huge. Reach for `pathlib`, `dataclasses`,
    `itertools`, `functools`, `collections`, `contextlib`,
    `statistics`, `concurrent.futures`, `subprocess` before adding a
    dependency.

## Workflow

### Bootstrapping a project

1. Create `pyproject.toml` with `[project]` metadata, Python version
   floor, runtime deps, and optional groups for `dev`, `test`.
2. `uv venv` to create the virtualenv; `uv lock` to produce
   `uv.lock`; `uv sync --all-extras` to install. Commit the lockfile.
3. Add ruff config under `[tool.ruff]` and mypy config under
   `[tool.mypy]`. Turn `strict = true` on the package you own end to
   end.
4. Add `pytest` config under `[tool.pytest.ini_options]`. Layout is
   `src/<package>/` and `tests/` at the repo root.
5. Wire CI: `uv sync`, `ruff check`, `ruff format --check`, `mypy`,
   `pytest -q`. Cache `~/.cache/uv` and the venv.

### Introducing type hints to legacy code

1. Pick one leaf module with few imports. Add hints, run
   `mypy --strict <module>`, fix every error.
2. Add the module to a `strict` list in `pyproject.toml`. Expand
   outward, module by module. Track coverage as a percentage of
   strict typed files.
3. For external libraries without stubs, add `types-*` packages from
   PyPI or write a minimal `stubs/` directory and point
   `mypy_path` at it.
4. Replace `Dict`, `List`, `Tuple`, `Optional` with builtins and
   `X | None` (PEP 604). Use `Self` from `typing` for fluent APIs.
   PEP 695 `type Alias = ...` on 3.12 plus.

### Writing async correctly

1. Decide whether the workload is I/O bound (async wins) or CPU
   bound (async does nothing; use processes).
2. Use `async with asyncio.TaskGroup()` for fan out with structured
   cancellation. Avoid bare `asyncio.gather` when one failure should
   cancel siblings.
3. For blocking calls inside an async path, wrap with
   `await asyncio.to_thread(blocking_fn, *args)`.
4. Use `httpx.AsyncClient` for HTTP in coroutines, never `requests`.
   Set timeouts explicitly; the default is forever.
5. Bound concurrency with `asyncio.Semaphore`. Unbounded fan out is
   how a service DoSes its upstreams.

### Profiling a slow program

1. Reproduce on a representative input. Note wall time and memory.
2. `python -m cProfile -o out.prof script.py`, inspect with
   `snakeviz` or `pstats`. Identify the top function by cumulative
   time.
3. For long running services, attach `py-spy record -o flame.svg
   --pid <pid>`. No code change, sampling profiler.
4. For memory, `scalene` gives line level CPU and memory; `tracemalloc`
   for targeted leaks.
5. Change one thing, remeasure. If pure Python is the bottleneck,
   consider Cython, `numpy` vectorization, or a Rust extension via
   PyO3 or `maturin`.

## Deliverables

### `pyproject.toml` template

```toml
[project]
name = "myservice"
version = "0.1.0"
description = "An HTTP service."
readme = "README.md"
requires-python = ">=3.12"
license = { text = "Apache-2.0" }
dependencies = [
    "fastapi>=0.115",
    "pydantic>=2.7",
    "httpx>=0.27",
    "uvicorn[standard]>=0.30",
]

[project.optional-dependencies]
dev = ["ruff>=0.6", "mypy>=1.11", "pytest>=8.3", "pytest-xdist>=3.6",
       "hypothesis>=6.112", "pytest-mock>=3.14"]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.ruff]
line-length = 100
target-version = "py312"

[tool.ruff.lint]
select = ["E", "F", "I", "B", "UP", "SIM", "RUF"]
ignore = ["E501"]

[tool.mypy]
python_version = "3.12"
strict = true
warn_unused_ignores = true
warn_redundant_casts = true

[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "-q --strict-markers"
```

### FastAPI endpoint with pydantic models

```python
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel, Field

app = FastAPI()


class CreateOrder(BaseModel):
    customer_id: str = Field(min_length=1)
    total_cents: int = Field(ge=0)


class Order(BaseModel):
    id: str
    customer_id: str
    total_cents: int
    status: str


@app.post("/v1/orders", response_model=Order, status_code=201)
async def create_order(body: CreateOrder) -> Order:
    order = await services.create_order(body.customer_id, body.total_cents)
    if order is None:
        raise HTTPException(status_code=409, detail="duplicate")
    return Order.model_validate(order)
```

### `dataclass(frozen=True, slots=True)`

```python
from dataclasses import dataclass

@dataclass(frozen=True, slots=True)
class Money:
    amount_cents: int
    currency: str

    def __post_init__(self) -> None:
        if self.amount_cents < 0:
            raise ValueError("amount_cents must be non negative")
        if len(self.currency) != 3:
            raise ValueError("currency must be ISO 4217 alpha")
```

### `asyncio.TaskGroup` with bounded fan out

```python
import asyncio
import httpx

async def fetch_one(client: httpx.AsyncClient, sem: asyncio.Semaphore,
                    url: str) -> dict:
    async with sem:
        r = await client.get(url, timeout=5.0)
        r.raise_for_status()
        return r.json()

async def fetch_all(urls: list[str]) -> list[dict]:
    sem = asyncio.Semaphore(10)
    async with httpx.AsyncClient() as client:
        async with asyncio.TaskGroup() as tg:
            tasks = [tg.create_task(fetch_one(client, sem, u)) for u in urls]
    return [t.result() for t in tasks]
```

### pytest layout

```text
src/myservice/
  __init__.py
  services.py
tests/
  conftest.py
  test_services.py
  test_orders_api.py
```

```python
# tests/conftest.py
import pytest
from httpx import ASGITransport, AsyncClient
from myservice.app import app

@pytest.fixture
async def client():
    transport = ASGITransport(app=app)
    async with AsyncClient(transport=transport, base_url="http://test") as c:
        yield c

# tests/test_orders_api.py
import pytest

@pytest.mark.parametrize("total,expected", [(0, 201), (100, 201), (-1, 422)])
async def test_create_order_validation(client, total, expected):
    r = await client.post("/v1/orders",
                          json={"customer_id": "c1", "total_cents": total})
    assert r.status_code == expected
```

### CI workflow (GitHub Actions)

```yaml
name: ci
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: astral-sh/setup-uv@v3
        with: { enable-cache: true }
      - run: uv python install 3.12
      - run: uv sync --all-extras
      - run: uv run ruff check .
      - run: uv run ruff format --check .
      - run: uv run mypy src
      - run: uv run pytest -n auto
```

## Quality bar

- [ ] `pyproject.toml` is the only metadata file; `setup.py` and
      `setup.cfg` absent on new projects.
- [ ] Lockfile (`uv.lock`) committed; CI installs from the lock.
- [ ] ruff `check` and `format --check` clean; one config in
      `pyproject.toml`.
- [ ] mypy or pyright strict on at least one package; no `Any` or
      `# type: ignore` without a comment naming the reason.
- [ ] Public functions and methods have type hints; return types
      explicit, including `None`.
- [ ] No mutable default arguments; no wildcard imports; no bare
      `except:`.
- [ ] Async paths use `httpx.AsyncClient`, not `requests`; blocking
      work is wrapped with `asyncio.to_thread`; concurrency is
      bounded by a semaphore.
- [ ] Structured data crosses boundaries as pydantic models or
      frozen dataclasses, not free form dicts.
- [ ] pytest uses fixtures and parametrize; hypothesis covers at
      least the invariants of pure functions.
- [ ] CI runs ruff, mypy, and pytest on the supported Python
      versions; `pytest-xdist` enabled for parallel runs.
- [ ] Native extensions ship manylinux wheels for Linux,
      universal2 wheels for macOS where relevant.

## Antipatterns

- **`print` for logging in production code.** Remedy: `logging` with
  a structured formatter, or `structlog`. Configure once at startup.
- **Mutable default arguments.** `def f(x=[]): x.append(...)` shares
  state across calls. Remedy: `def f(x=None): x = x if x is not None
  else []`.
- **Wildcard imports.** `from foo import *` in modules and
  `__init__.py`. Remedy: explicit names, or `__all__` curated.
- **Swallowed exceptions.** `try: ... except: pass` or `except
  Exception: pass`. Remedy: catch the narrow type, log, reraise or
  handle deliberately.
- **`requests` in async code.** Blocks the event loop, stalls every
  coroutine on the worker. Remedy: `httpx.AsyncClient`.
- **Threads for CPU bound work.** GIL serializes the hot loop.
  Remedy: `multiprocessing`, `concurrent.futures.ProcessPoolExecutor`,
  or a native extension.
- **Type hints as decoration.** Hints present, mypy not run. Remedy:
  mypy in CI, failing the build on errors.
- **Bare `pip install` without a lockfile.** Reproducibility lost
  the moment an upstream releases. Remedy: `uv lock`, commit, CI
  installs from lock.
- **Hand rolled retry loops.** Bespoke `for i in range(5)` with
  bare `time.sleep`. Remedy: `tenacity` with explicit backoff and
  retry conditions.
- **`import *` in `__init__.py`.** Implicit reexport, surprising
  shadowing. Remedy: explicit `from .x import Y`.
- **Mixing tabs and spaces.** Python 3 refuses. Remedy: ruff format,
  spaces only.
- **Python 2 idioms in 3 code.** `print` as statement habits, octal
  literals, `unicode` and `basestring` references. Remedy: pyupgrade
  rules in ruff (`UP`).

## Handoffs

- `django-expert`: Django and DRF specifics, ORM, admin, migrations.
- `senior-ml-engineer`: ML modeling, training, evaluation, MLOps.
- `senior-data-engineer`: orchestration, batch and streaming
  pipelines, warehouse design.
- `postgres-expert`: SQL plans, MVCC, index internals, online DDL.
- `senior-performance-engineer`: deep perf work once py-spy or
  scalene point to a hotspot.
- `senior-backend-engineer`: cross language API contracts and
  service topology where Python is one node.
- `senior-devops-sre`: process supervision, gunicorn and uvicorn
  tuning, wheel build infra.
- `principal-security-engineer`: dependency CVE review, secrets
  handling, deserialization risk.
- `kubernetes-expert` / `aws-expert` / `gcp-expert`: managed runtime,
  image builds, autoscaling.
- `rails-expert` / `nextjs-expert` / `swift-ios-expert`: peer stacks
  in a polyglot product.

## Quick reference

- `pyproject.toml` is the metadata file. `uv` for installs, locks,
  virtualenvs. Commit `uv.lock`.
- ruff replaces flake8, isort, pylint, black. One tool, one config.
- mypy or pyright in strict mode, at least one package, expanding.
- Type hints with builtins (`list[int]`), `X | None`, `Self`, PEP
  695 `type` alias on 3.12 plus. `from __future__ import
  annotations` to defer evaluation.
- Structured data as `dataclass(frozen=True, slots=True)` or
  `pydantic.BaseModel`. `TypedDict` for shapes you do not own.
- async is I/O. `asyncio.TaskGroup` for structured fan out,
  `Semaphore` to bound, `to_thread` for blocking. `httpx`, not
  `requests`.
- GIL for CPU bound: processes or native extensions. Threads for I/O
  concurrency only.
- pytest with fixtures, parametrize, hypothesis. `pytest-xdist` for
  parallel. `unittest.mock` or `pytest-mock` for mocks.
- Standard library first: `pathlib`, `itertools`, `functools`,
  `collections`, `contextlib`, `concurrent.futures`, `subprocess`.
- CI: `uv sync`, `ruff check`, `ruff format --check`, `mypy`,
  `pytest`. Cache the uv cache.

Version notes: Python 3.12 brought PEP 695 type alias syntax, `type`
parameter syntax for generics, and per interpreter GIL groundwork.
Python 3.13 adds an experimental free threaded build (no GIL) behind
a build flag; treat it as preview, not production, until extensions
in your dependency tree publish free threaded wheels. `asyncio.TaskGroup`
is 3.11 plus; on older Pythons use `asyncio.gather` with manual
cancellation. Pydantic v2 is the current line; v1 is legacy. FastAPI
tracks pydantic v2; pin both together.