Modern Python library choices
Default to current, well-maintained libraries over dated stdlib or legacy packages when the
better option is clear. Install third-party deps via uv (a tool with uvx, or a script with
PEP-723 inline metadata run by uv run). Libraries on this list are pre-approved: just use
them and let uv fetch them, no need to ask. These are general public defaults, not absolutes:
a project's own conventions win.
Adding an entry
Before adding a library here, vet it: trustworthy (reputable maintainer or community, not a
typo-squat), common (widely adopted), and modern (actively maintained, current releases). Each
new row must carry: a short description of what it is for (in the Use cell), the older
library/libraries it replaces (in the Avoid cell), and - whenever the pick is not
self-evident - why it is better (the parenthetical note in the Use cell). A swap that
explains itself (pathlib.Path over os.path, pytest over unittest) needs no parenthetical;
a contested or trade-off pick (isal vs deflate, one MySQL driver over another) always does.
Keep rows one line; do not add anyone's private or in-house packages here.
If the new library overlaps in function with an existing entry, resolve it - never leave two
rows silently competing for the same job:
- Both have a place: keep both and make the distinction explicit - state when to use which
and why (e.g. one streaming vs one one-shot, sync vs async, simple vs full-featured).
- The new one supersedes the old: only when you are sure, replace the old row with the new
one, move the now-obsolete library into the new row's
Avoid cell, and say why it was
replaced.
Quick reference
Use names the pick with a brief why; Avoid names what it replaces.
| Task |
Use |
Avoid |
| HTTP / REST |
httpx2 (HTTP/2; the new Pydantic-org successor to httpx) or httpx |
requests |
| JSON |
orjson (fast, correct, bytes) |
json stdlib for hot paths |
| TOML |
rtoml |
tomllib, tomli |
| YAML |
ruamel.yaml (round-trips comments) |
PyYAML |
| XML |
lxml (XPath, schema validation, fast C parser) |
xml.etree, minidom, xmltodict |
| Structured data (pure / internal layers) |
dataclasses (stdlib, lightweight; trusted internal data) |
bare dict, attrs, hand-rolled classes |
| Structured data at boundaries (parse input) |
pydantic (validates + parses untrusted input at the edge) |
bare dict, attrs, hand-rolled parsing |
| Enums |
IntEnum / StrEnum (StrEnum is 3.11+) |
plain Enum, magic strings |
| Terminal output |
rich |
colorama (fallback only) |
| TUI |
textual |
curses |
| Paths |
pathlib.Path |
os.path |
| Date / time |
stdlib datetime + zoneinfo (tz-aware) |
pytz, naive datetimes |
| Compression (streaming / web / high speed) |
isal (igzip) - speed tuned, bigger files |
gzip stdlib for throughput |
| Compression (archival, high ratio) |
deflate (libdeflate bindings, high ratio, smaller files, C-extension dependency) |
gzip stdlib |
| .env files |
python-dotenv |
manual parsing |
| Database (ODBC) |
pyodbc |
raw ODBC bindings |
| Database (MySQL) |
mysql-connector-python or SQLAlchemy |
PyMySQL, mysqlclient |
| ORM / complex queries |
SQLAlchemy |
custom ORM, raw SQL for complex apps |
| Testing |
pytest |
unittest |
| Lint / format |
ruff (ruff check + ruff format; one Rust-fast tool replacing the whole flake8/black/isort stack, single config in pyproject) |
flake8, black, isort, pylint, pyupgrade, autoflake |
| Type checking |
pyright in strict mode (fast, no plugin setup, same engine as the editor's Pylance, narrows better on TypedDict/Protocol/generics) |
mypy, untyped code, blanket # type: ignore |
| CLI args / parsing |
rich-click (Click-based, rich-formatted --help; drop-in for click) |
argparse, optparse, getopt, bare click |
| Subprocess |
subprocess.run([...]) (argv list) |
os.system, shell=True |
| PowerShell / Windows + OS admin objects |
pwshpy (typed records + lazy pipeline over native OS bindings, real exceptions, memory-bounded; use over the Subprocess row when the job is querying/mutating OS objects - services, registry, event log, ACLs, tasks, accounts - not launching a program; see bitranox:coding-python-pwshpy) |
shelling out to pwsh.exe / powershell / wevtutil / sc; raw pywin32 / wmi + manual text parsing |
| Retry / backoff |
tenacity (declarative retry, exponential backoff + jitter; see bitranox:coding-resilience) |
hand-rolled while/sleep retry loops |
| .gitignore parse / file filtering |
igittigitt (git-exact, include mode, memory-bounded; see bitranox:coding-python-gitignore) |
hand-rolled fnmatch/glob/re; gitignore_parser; pathspec |
| Text encoding / mojibake repair |
ftfy (repairs mixed / double-encoded mojibake, e.g. ü->ü; leaves already-correct text untouched) |
blanket .encode('latin-1').decode('utf-8') round-trips; manual char swaps |
| MCP server / client (Model Context Protocol) |
fastmcp (decorator API for tools/resources/prompts plus auth, server composition, proxying, OpenAPI generation, built-in testing) |
hand-rolled JSON-RPC; the official mcp SDK's low-level server API and its bundled mcp.server.fastmcp (frozen 1.0 feature set) |
| ICMP ping / reachability / traceroute / port scan / MAC / ARP / routes / interfaces |
ipscout (in-process, no subprocess; unprivileged ICMP via the ping socket on Linux/macOS and iphlpapi on Windows, so no root or Administrator; also port scanning, neighbour and route lookup, subnets, wake-on-LAN and path MTU; see bitranox:coding-python-network-probe) |
icmplib (forces raw sockets on Windows, needs Administrator), scapy, python-nmap, netifaces (unmaintained, needs a C build), subprocess around ping/tracert/arp/ip |
| Layered / cross-platform app config |
lib_layered_config (merges defaults/app/host/user/.env/env into one immutable object with per-key provenance and profiles; resolves Linux/macOS/Windows paths; library + CLI; see bitranox:coding-python-layered-config) |
ad-hoc os.environ reads plus scattered file loads with hand-rolled precedence |
Structured data: pydantic at the edges, dataclasses inside
For any structured data in a Python app, parse untrusted/external input into pydantic models at
every boundary, and use dataclasses for pure internal layers. Do NOT use attrs, hand-woven
classes, or raw dicts for structured data. For the full end-to-end discipline (parse once at the
boundary, typed models throughout, Enums for fixed values, minimal conversions), use the
bitranox:coding-python-enforce-data-architecture-strict skill.
HTTP example (httpx2)
import httpx2 as httpx # API-compatible with httpx
with httpx.Client(timeout=30.0, proxy="http://user:pass@host:port") as client:
r = client.get("https://api.example.com/data")
r.raise_for_status()
data = r.json()
Use a Client for many requests (connection reuse), and AsyncClient under asyncio. Both
support proxy= (one proxy), mounts= (per-scheme proxy transports), timeout=, and HTTP/2.
There is no proxies= kwarg - httpx dropped it in 0.28 and httpx2 never had it, so passing it
raises TypeError: Client.__init__() got an unexpected keyword argument 'proxies'.
httpx2 (github.com/pydantic/httpx2) is the legitimate Pydantic-org-stewarded successor to httpx
and a drop-in replacement. Some security scanners flag it as a typosquat - that is a FALSE POSITIVE
(verified: Pydantic org + Trusted Publisher + Sigstore). If a scanner re-flags it, re-verify the
publisher independently and surface it to the user rather than auto-dismissing; do not silently
swap it out.
Notes
- Prefer a library over an external command-line tool:
httpx2 instead of shelling out to
curl, stdlib/orjson instead of jq, re/stdlib instead of grep/sed. Libraries are
the same on every OS; external commands may be missing or take different flags on Windows.
- Reach for stdlib when no third-party library is warranted (small glue, no hot path): the
point is "best tool", not "most dependencies".
- Logging and CLI framework are deliberately left to each project's own conventions rather
than prescribed here.
subprocess raises a DIFFERENT exception type per OS for the same condition, so never branch
on which one you got. A missing cwd raises FileNotFoundError on POSIX but
OSError [WinError 267] "The directory name is invalid" on Windows, so a
FileNotFoundError -> 127 / OSError -> 126 mapping returns two different exit codes for one
condition - and the POSIX run stays green because POSIX happens to raise the type the code
expected. Pre-check the path yourself (Path(cwd).exists() / .is_dir()) and decide the result
explicitly, then run the missing-path test on Windows before trusting its exit code.
- On 3.14+,
except A, B: without parentheses is VALID (PEP 758) and catches both. On 3.13 and
earlier the same line is a SyntaxError under the old Python-2 except E, name: grammar, so it
reads like an obvious bug. Check requires-python or the interpreter and probe with python -c
before "fixing" it; on 3.14+ adding parentheses is a style choice, not a correction.
1---2name: coding-python-use-modern-libraries3description: Use when choosing a Python library for a task (HTTP, JSON, XML, TOML, YAML, data models, enums, dates, compression, database, testing, linting and formatting, type checking, CLI parsing, retry/backoff, text encoding, layered configuration, MCP servers), writing new Python code that needs a dependency, or reviewing imports for dated defaults. For building/editing JSON/XML/YAML files specifically, see bitranox:files-edit-json, bitranox:files-edit-xml, bitranox:files-edit-yml. Public, mainstream defaults; adjust per project.4---56# Modern Python library choices78Default to current, well-maintained libraries over dated stdlib or legacy packages when the9better option is clear. Install third-party deps via `uv` (a tool with `uvx`, or a script with10PEP-723 inline metadata run by `uv run`). Libraries on this list are pre-approved: just use11them and let `uv` fetch them, no need to ask. These are general public defaults, not absolutes:12a project's own conventions win.1314## Adding an entry1516Before adding a library here, vet it: trustworthy (reputable maintainer or community, not a17typo-squat), common (widely adopted), and modern (actively maintained, current releases). Each18new row must carry: a short **description** of what it is for (in the `Use` cell), the older19library/libraries it **replaces** (in the `Avoid` cell), and - **whenever the pick is not20self-evident** - **why** it is better (the parenthetical note in the `Use` cell). A swap that21explains itself (`pathlib.Path` over `os.path`, `pytest` over `unittest`) needs no parenthetical;22a contested or trade-off pick (`isal` vs `deflate`, one MySQL driver over another) always does.23Keep rows one line; do not add anyone's private or in-house packages here.2425If the new library overlaps in function with an existing entry, resolve it - never leave two26rows silently competing for the same job:27- **Both have a place:** keep both and make the distinction explicit - state when to use which28 and why (e.g. one streaming vs one one-shot, sync vs async, simple vs full-featured).29- **The new one supersedes the old:** only when you are sure, replace the old row with the new30 one, move the now-obsolete library into the new row's `Avoid` cell, and say why it was31 replaced.3233## Quick reference3435`Use` names the pick with a brief why; `Avoid` names what it replaces.3637| Task | Use | Avoid |38|-------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|39| HTTP / REST | `httpx2` (HTTP/2; the new Pydantic-org successor to httpx) or httpx | `requests` |40| JSON | `orjson` (fast, correct, bytes) | `json` stdlib for hot paths |41| TOML | `rtoml` | `tomllib`, `tomli` |42| YAML | `ruamel.yaml` (round-trips comments) | `PyYAML` |43| XML | `lxml` (XPath, schema validation, fast C parser) | `xml.etree`, `minidom`, `xmltodict` |44| Structured data (pure / internal layers) | `dataclasses` (stdlib, lightweight; trusted internal data) | bare `dict`, `attrs`, hand-rolled classes |45| Structured data at boundaries (parse input) | `pydantic` (validates + parses untrusted input at the edge) | bare `dict`, `attrs`, hand-rolled parsing |46| Enums | `IntEnum` / `StrEnum` (`StrEnum` is 3.11+) | plain `Enum`, magic strings |47| Terminal output | `rich` | `colorama` (fallback only) |48| TUI | `textual` | `curses` |49| Paths | `pathlib.Path` | `os.path` |50| Date / time | stdlib `datetime` + `zoneinfo` (tz-aware) | `pytz`, naive datetimes |51| Compression (streaming / web / high speed) | `isal` (igzip) - speed tuned, bigger files | `gzip` stdlib for throughput |52| Compression (archival, high ratio) | `deflate` (libdeflate bindings, high ratio, smaller files, C-extension dependency) | `gzip` stdlib |53| .env files | `python-dotenv` | manual parsing |54| Database (ODBC) | `pyodbc` | raw ODBC bindings |55| Database (MySQL) | `mysql-connector-python` or `SQLAlchemy` | `PyMySQL`, `mysqlclient` |56| ORM / complex queries | `SQLAlchemy` | custom ORM, raw SQL for complex apps |57| Testing | `pytest` | `unittest` |58| Lint / format | `ruff` (`ruff check` + `ruff format`; one Rust-fast tool replacing the whole flake8/black/isort stack, single config in pyproject) | `flake8`, `black`, `isort`, `pylint`, `pyupgrade`, `autoflake` |59| Type checking | `pyright` in strict mode (fast, no plugin setup, same engine as the editor's Pylance, narrows better on `TypedDict`/`Protocol`/generics) | `mypy`, untyped code, blanket `# type: ignore` |60| CLI args / parsing | `rich-click` (Click-based, rich-formatted --help; drop-in for click) | `argparse`, `optparse`, `getopt`, bare `click` |61| Subprocess | `subprocess.run([...])` (argv list) | `os.system`, `shell=True` |62| PowerShell / Windows + OS admin objects | `pwshpy` (typed records + lazy pipeline over native OS bindings, real exceptions, memory-bounded; use over the Subprocess row when the job is querying/mutating OS objects - services, registry, event log, ACLs, tasks, accounts - not launching a program; see bitranox:coding-python-pwshpy) | shelling out to `pwsh.exe` / `powershell` / `wevtutil` / `sc`; raw `pywin32` / `wmi` + manual text parsing |63| Retry / backoff | `tenacity` (declarative retry, exponential backoff + jitter; see bitranox:coding-resilience) | hand-rolled `while`/`sleep` retry loops |64| .gitignore parse / file filtering | `igittigitt` (git-exact, include mode, memory-bounded; see bitranox:coding-python-gitignore) | hand-rolled `fnmatch`/`glob`/`re`; `gitignore_parser`; `pathspec` |65| Text encoding / mojibake repair | `ftfy` (repairs mixed / double-encoded mojibake, e.g. `ü`->`ü`; leaves already-correct text untouched) | blanket `.encode('latin-1').decode('utf-8')` round-trips; manual char swaps |66| MCP server / client (Model Context Protocol) | `fastmcp` (decorator API for tools/resources/prompts plus auth, server composition, proxying, OpenAPI generation, built-in testing) | hand-rolled JSON-RPC; the official `mcp` SDK's low-level server API and its bundled `mcp.server.fastmcp` (frozen 1.0 feature set) |67| ICMP ping / reachability / traceroute / port scan / MAC / ARP / routes / interfaces | `ipscout` (in-process, no subprocess; unprivileged ICMP via the ping socket on Linux/macOS and iphlpapi on Windows, so no root or Administrator; also port scanning, neighbour and route lookup, subnets, wake-on-LAN and path MTU; see bitranox:coding-python-network-probe) | `icmplib` (forces raw sockets on Windows, needs Administrator), `scapy`, `python-nmap`, `netifaces` (unmaintained, needs a C build), `subprocess` around `ping`/`tracert`/`arp`/`ip` |68| Layered / cross-platform app config | `lib_layered_config` (merges defaults/app/host/user/.env/env into one immutable object with per-key provenance and profiles; resolves Linux/macOS/Windows paths; library + CLI; see bitranox:coding-python-layered-config) | ad-hoc `os.environ` reads plus scattered file loads with hand-rolled precedence |6970## Structured data: pydantic at the edges, dataclasses inside7172For any structured data in a Python app, parse untrusted/external input into `pydantic` models at73every boundary, and use `dataclasses` for pure internal layers. Do NOT use `attrs`, hand-woven74classes, or raw `dict`s for structured data. For the full end-to-end discipline (parse once at the75boundary, typed models throughout, Enums for fixed values, minimal conversions), use the76`bitranox:coding-python-enforce-data-architecture-strict` skill.7778## HTTP example (httpx2)7980 import httpx2 as httpx # API-compatible with httpx81 with httpx.Client(timeout=30.0, proxy="http://user:pass@host:port") as client:82 r = client.get("https://api.example.com/data")83 r.raise_for_status()84 data = r.json()8586Use a `Client` for many requests (connection reuse), and `AsyncClient` under asyncio. Both87support `proxy=` (one proxy), `mounts=` (per-scheme proxy transports), `timeout=`, and HTTP/2.88There is no `proxies=` kwarg - httpx dropped it in 0.28 and httpx2 never had it, so passing it89raises `TypeError: Client.__init__() got an unexpected keyword argument 'proxies'`.9091`httpx2` (`github.com/pydantic/httpx2`) is the legitimate Pydantic-org-stewarded successor to httpx92and a drop-in replacement. Some security scanners flag it as a typosquat - that is a FALSE POSITIVE93(verified: Pydantic org + Trusted Publisher + Sigstore). If a scanner re-flags it, re-verify the94publisher independently and surface it to the user rather than auto-dismissing; do not silently95swap it out.9697## Notes9899- Prefer a library over an external command-line tool: `httpx2` instead of shelling out to100 `curl`, stdlib/`orjson` instead of `jq`, `re`/stdlib instead of `grep`/`sed`. Libraries are101 the same on every OS; external commands may be missing or take different flags on Windows.102- Reach for stdlib when no third-party library is warranted (small glue, no hot path): the103 point is "best tool", not "most dependencies".104- Logging and CLI framework are deliberately left to each project's own conventions rather105 than prescribed here.106- **`subprocess` raises a DIFFERENT exception type per OS for the same condition, so never branch107 on which one you got.** A missing `cwd` raises `FileNotFoundError` on POSIX but108 `OSError [WinError 267] "The directory name is invalid"` on Windows, so a109 `FileNotFoundError -> 127 / OSError -> 126` mapping returns two different exit codes for one110 condition - and the POSIX run stays green because POSIX happens to raise the type the code111 expected. Pre-check the path yourself (`Path(cwd).exists()` / `.is_dir()`) and decide the result112 explicitly, then run the missing-path test on Windows before trusting its exit code.113- **On 3.14+, `except A, B:` without parentheses is VALID (PEP 758) and catches both.** On 3.13 and114 earlier the same line is a `SyntaxError` under the old Python-2 `except E, name:` grammar, so it115 reads like an obvious bug. Check `requires-python` or the interpreter and probe with `python -c`116 before "fixing" it; on 3.14+ adding parentheses is a style choice, not a correction.