Python Native: Use the Standard Library Before Writing Code
Python's standard library is one of the largest and most polished in any language. Most "utility"
code written by AI assistants — custom dispatch tables, ad-hoc memoization, manual grouping loops,
hand-rolled context managers, reinvented namedtuples — is already a one-liner in the stdlib.
Core Principle: Before writing a loop, a class, or a helper, ask: "Is this already in
functools,itertools,collections,operator, orcontextlib?" The answer is "yes" more often than not. Check the stdlib first.
Steps
- Determine the scope.
- Single function or short snippet (to write, refactor, or review): read it whole, plus any surrounding context that defines its inputs/outputs.
- Whole file or module (to refactor, optimize, or review): read it function by function, never the whole file at once. Function boundaries are the top-level
def/async defblocks (and methods inside a class). Process one function at a time and, within each function, apply the following steps statement by statement.
- Determine the target version first, before writing anything:
python --version, orrequires-pythoninpyproject.toml. Every replacement you propose must exist in that version — checkreferences/python-3.<X>.mdwhen in doubt. - Detect what must change. For each function, walk the §0 Mental Checklist item by item: does the body show any of the listed triggers (a dict-building loop, an
isinstancechain, a hand-rolled memo dict,sorted(...)[:k],list.pop(0), ...)? Also scan its statements against the left column of the §17 table. Each match is a defect to fix, not a style hint. If nothing matches, the function passes — do not rewrite working stdlib code. - Rewrite only the matched fragments. Pick the stdlib replacement from §0/§17, then WebFetch
docs.python.org/3/library/<module>.htmlfor its exact signature and nearest code example — REQUIRED even if you remember the API — and mirror the documented shape when applying targeted edits. Keep the function's logic, variable names, and style intact outside the replaced fragment. - Self-check the replacement before moving on: version available? Caveats respected (
groupbyneeds sorted input,@cachepinsself,zipneedsstrict=True, hashable fields infrozendataclasses, ...)? Deviation needs one stated reason. - Final pass, once per file: re-scan the whole edited source against the §17 table and
references/anti-patterns.md. Fix anything still matching before delivering. - Deliver function by function, with one line per change (function name + what was replaced and with what), before moving to the next function.
Enforcement — strict, not advisory
Every rule in this skill is a requirement. Read every "prefer X" as "emit X".
- Any pattern in §17 or
references/anti-patterns.mdappearing in emitted code is a defect, same severity as a logic bug. Rewrite before presenting. - Logic that already exists as a stdlib primitive is replaced at its first occurrence. Your own logic repeated a third time is extracted into one function. Do NOT emit copy-pasted variants of the same block.
- Fetching docs.python.org for every non-trivial stdlib API is a REQUIRED step, not advisory. Code emitted without the corresponding fetch (where WebFetch is available) is a defect, same severity as a §17 match.
- Two passes, every time: the §0 checklist BEFORE writing; a scan against the §17 table AFTER writing.
- Deviating from a rule requires one stated line: the target version lacks the feature, a measured performance need, or the user's explicit instruction. No silent deviations.
- Only the user's explicit instruction and the edited file's existing convention outrank this skill.
Verify APIs in the official docs — WebFetch is MANDATORY, never memory
You MUST fetch the official documentation BEFORE emitting or proposing any stdlib API call beyond trivial built-ins. This is a hard precondition, not a fallback for uncertainty: do not rely on memory even for APIs you "know". One fetch per module can cover several functions from that module; a module already fetched this session does not need a re-fetch.
Division of labor — references/ covers LOOKUP, docs.python.org covers the SIGNATURE:
references/stdlib-decision-tree.mdpicks WHICH module/function solves the task.references/python-3.<X>.mdsettles VERSION availability for the pinned target — do not fetch whatsnew for what these files already state.references/anti-patterns.mdcatalogs what to replace.- docs.python.org supplies the exact signature, parameter names, defaults, and the nearest official code example. The local files deliberately omit full signatures — that gap is exactly what the mandatory fetch fills.
So the mandatory order is: consult references/ first (local, instant), then fetch the doc page for the chosen API. Skipping the fetch is only allowed when the local file fully answers the question (pure version availability) or for the trivial built-ins listed below.
| Need | URL |
|---|---|
| Module API + official code examples | https://docs.python.org/3/library/<module>.html |
| Behavior on a pinned version | https://docs.python.org/3.<X>/library/<module>.html |
| What a version added or removed | https://docs.python.org/3/whatsnew/3.<X>.html |
| Design rationale / edge semantics | https://peps.python.org/pep-XXXX/ |
- DO determine the target version first:
python --version, orrequires-pythoninpyproject.toml. - DO check
references/python-3.<X>.mdfor version availability BEFORE fetching; if those files settle the question, no whatsnew fetch is needed. - DO mirror the documented example closest to the task instead of inventing a shape.
- DO make the fetch prompt ask for the exact signature and the nearest code example, not a summary.
- DO fetch when a behavior detail decides correctness:
groupbyrequires sorted input,teebuffers unevenly-consumed iterators,lru_cachepinsself,ziptruncates silently withoutstrict=True. - Do NOT fetch ONLY when one of these holds: trivial unambiguous built-ins (
len,range,print, bareenumerate); a module already fetched this session; or a version-availability question fully answered byreferences/python-3.<X>.md. Signature-level detail NEVER has a local substitute — it REQUIRES the fetch. If WebFetch is unavailable in the environment, say so explicitly and mark the affected API as unverified; do not silently fall back to memory.
0. The Mental Checklist (Run This Before Writing Any Python)
Every time you're about to write Python, walk this checklist:
- Loop building up a dict/list/set? →
collections.Counter,defaultdict, comprehension, oritertools. - Dispatching on type or value? →
functools.singledispatchormatch/case(3.10+). - Caching results of a pure function? →
functools.cache(3.9+) orlru_cache. - Sorting/finding by an attribute or key? →
operator.itemgetter/attrgetter, notlambda. - Combining or slicing iterables? →
itertools.chain,islice,pairwise,batched. - Managing setup/teardown? →
contextlib.contextmanager,ExitStack,suppress,nullcontext. - Holding plain data? →
dataclasses.dataclass,typing.NamedTuple, ortypes.SimpleNamespace. - Enumerations or flags? →
enum.Enum,IntEnum,Flag,StrEnum(3.11+). - Filesystem paths? →
pathlib.Path, neveros.pathstring juggling. - Priority queue / k-smallest / k-largest? →
heapq.nsmallest,nlargest,heappush. - Sorted insertion / binary search? →
bisect.insort,bisect_left,bisect_right. - Running statistics? →
statistics.mean,median,stdev,fmean,correlation. - Cryptographic / token / password use? →
secrets, neverrandom. - Random sampling without replacement? →
random.sample(not a manual loop). - Need an iterator-aware helper? →
itertools.tee,accumulate,starmap,groupby.
If any answer is "yes", stop and use the stdlib feature. Reinventing it is a defect — rewrite before emitting.
1. functools — The Most Under-Used Module
1.1 singledispatch — Polymorphism Without Class Hierarchies
Replace long isinstance chains and visitor patterns with type-dispatched functions.
from functools import singledispatch
@singledispatch
def serialize(value):
raise TypeError(f"Cannot serialize {type(value).__name__}")
@serialize.register
def _(value: int) -> str:
return str(value)
@serialize.register
def _(value: list) -> str:
return "[" + ", ".join(serialize(v) for v in value) + "]"
@serialize.register(dict)
def _(value):
return "{" + ", ".join(f"{k}:{serialize(v)}" for k, v in value.items()) + "}"
For instance methods, use singledispatchmethod:
from functools import singledispatchmethod
class Renderer:
@singledispatchmethod
def render(self, node):
raise NotImplementedError
@render.register
def _(self, node: str):
return f"<text>{node}</text>"
@render.register
def _(self, node: list):
return "".join(self.render(child) for child in node)
AI anti-pattern: writing
if isinstance(x, int): ... elif isinstance(x, list): .... Usesingledispatch. It's discoverable, extensible by registration, and removes the chain.
1.2 cache and lru_cache — Memoization in One Decorator
from functools import cache, lru_cache
@cache # Python 3.9+, unbounded
def fib(n: int) -> int:
return n if n < 2 else fib(n - 1) + fib(n - 2)
@lru_cache(maxsize=1024) # bounded cache for hot paths
def expensive(query: str) -> bytes:
...
cache=lru_cache(maxsize=None), slightly faster — use it when memory is not a concern.- Arguments must be hashable. Use
frozenset,tuple, or convert before calling. .cache_info()and.cache_clear()are available on both.
CAVEAT — bound methods:
@cache/@lru_cacheon instance methods keepsselfstrongly referenced through the cache key, preventing garbage collection of the instance until the cache is cleared. For per-instance memoization, prefercached_property. For class-level methods, use boundedlru_cache(maxsize=N), weak-key caches, or explicitcache_clear().
1.3 cached_property — Lazy, Memoized Attributes
from functools import cached_property
class Dataset:
def __init__(self, path):
self.path = path
@cached_property
def rows(self):
return self._load() # runs once, then stored as instance attribute
- Stored in
__dict__, so it works only on classes without__slots__(unless you include__dict__). - Replace
@property+ manual_cacheattribute pattern entirely.
1.4 partial and partialmethod — Freezing Arguments
from functools import partial
read_utf8 = partial(open, encoding="utf-8")
debug_log = partial(logger.log, logging.DEBUG)
# Use in callbacks and maps:
sorted(items, key=partial(score, weights=w))
Better than lambda because it's introspectable (has .func, .args, .keywords).
1.5 reduce — Last Resort for Accumulators
reduce is rarely needed: sum, min, max, any, all, math.prod cover most cases.
from functools import reduce
from math import prod
prod([1, 2, 3, 4]) # → 24 (prefer this)
reduce(lambda a, b: a * b, [1, 2, 3, 4]) # works, but verbose
Use reduce only when the operation is genuinely non-builtin and a generator/loop wouldn't be clearer.
1.6 wraps, total_ordering, reduce
from functools import wraps, total_ordering
def trace(fn):
@wraps(fn) # preserves __name__, __doc__, __module__
def inner(*args, **kw):
print(fn.__name__, args, kw)
return fn(*args, **kw)
return inner
@total_ordering # Define __eq__ and one of <, <=, >, >= — get the rest for free
class Version:
def __init__(self, parts): self.parts = parts
def __eq__(self, other): return self.parts == other.parts
def __lt__(self, other): return self.parts < other.parts
2. itertools — The Iterator Algebra
Iterators are O(1) memory. Use these primitives instead of building lists and re-iterating.
2.1 Combining Iterables
from itertools import chain, zip_longest, product
list(chain([1, 2], [3, 4], [5])) # [1, 2, 3, 4, 5]
list(chain.from_iterable([[1, 2], [3, 4]])) # [1, 2, 3, 4] ← flatten one level
list(zip_longest("ABCD", "12", fillvalue="-")) # [('A','1'),('B','2'),('C','-'),('D','-')]
list(product([0, 1], repeat=3)) # all 3-bit tuples
2.2 Slicing and Windowing
from itertools import islice, takewhile, dropwhile, pairwise # pairwise: 3.10+
islice(iterable, 10) # first 10 elements lazily — works on generators
list(pairwise([1, 2, 3, 4])) # [(1,2), (2,3), (3,4)]
list(takewhile(lambda x: x < 5, [1, 3, 5, 2])) # [1, 3]
For Python 3.12+: itertools.batched(iterable, n) chunks an iterable into n-tuples.
2.3 Grouping (Replaces Manual defaultdict Loops)
from itertools import groupby
from operator import itemgetter
rows = sorted(rows, key=itemgetter("dept")) # groupby requires sorted input!
for dept, group in groupby(rows, key=itemgetter("dept")):
print(dept, list(group))
Common bug: forgetting that
groupbyonly groups adjacent equal keys. Sort first.
2.4 Accumulators and Counters
from itertools import accumulate, count, cycle, repeat
list(accumulate([1, 2, 3, 4])) # [1, 3, 6, 10] running sum
list(accumulate([1, 2, 3, 4], initial=100)) # [100, 101, 103, 106, 110]
list(accumulate([3, 1, 4, 1, 5], max)) # [3, 3, 4, 4, 5] running max
for i in count(start=1): ... # unbounded counter
for color in cycle(["r", "g", "b"]): ... # infinite cycle
2.5 Combinations and Permutations
from itertools import combinations, permutations, combinations_with_replacement
list(combinations("ABC", 2)) # [('A','B'), ('A','C'), ('B','C')]
list(permutations("ABC", 2)) # [('A','B'),('A','C'),('B','A'),...]
Replace any nested loop that filters duplicates with the right combinatoric primitive.
2.6 tee and starmap
from itertools import tee, starmap
a, b = tee(iterator) # split one iterator into two independent ones
list(starmap(pow, [(2, 3), (10, 2)])) # [8, 100] — like map() but unpacks each tuple
Caveat on
tee: when the two consumers advance unevenly,teeaccumulates an internal buffer of every element produced since the slower consumer last advanced. On long iterators with skewed consumption, this can blow memory. Useteeonly when consumers stay roughly in step, or materialize the iterator into a list explicitly.
3. collections — Data Structures You Should Default To
3.1 Counter — Frequency Counting in One Line
from collections import Counter
c = Counter("abracadabra") # Counter({'a': 5, 'b': 2, 'r': 2, ...})
c.most_common(3) # [('a', 5), ('b', 2), ('r', 2)]
c + Counter("aaa") # multiset addition
c & Counter("abxxx") # multiset intersection
+c # drop zero/negative counts
AI anti-pattern:
freq = {}; for x in xs: freq[x] = freq.get(x, 0) + 1. UseCounter(xs).
3.2 defaultdict — Auto-Initialized Dicts
from collections import defaultdict
groups = defaultdict(list)
for user, action in events:
groups[user].append(action)
graph = defaultdict(set)
for a, b in edges:
graph[a].add(b)
graph[b].add(a)
Replaces the dict.setdefault pattern and the if key not in d: d[key] = [] antipattern.
3.3 deque — O(1) Append/Pop at Both Ends
from collections import deque
q = deque(maxlen=1000) # bounded ring buffer — drops oldest on overflow
q.appendleft(x); q.pop(); q.rotate(-1)
Use deque for:
- BFS queues
- Sliding windows
- Rolling logs / ring buffers (
maxlen) - Any FIFO —
list.pop(0)is O(n).
3.4 ChainMap — Layered Lookups
from collections import ChainMap
config = ChainMap(cli_args, env_vars, defaults) # lookup falls through in order
Avoid manually merging dicts to implement precedence; ChainMap does it without copying.
3.5 namedtuple (and Why Often dataclass Wins)
from collections import namedtuple
Point = namedtuple("Point", "x y")
p = Point(3, 4)
p.x, p.y, p._asdict(), p._replace(x=10)
- Use
namedtuplefor immutable lightweight records. - Use
typing.NamedTuplefor type hints. - Use
dataclass(frozen=True, slots=True)when you want defaults, methods, or richer behavior.
4. operator — Functions for Operators
Replaces tiny lambdas with introspectable, pickleable callables.
from operator import itemgetter, attrgetter, methodcaller, mul
sorted(people, key=attrgetter("age")) # vs. lambda p: p.age
sorted(rows, key=itemgetter("dept", "salary")) # multi-key
list(map(methodcaller("strip"), lines)) # vs. lambda s: s.strip()
list(map(mul, xs, ys)) # elementwise multiply
itemgetter("a", "b")returns a tuple — useful for sort keys.methodcaller("split", ",")is callable; great inmap,filter,sort.
5. contextlib — Context Managers Beyond with open(...)
5.1 @contextmanager — Build One in 4 Lines
from contextlib import contextmanager
@contextmanager
def timer(label):
start = time.perf_counter()
try:
yield
finally:
print(f"{label}: {time.perf_counter() - start:.3f}s")
with timer("load"):
data = load()
5.2 suppress, nullcontext, closing, redirect_stdout
from contextlib import suppress, nullcontext, closing, redirect_stdout
import io
with suppress(FileNotFoundError):
os.remove(path) # silently no-op if missing
cm = open(path) if log else nullcontext() # uniform `with` without branching
with cm as f: ...
with closing(urllib.request.urlopen(url)) as resp: ...
buf = io.StringIO()
with redirect_stdout(buf):
noisy_function()
5.3 ExitStack — Dynamic Stacks of Context Managers
from contextlib import ExitStack
with ExitStack() as stack:
files = [stack.enter_context(open(p)) for p in paths] # any number, any order
process(files)
Replaces deeply nested with blocks and conditional setup/teardown.
6. dataclasses — The Right Default for Data Classes
from dataclasses import dataclass, field, asdict, replace
@dataclass(frozen=True, slots=True, kw_only=True) # slots, kw_only: 3.10+
class Order:
id: int
items: tuple[str, ...] = () # tuple, not list — see hash caveat below
total: float = 0.0
o = Order(id=1, items=("a", "b"), total=9.99)
o2 = replace(o, total=10.5) # functional update
asdict(o) # → plain dict
frozen=Trueblocks attribute rebinding and auto-generates__hash__— but the hash succeeds only when every field value is hashable. Alist[str]field will pass type checking yet raiseTypeError: unhashable type: 'list'at hash time. For hashable records, use immutable field types (tuple,frozenset,str, numbers) or mark mutable fieldsfield(compare=False, hash=False)to keep them out of the generated__hash__and__eq__.slots=True→ smaller memory, faster attribute access, no accidental new attrs. Use slots for stable leaf data models.kw_only=True→ forces keyword args; prevents positional-arg bugs.field(default_factory=list)→ never use a mutable default directly (default values are evaluated once at class-definition time, so a bareitems: list = []would share one list across all instances).
For inherited or computed fields, use field(init=False, default=...) and __post_init__.
CAVEAT — slots edge cases: multiple inheritance with slots can hit layout conflicts when multiple bases declare their own
__slots__— test before enabling on hierarchical classes. Breaksweakref.ref()unless you passweakref_slot=True(3.11+). Breaks@cached_propertybecause slots prevents writing to__dict__.
7. pathlib — Stop Using os.path Strings
from pathlib import Path
p = Path("/var/log") / "app" / "today.log" # join with /
p.parent, p.name, p.stem, p.suffix
p.exists(), p.is_file(), p.with_suffix(".bak")
p.read_text(encoding="utf-8")
p.write_text("hello", encoding="utf-8") # always pass encoding for text files
list(p.parent.glob("*.log"))
list(p.parent.rglob("*.py"))
Path.home(), Path.cwd()
pathlib is OS-agnostic, returns rich objects, and supports operators. There is no good reason
to use os.path.join in new code.
8. heapq and bisect — Specialized Algorithms
8.1 heapq — Priority Queues and Top-K
import heapq
heap = []
for x in xs:
heapq.heappush(heap, x)
smallest = heapq.heappop(heap)
heapq.nsmallest(5, items, key=lambda x: x.cost) # O(n log k), not full sort
heapq.nlargest(10, scores)
AI anti-pattern:
sorted(xs)[:k]to get the k smallest, when k is much smaller than n. Usensmallest(k, xs)— O(n log k), which beats sort for smallkand for streaming inputs. Whenkapproachesn(rule of thumb:k ≥ n/3) or you also need the rest of the sequence sorted, plainsorted(xs)[:k]is competitive or faster.
8.2 bisect — Sorted Lists Without Re-Sorting
import bisect
bisect.insort(sorted_list, value) # insert keeping sort order
i = bisect.bisect_left(sorted_list, value) # leftmost insertion point
j = bisect.bisect_right(sorted_list, value) # rightmost — count = j - i
Use bisect for sorted in-memory indexes, range queries, and rank lookups.
9. enum — Real Enumerations
from enum import Enum, IntEnum, Flag, auto
class Status(Enum):
PENDING = auto()
ACTIVE = auto()
DONE = auto()
class Permission(Flag):
READ = auto()
WRITE = auto()
EXEC = auto()
ALL = READ | WRITE | EXEC
Permission.READ | Permission.WRITE # composable bitfield
- Use
Enumfor closed sets of named constants. - Use
IntEnumonly when integer interop matters. - Use
StrEnum(3.11+) for string-based enums (no manual.valueeverywhere). - Use
Flag/IntFlagfor bitfields.
10. typing — Use the Modern Forms
from typing import Protocol, NamedTuple, TypedDict, Literal, Annotated, Self, TypeAlias
# Structural typing — duck typing with checking
class Closeable(Protocol):
def close(self) -> None: ...
# Lightweight schema for dicts
class User(TypedDict):
id: int
name: str
# Constrain accepted values
Mode = Literal["r", "w", "a"]
# Builder pattern with self-type (3.11+)
class Query:
def where(self, **kw) -> Self: ...
def order_by(self, key) -> Self: ...
- Built-in generics:
list[int],dict[str, int],tuple[int, ...](3.9+) — no need forList,Dict. X | Yunion syntax (3.10+) replacesUnion[X, Y].X | NonereplacesOptional[X].TypeAlias(3.10+) for documented aliases;typestatement (3.12+).
11. match / case — Structural Pattern Matching (3.10+)
def evaluate(node):
match node:
case {"op": "add", "left": l, "right": r}:
return evaluate(l) + evaluate(r)
case [first, *rest]:
return [evaluate(first), *map(evaluate, rest)]
case int() | float():
return node
case Point(x=0, y=y):
return f"on y-axis at {y}"
case _:
raise ValueError(node)
- Patterns include literals, types, sequences, mappings, class patterns, OR (
|), guards (if). - Names in patterns bind —
case x:matches anything and binds. Usecase _:for wildcard. - Use class patterns to destructure dataclasses by attribute.
12. Walrus Operator (:=) — Use Where It Reduces Repetition
# Read in chunks until EOF
while chunk := f.read(4096):
process(chunk)
# Reuse a computed value in a comprehension
results = [(x, y) for x in xs if (y := compute(x)) is not None]
# Avoid double-call in an if
if (match := pattern.search(line)) is not None:
use(match.group(1))
Don't use it just to be clever; use it when it removes a duplicated computation.
13. f-strings — Use the Full Format Spec
f"{value:>10.2f}" # right-aligned, width 10, 2 decimals
f"{value:,.0f}" # thousands separators
f"{value:.2%}" # percent
f"{n:#06x}" # 0x00ff
f"{name!r}" # repr()
f"{obj=}" # → "obj=<value>" (3.8+, great for debug)
f"{dt:%Y-%m-%d}" # datetime format
f"{a + b = :,}" # debug + format spec combined
f-strings (3.12+, PEP 701) allow reusing the same quote character inside the expression:
f"{ ", ".join(parts) }" # outer and inner both double-quoted, SyntaxError before 3.12
14. Generators and Comprehensions
- Prefer generator expressions for one-shot iteration:
sum(x*x for x in xs)— no list built. - Use
yield fromfor delegation:yield from inner()is equivalent to aforloop. - A function with
yieldis already an iterator; no class needed. - Generators are O(1) memory — chain them with
itertools.chainfor streaming pipelines.
def lines_of(path):
with open(path, encoding="utf-8") as f:
yield from f # yields lines lazily
def non_empty(it):
return (line.strip() for line in it if line.strip())
for line in non_empty(lines_of("big.txt")):
process(line)
15. Built-in Functions That Are Often Forgotten
| Function | What it does that you might re-implement |
|---|---|
any(p(x) for x in xs) |
Short-circuit "exists" — don't loop and return early manually. |
all(...) |
Short-circuit "for all". |
sum(iter, start=0) |
Numeric sums with optional start; start cannot be a string (use ''.join()); for concatenating iterables use itertools.chain.from_iterable. Use math.prod for products. |
min(iter, key=, default=) / max(...) |
Use key= instead of pre-sorting. default= for empty iterables. |
sorted(iter, key=, reverse=) |
Stable sort. key is computed once per element. |
reversed(seq) |
O(1) lazy iterator over a reversible sequence or any object with __reversed__; generators are not reversible — materialize to a list first if needed. |
enumerate(iter, start=N) |
Pass start= instead of i + offset. |
zip(*iters, strict=True) |
strict=True (3.10+) raises on length mismatch — use it. |
divmod(a, b) |
Returns (quotient, remainder) — replaces two operations. |
pow(b, e, mod) |
Modular exponentiation — vastly faster than (b**e) % mod. |
round(x, n) |
Banker's rounding by default; pass ndigits for precision. |
iter(callable, sentinel) |
Loops calling callable() until it returns sentinel. |
next(iter, default) |
Pass a default to avoid StopIteration handling. |
vars(obj) |
Inspect attributes — sometimes simpler than __dict__. |
getattr(obj, name, default) |
Don't try/except AttributeError — pass default. |
hash(x) |
Test hashability quickly. |
id(x) / is |
Identity, not equality. Don't use == for None / singletons. |
16. Other Stdlib Modules Worth Reaching For
| Module | Use when… |
|---|---|
statistics |
Mean, median, stdev, mode, correlation, linear regression — no need for NumPy on small data. |
math |
gcd, lcm, isclose, prod, comb, perm, hypot. |
secrets |
Tokens, passwords, anything security-sensitive. Never random here. |
random |
Non-security randomness. Use random.choices (weighted), sample (no replacement). |
hashlib |
Hashing. Use hashlib.file_digest (3.11+) for streaming file hashes. |
struct |
Pack/unpack binary data. |
io.StringIO, io.BytesIO |
In-memory file-like objects for testing or pipelines. |
tempfile |
TemporaryDirectory, NamedTemporaryFile. Don't hand-roll temp files. |
shutil |
File ops (copy, move, rmtree, which, make_archive). |
subprocess |
subprocess.run(..., check=True, capture_output=True, text=True). |
argparse |
Don't parse sys.argv by hand. |
logging |
Use the logging tree — don't sprinkle print(). |
json |
Use default= hook for custom serializers; indent=2 for human output. |
csv |
csv.DictReader / DictWriter — never split(","). |
sqlite3 |
Built-in DB. Great for caches, tests, prototypes. |
concurrent.futures |
ThreadPoolExecutor / ProcessPoolExecutor — don't manage threads by hand. |
asyncio |
asyncio.run, gather, TaskGroup (3.11+), to_thread. |
weakref |
WeakValueDictionary, WeakSet — caches that don't keep objects alive. |
types.SimpleNamespace |
Quick attribute-bag without defining a class. |
types.MappingProxyType |
Read-only view of a dict. |
abc |
ABC, @abstractmethod for interfaces. |
dataclasses |
See §6. |
typing |
See §10. |
textwrap |
dedent, indent, fill, shorten. |
string |
string.Template, string.ascii_letters, digits, Formatter. |
unicodedata |
Normalize, categorize, strip accents. |
decimal |
Exact decimal arithmetic (money). Never use float for currency. |
fractions |
Exact rationals. |
ipaddress |
Parse and manipulate IPs/networks; don't regex them. |
urllib.parse |
URL parsing / encoding; don't string-manipulate URLs. |
inspect |
Introspect signatures, source, callables. |
traceback |
Format/print tracebacks programmatically. |
pprint |
Readable structure dumps. |
17. Things AI Routinely Reinvents (Stop Doing These)
This is the post-write scan table: any left-column pattern in emitted code is a bug — replace it with the right column before presenting.
| Reinvention | Replace with |
|---|---|
Manual freq = {} loop |
Counter(iterable) |
dict.setdefault(k, []).append(v) |
defaultdict(list) |
if x not in d: d[x] = … |
dict.setdefault or defaultdict |
sorted(x)[:k] for k-smallest |
heapq.nsmallest(k, x) |
sorted(x, reverse=True)[:k] for k-largest |
heapq.nlargest(k, x) |
list.pop(0) for queues |
collections.deque |
isinstance chains |
functools.singledispatch or match |
lambda p: p.attr for sort |
operator.attrgetter("attr") |
| Hand-rolled memo dict | functools.cache / lru_cache |
| Hand-rolled context manager class | @contextlib.contextmanager |
| Manual flattening with nested loops | itertools.chain.from_iterable |
| Manual pairwise loop | itertools.pairwise (3.10+) |
| Manual chunking loop | itertools.batched (3.12+) |
Hand-rolled Result/Maybe for dispatch |
match / case |
class Foo: __init__: self.x=x; self.y=y; __repr__=... |
@dataclass |
os.path.join(...) strings |
pathlib.Path(...) |
Manual try/except for "absent file" deletion |
contextlib.suppress(FileNotFoundError) |
| Hand-rolled CLI parsing | argparse |
random.SystemRandom() for tokens |
secrets.token_urlsafe(n) |
18. Version Support Strategy (3.9 → 3.14)
This skill works on Python 3.9+. When a feature is newer, prefer it on supported versions but fall back to the stdlib equivalent on older ones. Always check the target's Python version first.
- Look in
references/python-3.X.mdfor the precise list of features introduced or changed. - When unsure of version:
python --version, or readpyproject.toml(requires-python). - Use
sys.version_info >= (3, 11)for runtime branches; do not rely ontry/except ImportErrorunless the feature actually moved between modules.
PEP 594 dead-battery removals in 3.13
If your code imports any of these, plan a replacement before upgrading. The 19 modules removed per PEP 594 (Python 3.13): aifc, audioop, cgi, cgitb, chunk, crypt, imghdr, mailcap, msilib, nis, nntplib, ossaudiodev, pipes, sndhdr, spwd, sunau, telnetlib, uu, xdrlib. Separately removed: lib2to3 (the 2to3 source-migration tool).
Compatibility Cheat-Sheet (Highlights)
| Feature | Introduced |
|---|---|
functools.cache, dict | dict merge, removeprefix/suffix, built-in generics (list[int]) |
3.9 |
match / case, X | Y unions, zip(strict=True), dataclass(slots=, kw_only=), itertools.pairwise, parenthesized context managers |
3.10 |
tomllib, ExceptionGroup, except*, Self, StrEnum, TaskGroup, hashlib.file_digest, exception notes (__notes__) |
3.11 |
itertools.batched, type statement, PEP 695 generics, @override, PEP 701 f-strings, per-interpreter GIL (PEP 684), comprehension inlining, pathlib.Path.walk, random.binomialvariate |
3.12 |
Free-threaded build (PEP 703 experimental), JIT (PEP 744 experimental), dbm.sqlite3, new REPL, itertools.batched(strict=), copy.replace, typing.TypeIs/ReadOnly, TypeVar defaults (PEP 696), os.process_cpu_count, removal of 19 dead-battery modules |
3.13 |
Deferred annotations (PEP 649) + annotationlib, t-strings (PEP 750), concurrent.interpreters (PEP 734), InterpreterPoolExecutor, free-threaded officially supported (PEP 779), pathlib.Path.copy/move |
3.14 |
See the references/ directory for the full feature list per version.
19. Reference Files
For per-version deep dives, consult:
references/python-3.9.md— built-in generics,cache,removeprefix/suffix, dict unionreferences/python-3.10.md—match/case,X | Y,zip(strict=),dataclass(slots=), parenthesizedwithreferences/python-3.11.md—ExceptionGroup,Self,StrEnum,tomllib,TaskGroup, fine-grained tracebacksreferences/python-3.12.md— PEP 695 generics,typestatement,@override,itertools.batchedreferences/python-3.13.md— free-threaded build, JIT,dbm.sqlite3, removalsreferences/python-3.14.md— PEP 649 deferred annotations +annotationlib, t-strings (PEP 750),concurrent.interpreters(PEP 734), PEP 779 free-threaded official,pathlib.Path.copy/movereferences/stdlib-decision-tree.md— "I want to do X; which module?" lookup tablereferences/anti-patterns.md— full catalog organized by Correctness / Maintainability / Readability / Performance / Security, expanded from QuantifiedCode's Python Anti-Patterns book and modernized for Python 3.9+
Sources
Content in this skill was verified against the official "What's New in Python" pages:
- Python 3.9: https://docs.python.org/3/whatsnew/3.9.html
- Python 3.10: https://docs.python.org/3/whatsnew/3.10.html
- Python 3.11: https://docs.python.org/3/whatsnew/3.11.html
- Python 3.12: https://docs.python.org/3/whatsnew/3.12.html
- Python 3.13: https://docs.python.org/3/whatsnew/3.13.html
- Python 3.14: https://docs.python.org/3/whatsnew/3.14.html
For PEPs: https://peps.python.org/ (e.g. PEP 585 → https://peps.python.org/pep-0585/). For module references: https://docs.python.org/3/library/.html.
Quick Reference Card
BEFORE writing a loop: Can it be a comprehension / itertools call?
BEFORE writing a class: Can it be @dataclass / NamedTuple / SimpleNamespace?
BEFORE writing isinstance: Can it be singledispatch / match?
BEFORE writing a cache dict: Use functools.cache.
BEFORE writing os.path: Use pathlib.Path.
BEFORE writing lambda key: Use operator.itemgetter / attrgetter.
BEFORE writing try/except: Use contextlib.suppress where applicable.
BEFORE manual freq counting: Use collections.Counter.
BEFORE list.pop(0): Use collections.deque.
BEFORE sorted(...)[:k] (smallest): Use heapq.nsmallest.
BEFORE sorted(..., reverse=True)[:k]: Use heapq.nlargest.
WHEN dispatching: functools.singledispatch or match/case.
FOR EVERY non-trivial API: Check references/ (decision-tree + python-3.<X>) locally, then WebFetch docs.python.org/3/library/<module>.html for the signature. Memory alone is a defect; "I already know this API" is NOT an excuse.
AFTER writing: Scan against §17 — any match is a defect; fix before emitting.
WHEN in doubt: Search the stdlib index before writing code.