Python Dev

Dependencies

If you need to install, update, or otherwise modify Python dependencies, use the python-dependencies skill.

Python version

Assume you are using Python 3.10 or greater; older versions have reached end of life. This means you can safely use the following:

• Improved type hinting, with less reliance on Typing module

  • Generics can be used --- such as list[str] and dict[str, int] --- without needing to use List, Dict, etc. from the Typing module
  • Can use | for union types. For example, use str | None instead of Optional[str] and int | float instead of Union[int, float]

• The walrus operator (:=) that assigns values to variables as part of a larger expression. Don't over use this, but use it where it is clear and idiomatic

  if (n := len(a)) > 10:  # Assign to `n` and return; avoids redundant `len` call.
      print(f"List is too long ({n} elements, expected <= 10)")
  
  # Regular expression match objects
  discount = 0.0
  if (mo := re.search(r'(\d+)% discount', advertisement)):
      discount = float(mo.group(1)) / 100.0
  
  while (block := f.read(256)) != '':
      process(block)
  
  # List comprehensions with filtering conditions
  [clean_name.title() for name in names
   if (clean_name := normalize('NFC', name)) in allowed_names]
  

• Positional-only parameters. Use the function parameter syntax / to indicate that some function parameters must be specified positionally and cannot be used as keyword arguments. One useful application is that it allows the parameter name to be changed in the future without risk of breaking client code.

  # a, b -- positional only
  # c, d -- positional or keyword
  # e, f -- required to be keywords
  def f(a, b, /, c, d, *, e, f):
      print(a, b, c, d, e, f)
  
  # Valid
  f(10, 20, 30, d=40, e=50, f=60)
  
  # Invalid
  f(10, b=20, c=30, d=40, e=50, f=60)   # b cannot be a keyword argument
  f(10, 20, 30, 40, 50, f=60)           # e must be a keyword argument
  

• = in f-strings. An f-string such as f'{expr=}' will expand to the text of the expression, an equal sign, then the representation of the evaluated expression

  >>> user = 'eric_idle'
  >>> member_since = date(1975, 7, 31)
  >>> f'{user=} {member_since=}'
  "user='eric_idle' member_since=datetime.date(1975, 7, 31)"
  
  >>> delta = date.today() - member_since
  >>> f'{user=!s}  {delta.days=:,d}'
  'user=eric_idle  delta.days=16,075'
  
  # left side of `=` will always display the whole expression
  >>> print(f'{theta=}  {cos(radians(theta))=:.3f}')
  "theta=30  cos(radians(theta))=0.866"
  

• dict merge (|) and update (|=) operators

• str.removeprefix(prefix) and str.removesuffix(suffix)

• zoneinfo standard library module for getting timezone info from IANA time zone database

  from zoneinfo import ZoneInfo
  from datetime import datetime, timedelta
  
  # Daylight saving time
  dt = datetime(2020, 10, 31, 12, tzinfo=ZoneInfo("America/Los_Angeles"))
  print(dt)
  # 2020-10-31 12:00:00-07:00
  
  dt.tzname()
  # PDT
  
  # Standard time
  dt += timedelta(days=7)
  print(dt)
  # 2020-11-07 12:00:00-08:00
  
  print(dt.tzname())
  # PST
  

• graphlib standard library module with graphlib.TopologicalSorter method

• Structural pattern matching with match <...> statement. For more examples, see pattern-matching.md.

  def http_error(status):
      match status:
          case 400:
              return "Bad request"
          case 404:
              return "Not found"
          case 418:
              return "I'm a teapot"
          # Combine multiple cases with `|`
          case 401 | 403:
              return "Not allowed"
          case _:
              return "Something's wrong with the internet"
  

Logging

Prefer the logging module to print statements in production code. Use print only in short code snippets, minimal examples (when explaining something to the user), or for debugging.

Reporting

Prefer structured reporting via classes and rendering methods. For example:

@dataclass
class BenchmarkResults:
    name: str
    mean_ms: float
    std_ms: float

def render_text_report(list[BenchmarkResults]) -> str:
    lines = [
        "Benchmark Report",
        "=" * 40,
        "",
    ]
    for r in results:
        lines.append(f"{r.name:20s}: {r.mean_ms:.3f} ± {r.std_ms:.3f} ms")
    return "\n".join(lines)

report = render_text_report(results)

with open("report.txt", "w") as f:
    f.write(report)

Avoid long sequences of multiple f.write("...") statements.

Type Safety and Data Structures

Dataclasses for structured data

When defining data structures that hold multiple related fields, prefer dataclasses over dictionaries or plain classes:

from dataclasses import dataclass

@dataclass
class Config:
    model_name: str
    learning_rate: float
    batch_size: int = 32  # with sensible defaults

When to use dataclasses:

• Representing configuration objects
• Return values with multiple related fields
• Domain objects with clear schemas
• Data that gets passed between functions

When dictionaries are fine:

• Simple key-value lookups with dynamic keys
• JSON-like data being serialized/deserialized immediately
• Truly heterogeneous collections
• One-off data structures in small scripts

Type Annotations

Use precise type annotations for function signatures and class attributes, especially in:

• Public APIs and library code
• Functions with non-obvious parameter types
• Code that will be maintained by multiple people

from pathlib import Path

def load_config(path: Path, validate: bool = True) -> Config:
    """Load configuration from file."""
    ...

def process_results(
    data: list[dict[str, float]], 
    threshold: float | None = None
) -> dict[str, list[float]]:
    """Process and group results by category."""
    ...

Don't over-annotate:

• Skip annotations where types are immediately obvious from context
• Use # type: ignore sparingly for legitimate edge cases
• Avoid overly complex generic types that reduce readability

When to skip this pattern

• Quick exploratory scripts or notebooks
• Prototyping where the data structure is still evolving
• Simple scripts under ~100 lines with obvious data flow
• Performance-critical code where overhead matters (though dataclasses are quite fast)

Early Returns

Prefer early returns over nested if/else structures. Early returns reduce nesting depth, make error conditions explicit upfront, and keep the main logic (the "happy path") unindented and easier to follow. This pattern helps agents reason about control flow more clearly.

Simple validation

# ❌ Nested condition obscures main logic
def load_config(path: Path) -> dict[str, Any]:
    if path.exists():
        with open(path) as f:
            return json.load(f)
    else:
        return {}

# ✅ Guard clause makes intent clear; main logic follows naturally
def load_config(path: Path) -> dict[str, Any]:
    if not path.exists():
        return {}
    
    with open(path) as f:
        return json.load(f)

Multiple guard clauses

# ❌ Deep nesting makes code hard to follow
def process_data(data: list[float], threshold: float | None) -> list[float]:
    if len(data) > 0:
        if not any(math.isnan(x) for x in data):
            if threshold is not None:
                return [x for x in data if x > threshold]
            else:
                return data
        else:
            raise ValueError("Data contains NaN values")
    else:
        return []

# ✅ Stack guard clauses at the top; happy path is clear and unindented
def process_data(data: list[float], threshold: float | None) -> list[float]:
    if len(data) == 0:
        return []
    if any(math.isnan(x) for x in data):
        raise ValueError("Data contains NaN values")
    if threshold is None:
        return data
    
    return [x for x in data if x > threshold]  # Main logic is clear

Raising exceptions as early exits

# ❌ Nested validation with else blocks
def calculate_metric(values: list[float]) -> float:
    if isinstance(values, list):
        if all(isinstance(x, (int, float)) for x in values):
            if len(values) >= 3:
                return statistics.mean(values) / statistics.stdev(values)
            else:
                raise ValueError("Need at least 3 values")
        else:
            raise TypeError("All values must be numeric")
    else:
        raise TypeError("values must be a list")

# ✅ Validation guards upfront; calculation stands alone
def calculate_metric(values: list[float]) -> float:
    if not isinstance(values, list):
        raise TypeError("values must be a list")
    if not all(isinstance(x, (int, float)) for x in values):
        raise TypeError("All values must be numeric")
    if len(values) < 3:
        raise ValueError("Need at least 3 values")
    
    return statistics.mean(values) / statistics.stdev(values)

Dataclass and dict validation

# ❌ Unnecessary else after conditional processing
@dataclass
class Config:
    model_name: str
    learning_rate: float
    batch_size: int = 32

def load_experiment(config_dict: dict[str, Any]) -> Config | None:
    if "model_name" not in config_dict:
        logging.warning("Missing model_name in config")
        return None
    else:
        if "learning_rate" not in config_dict:
            logging.warning("Missing learning_rate in config")
            return None
        else:
            return Config(
                model_name=config_dict["model_name"],
                learning_rate=config_dict["learning_rate"],
                batch_size=config_dict.get("batch_size", 32),
            )

# ✅ Handle validation early; construction has no unnecessary else blocks
def load_experiment(config_dict: dict[str, Any]) -> Config | None:
    if "model_name" not in config_dict:
        logging.warning("Missing model_name in config")
        return None
    if "learning_rate" not in config_dict:
        logging.warning("Missing learning_rate in config")
        return None
    
    return Config(
        model_name=config_dict["model_name"],
        learning_rate=config_dict["learning_rate"],
        batch_size=config_dict.get("batch_size", 32),
    )

Paths

Use pathlib.Path objects and their methods when representing and manipulating paths. Avoid using strings for paths.

When writing functions that work with paths, prefer the strict argument: Path annotation.

Be aware that some Python functions may expect strings and be unable to handle Path objects. In these cases, cast the Path to a str when calling the function:

from pathlib import Path

myfile = Path("~").expanduser() / "data" / "myfile"

def some_function(filename: str):  # NOTE: Expects `str`, not `Path`
    # ...

some_function(str(myfile))  # Convert path to string only for the function call

Converted and distributed by TomeVault — claim your Tome and manage your conversions.