Python Troubleshooting

Python Troubleshooting

Overview

Condensed playbook for debugging Python 3.12 compile-time and runtime failures: gather context, reproduce cleanly, classify error type, and apply targeted fixes with quick commands and scripts.

Quick Start

• Capture the exact command, stack trace, and inputs that fail; keep the failing file path and line numbers.
• Confirm interpreter with python --version (must be 3.12.x) and which -a python to spot unexpected shims.
• Reproduce inside a fresh venv: python -m venv .venv && source .venv/bin/activate && pip install -r requirements.txt (or pip install -e . for packages).
• Run with extra diagnostics: PYTHONWARNINGS=error python -X dev -X tracemalloc=5 <cmd> to surface warnings and allocation sources.
• If syntax/import failures persist, run python scripts/compile_all.py <paths> for a fast preflight; consult references/error-playbook.md for fixes.

Triage Workflow

1. Gather facts: error text, command, inputs, recent code/config changes, OS/arch, presence of compiled extensions, and whether it worked on Python <3.12.
2. Reproduce cleanly: new venv, pip install -r requirements.txt, pin tooling (pip~=23, setuptools, build, uv/pip-tools as used). Avoid system site-packages unless required (python -m site --user-site should usually be empty).
3. Classify:
   • Compile/Syntax: SyntaxError, IndentationError, NameError at import, f-string parse issues.
   • Import resolution: ModuleNotFoundError/ImportError, wrong module picked, circular import, namespace package confusion.
   • Runtime exception: stack trace during execution/tests.
   • Binary/ABI: segfaults, undefined symbol, .so load failure, mismatched wheel.
4. Apply targeted section below, then re-run under the same command with and without -X dev to confirm. Keep a minimal repro once fixed.

Compile-Time / Syntax Issues

• Run python scripts/compile_all.py src tests (or python -m compileall <path>) to surface syntax errors before runtime; the script reports failing files.
• For single files, python -X dev -m py_compile path/to/file.py gives richer context.
• Common causes: mixed tabs/spaces, stray non-ASCII quotes, unmatched brackets, f-strings missing braces, using match without trailing colon. Normalize line endings if code came from Windows.
• When refactoring annotations, remember that postponed evaluation (from __future__ import annotations) is default in 3.12; remove quotes when not needed.
• If codegen creates bad syntax, dump the generated string to verify before exec/eval.

Import and Module Resolution

• Ensure install vs import names match: pip show <pkg> and python - <<'PY'\nimport pkg_resources; print(pkg_resources.get_distribution('<pkg>').location)\nPY.
• Avoid name collisions with local files: rename json.py, logging.py, test.py, etc., that shadow stdlib or package modules.
• Check packages are installed in the active env: python -m pip list, python -m pip check. For editable installs, re-run pip install -e . after moving files.
• Fix relative/absolute imports: prefer absolute package imports; ensure __init__.py exists for packages that are not namespace packages.
• Circular import hints: AttributeError/ImportError during module init. Break cycles by moving imports inside functions or consolidating shared constants.
• For sys.path surprises, log it early: import sys, pprint; pprint.pprint(sys.path). Clear PYTHONPATH when debugging.

Runtime Exceptions

• Read the stack trace bottom-up for the originating frame. Add minimal logging:

  import logging; logging.basicConfig(level=logging.DEBUG, format="%(levelname)s %(message)s")
  

• Re-run with python -X dev -X tracemalloc=5 <cmd> to surface resource warnings and allocation traces.
• For logic bugs, reproduce with smallest input; add assertions near suspected invariants.
• Drop into a debugger where it fails: python -m pdb -c continue <script> ... or breakpoint() in the code.
• For async code, enable loop debug: PYTHONASYNCIODEBUG=1 python -X dev <cmd>.
• If tests fail, isolate with pytest path -k <expr> -vv and use --maxfail=1 --lf for fast iteration.

Binary/ABI or Environment Issues

• Python 3.12 removes distutils; ensure packaging uses setuptools>=68 or build. Avoid legacy setup.py install.
• Rebuild native extensions for 3.12: python -m pip install --no-binary=:all: --force-reinstall <pkg> or pip install -e . for local C extensions.
• Check architecture and glibc/libc compatibility for wheels (arm64 vs x86_64).
• If segfaulting, enable faulthandler: PYTHONFAULTHANDLER=1 python <cmd> and consider python -X dev -m pytest -k failing_test --maxfail=1.
• Validate dependency pins: python -m pip check; if dependency resolution drifts, regenerate lock files with the active interpreter.

Resources

• references/error-playbook.md: Common error patterns with targeted fixes and quick commands.
• scripts/compile_all.py: Fast syntax/import preflight across paths. Run python scripts/compile_all.py <path ...>; exit code is non-zero if any file fails to compile.