Project System

Python Project System

Overview

Modern Python packaging has converged on pyproject.toml as the single source of truth for project metadata, build configuration, and tool settings. A series of PEPs have standardized how Python projects are built, distributed, and installed:

The key insight: the build backend is now pluggable. You choose a backend in [build-system], define metadata in [project], and any PEP 517-compatible frontend (pip, build, uv) can build your package.

pyproject.toml Anatomy

A complete, annotated pyproject.toml:

# ============================================================
# Build System (PEP 518)
# Tells pip/build which backend to use and what to install first
# ============================================================
[build-system]
requires = ["hatchling"]            # Build dependencies
build-backend = "hatchling.build"   # The entry point for the backend

# ============================================================
# Project Metadata (PEP 621)
# Standardized metadata that all backends understand
# ============================================================
[project]
name = "my-package"
version = "1.2.0"                   # Static version (or use dynamic below)
description = "A short summary of the package"
readme = "README.md"
license = "MIT"                     # SPDX expression (PEP 639)
requires-python = ">=3.11"
authors = [
    { name = "Jane Doe", email = "jane@example.com" },
]
maintainers = [
    { name = "Team Lead", email = "lead@example.com" },
]
keywords = ["automation", "tooling"]
classifiers = [
    "Development Status :: 4 - Beta",
    "Programming Language :: Python :: 3",
    "Programming Language :: Python :: 3.11",
    "Programming Language :: Python :: 3.12",
    "Programming Language :: Python :: 3.13",
    "Intended Audience :: Developers",
    "License :: OSI Approved :: MIT License",
    "Typing :: Typed",
]

# Core dependencies
dependencies = [
    "httpx>=0.27",
    "pydantic>=2.0,<3",
    "rich>=13.0",
]

# ============================================================
# Optional dependency groups
# Install with: pip install my-package[dev]
# ============================================================
[project.optional-dependencies]
dev = [
    "pytest>=8.0",
    "pytest-cov>=5.0",
    "mypy>=1.10",
    "ruff>=0.5",
]
docs = [
    "sphinx>=7.0",
    "sphinx-rtd-theme>=2.0",
]

# ============================================================
# Entry Points
# ============================================================
[project.scripts]
my-cli = "my_package.cli:main"         # Console script

[project.gui-scripts]
my-gui = "my_package.gui:launch"       # GUI script (no console window on Windows)

[project.entry-points."my_package.plugins"]
builtin = "my_package.plugins.builtin:BuiltinPlugin"  # Plugin entry point group

# ============================================================
# URLs shown on PyPI
# ============================================================
[project.urls]
Homepage = "https://github.com/example/my-package"
Documentation = "https://my-package.readthedocs.io"
Repository = "https://github.com/example/my-package"
Issues = "https://github.com/example/my-package/issues"
Changelog = "https://github.com/example/my-package/blob/main/CHANGELOG.md"

# ============================================================
# Backend-specific configuration
# (This section varies by backend -- example is for hatchling)
# ============================================================
[tool.hatch.build.targets.wheel]
packages = ["src/my_package"]

# ============================================================
# Tool configuration (pytest, mypy, ruff, coverage, etc.)
# ============================================================
[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "-ra -q --strict-markers"

[tool.mypy]
python_version = "3.11"
strict = true
warn_return_any = true
warn_unused_configs = true

[tool.ruff]
target-version = "py311"
line-length = 88

[tool.ruff.lint]
select = ["E", "W", "F", "I", "UP", "B", "SIM"]

[tool.coverage.run]
source = ["src/my_package"]
branch = true

[tool.coverage.report]
show_missing = true
fail_under = 80

Build Backends Comparison

Build System Configuration by Backend

setuptools (most common, most flexible):

[build-system]
requires = ["setuptools>=75.0", "wheel"]
build-backend = "setuptools.build_meta"

hatchling (recommended for new projects):

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

flit-core (minimal):

[build-system]
requires = ["flit_core>=3.9"]
build-backend = "flit_core.buildapi"

pdm-backend:

[build-system]
requires = ["pdm-backend"]
build-backend = "pdm.backend"

maturin (Rust extensions):

[build-system]
requires = ["maturin>=1.7"]
build-backend = "maturin"

poetry-core:

[build-system]
requires = ["poetry-core>=1.9"]
build-backend = "poetry.core.masonry.api"

Source Layout vs Flat Layout

Source Layout (Recommended)

my-package/
  pyproject.toml
  README.md
  LICENSE
  src/
    my_package/
      __init__.py
      core.py
      utils.py
  tests/
    __init__.py
    test_core.py
    test_utils.py

Advantages:

• Prevents accidental imports of the development version (forces installation)
• Clear separation between source code and project metadata
• Avoids name collisions between the package directory and test/script imports
• Required by some backends (flit) by default

setuptools configuration for src layout:

[tool.setuptools.packages.find]
where = ["src"]

hatchling configuration for src layout:

[tool.hatch.build.targets.wheel]
packages = ["src/my_package"]

Flat Layout

my-package/
  pyproject.toml
  README.md
  LICENSE
  my_package/
    __init__.py
    core.py
    utils.py
  tests/
    test_core.py

Advantages:

• Simpler directory structure
• No src/ prefix in import paths during development
• Works out of the box with most backends

setuptools configuration for flat layout:

[tool.setuptools.packages.find]
include = ["my_package*"]

Recommendation: Use the src layout for libraries and packages published to PyPI. Use the flat layout for applications and scripts that will not be distributed as packages.

Entry Points

Console Scripts

Console scripts create executable commands that are installed into the user's PATH:

[project.scripts]
my-cli = "my_package.cli:main"
my-tool = "my_package.tools:run"

The format is command-name = "module.path:function". The function is called with no arguments.

# src/my_package/cli.py
import sys

def main() -> int:
    """Entry point for the my-cli command."""
    print("Hello from my-cli!")
    return 0

if __name__ == "__main__":
    sys.exit(main())

GUI Scripts

Same as console scripts but without opening a console window on Windows:

[project.gui-scripts]
my-gui = "my_package.gui:launch"

Plugin Entry Point Groups

Used for plugin systems where third-party packages can register extensions:

# In the plugin package's pyproject.toml
[project.entry-points."myapp.plugins"]
csv-export = "myapp_csv:CsvExporter"
json-export = "myapp_json:JsonExporter"

Discovering plugins at runtime:

from importlib.metadata import entry_points

def load_plugins():
    eps = entry_points(group="myapp.plugins")
    plugins = {}
    for ep in eps:
        plugins[ep.name] = ep.load()
    return plugins

Version Management Strategies

Static Versioning

Define the version directly in pyproject.toml:

[project]
version = "1.2.0"

Update manually before each release. Simple but error-prone.

Dynamic Versioning from a Python File

[project]
dynamic = ["version"]

[tool.setuptools.dynamic]
version = {attr = "my_package.__version__"}

# src/my_package/__init__.py
__version__ = "1.2.0"

SCM-Based Versioning with setuptools-scm

Derive versions automatically from git tags:

[build-system]
requires = ["setuptools>=75.0", "setuptools-scm>=8"]
build-backend = "setuptools.build_meta"

[project]
dynamic = ["version"]

[tool.setuptools_scm]
# Version derived from latest git tag
# Tag "v1.2.0" -> version "1.2.0"
# 3 commits after tag -> version "1.2.1.dev3+g1234abc"

Workflow:

git tag v1.0.0
git push --tags
# Version is now 1.0.0

# After more commits without a tag:
# Version becomes 1.0.1.dev3+g1234abc

Hatch Version Management

[project]
dynamic = ["version"]

[tool.hatch.version]
path = "src/my_package/__about__.py"

# src/my_package/__about__.py
__version__ = "1.2.0"

Bump with the CLI:

hatch version minor   # 1.2.0 -> 1.3.0
hatch version patch   # 1.3.0 -> 1.3.1
hatch version major   # 1.3.1 -> 2.0.0

Package Discovery Configuration

setuptools Auto-Discovery

# Find all packages under src/
[tool.setuptools.packages.find]
where = ["src"]

# Exclude test packages
[tool.setuptools.packages.find]
where = ["src"]
exclude = ["tests*"]

Including Data Files

# setuptools
[tool.setuptools.package-data]
my_package = ["data/*.json", "templates/*.html"]

# hatchling
[tool.hatch.build.targets.wheel]
packages = ["src/my_package"]

[tool.hatch.build.targets.wheel.force-include]
"config/defaults.json" = "my_package/defaults.json"

Including or Excluding Files from SDist

# hatchling
[tool.hatch.build.targets.sdist]
include = ["src/", "tests/", "README.md", "LICENSE"]
exclude = ["*.pyc", "__pycache__"]

# setuptools -- use MANIFEST.in
# include src/my_package/data/*.json
# exclude tests/*

Building and Publishing Workflow

Building Distributions

Use the standard build tool (PEP 517 frontend):

# Install the build tool
pip install build

# Build both sdist and wheel
python -m build

# Output goes to dist/
# dist/my_package-1.2.0.tar.gz      (sdist)
# dist/my_package-1.2.0-py3-none-any.whl  (wheel)

Publishing to PyPI

Using twine (traditional):

pip install twine

# Upload to Test PyPI first
twine upload --repository testpypi dist/*

# Upload to production PyPI
twine upload dist/*

Using Trusted Publishers (recommended for CI/CD):

Trusted publishing eliminates API tokens by using OIDC identity from your CI provider. Configure on PyPI:

1. Go to PyPI project settings and add a "trusted publisher"
2. Configure your CI provider (GitHub Actions, GitLab CI, etc.)
3. No secrets or tokens needed

GitHub Actions example:

name: Publish to PyPI

on:
  release:
    types: [published]

permissions:
  id-token: write  # Required for trusted publishing

jobs:
  publish:
    runs-on: ubuntu-latest
    environment: pypi
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install build
      - run: python -m build
      - uses: pypa/gh-action-pypi-publish@release/v1
        # No token needed -- uses OIDC

Using uv publish (modern alternative):

uv publish
# or to Test PyPI
uv publish --index-url https://test.pypi.org/legacy/

Editable Installs (PEP 660)

Editable installs let you develop a package without reinstalling after every change:

# Standard editable install
pip install -e .

# With optional dependencies
pip install -e ".[dev,docs]"

# Using uv
uv pip install -e .

How it works: The backend creates a special .pth file or import hook that redirects imports to your source directory. Changes to source files take effect immediately.

Backend support:

• setuptools: Full support (uses import hooks or .pth files)
• hatchling: Full support
• flit: Full support
• pdm-backend: Full support
• maturin: Supported (rebuilds Rust code on import with maturin develop)

Monorepo Patterns

Workspace with Multiple Packages

monorepo/
  pyproject.toml          # Root project (optional, for workspace tools)
  packages/
    core/
      pyproject.toml
      src/core/
        __init__.py
    api/
      pyproject.toml
      src/api/
        __init__.py
    cli/
      pyproject.toml
      src/cli/
        __init__.py

Inter-Package Dependencies

Reference sibling packages by path during development:

# packages/api/pyproject.toml
[project]
dependencies = [
    "core",  # Published name for installation from PyPI
]

# For local development, install with:
# pip install -e packages/core
# pip install -e packages/api

uv Workspaces

uv supports native workspace management:

# Root pyproject.toml
[tool.uv.workspace]
members = ["packages/*"]

# Install all workspace members in editable mode
uv sync

Hatch Workspaces

# Root pyproject.toml
[tool.hatch.envs.default]
dependencies = [
    "core @ {root:uri}/packages/core",
    "api @ {root:uri}/packages/api",
]

Best Practices

1. Always use pyproject.toml. Do not create new setup.py or setup.cfg files. These are legacy.

2. Choose src layout for libraries. It prevents import confusion and is the community standard for published packages.

3. Pin your build backend version in [build-system].requires to avoid surprises:

   requires = ["hatchling>=1.25,<2"]
   

4. Use requires-python to declare the minimum Python version:

   requires-python = ">=3.11"
   

5. Specify dependency bounds. Use minimum versions with >= and optional upper bounds:

   dependencies = [
       "httpx>=0.27",
       "pydantic>=2.0,<3",
   ]
   

6. Use optional dependency groups for dev, test, and docs dependencies to keep the core package lean.

7. Include a py.typed marker for typed packages:

   src/my_package/py.typed   # Empty file -- signals PEP 561 compliance
   

8. Use SCM-based versioning (setuptools-scm) for libraries to avoid manual version bumps.

9. Test your packaging before publishing:

   python -m build
   twine check dist/*
   pip install dist/*.whl
   

10. Use trusted publishers for PyPI uploads from CI/CD instead of long-lived API tokens.