AiiDA Core Architecture
Source layout
The source code lives under src/aiida/ with these main packages:
| Package |
Purpose |
brokers/ |
Message broker interface (RabbitMQ via kiwipy) |
calculations/ |
Built-in calculations |
cmdline/ |
CLI (verdi command) built with click |
common/ |
Shared utilities, exceptions, warnings, constants |
engine/ |
Workflow engine: process runner, daemon, persistence, transport tasks (with plumpy dependency) |
manage/ |
Configuration management, manager singleton |
orm/ |
Object-relational mapping: nodes, groups, users, computers, querybuilder |
parsers/ |
Built-in parser plugins |
plugins/ |
Plugin entry point system and factories |
repository/ |
File repository abstraction layer |
restapi/ |
Flask-based REST API (soon to be replaced by aiida-restapi) |
schedulers/ |
Built-in HPC scheduler plugins (SLURM, PBS, SGE, LSF, etc.) |
storage/ |
Storage backends (primarily psql_dos (sqlite_dos) for PostgreSQL (SQLite) + disk-objectstore) |
tools/ |
Utility tools (graph visualization, archive operations, data dumping, etc.) |
transports/ |
Built-in Transport plugins (SSH, local) |
workflows/ |
Built-in workflows |
Key entry points
| Area |
Key file(s) |
Purpose |
| Engine core |
src/aiida/engine/processes/process.py |
Base Process class |
| CalcJob |
src/aiida/engine/processes/calcjobs/calcjob.py |
CalcJob implementation |
| CalcJob file ops |
src/aiida/engine/daemon/execmanager.py |
File copying, job submission, retrieval |
| WorkChain |
src/aiida/engine/processes/workchains/workchain.py |
WorkChain implementation |
| ORM node |
src/aiida/orm/nodes/node.py |
Base Node class |
| QueryBuilder |
src/aiida/orm/querybuilder.py |
Query interface for the provenance graph |
| Process runner |
src/aiida/engine/runners.py |
Runner executes and submits processes |
| Plugin factories |
src/aiida/plugins/factories.py |
DataFactory, CalculationFactory, etc. |
| Storage ABC |
src/aiida/orm/implementation/storage_backend.py |
StorageBackend abstract base class |
| Transport ABC |
src/aiida/transports/transport.py |
Transport, BlockingTransport, AsyncTransport |
| Scheduler ABC |
src/aiida/schedulers/scheduler.py |
Scheduler base class |
Other notable files: ProcessBuilder (engine/processes/builder.py), Computer (orm/computers.py), Config (manage/configuration/config.py), Manager (manage/manager.py), DaemonClient (engine/daemon/client.py), Profile (manage/configuration/profile.py), psql_dos backend (storage/psql_dos/backend.py), RabbitmqBroker (brokers/rabbitmq/broker.py), Repository (repository/repository.py).
Database and file storage
- ORM: SQLAlchemy. File storage: disk-objectstore. Migrations: Alembic (under
src/aiida/storage/psql_dos/migrations/).
- Main backend:
psql_dos (PostgreSQL + disk-objectstore). Lightweight: sqlite_dos (SQLite + disk-objectstore).
Abstract base classes (ABCs)
AiiDA defines ABCs for extensible components.
To create a plugin, implement the corresponding ABC and register it as an entry point.
| ABC |
Location |
Purpose |
Entry point |
Transport |
aiida.transports.transport |
File transfer and remote command execution |
aiida.transports |
Scheduler |
aiida.schedulers.scheduler |
HPC job scheduler interface |
aiida.schedulers |
Parser |
aiida.parsers.parser |
Parse calculation outputs |
aiida.parsers |
StorageBackend |
aiida.orm.implementation.storage_backend |
Database and file storage |
aiida.storage |
AbstractCode |
aiida.orm.nodes.data.code.abstract |
Code/executable representation |
aiida.data |
CalcJobImporter |
aiida.engine.processes.calcjobs.importer |
Import existing calculation results |
aiida.calculations.importers |
Quick API overview via stubs
To get a compact view of a module's public API without reading the full source (which can pollute context), generate type stubs:
uv run stubgen -p aiida.orm -o /tmp/stubs # public API only
uv run stubgen -p aiida.orm -o /tmp/stubs --include-private # include _private members
The generated .pyi files show only signatures, classes, and type annotations, useful for understanding an API surface quickly.
stubgen ships with mypy, which is part of the pre-commit optional dependencies (uv sync --extra pre-commit or just uv sync if already installed).
Project configuration
pyproject.toml (dependencies, entry points, ruff/mypy config), uv.lock, .pre-commit-config.yaml, .readthedocs.yml, .github/workflows/, .docker/.
1---2name: architecture-overview3description: Use when exploring the aiida-core codebase structure, looking for key files, or understanding how packages relate to each other.4---56# AiiDA Core Architecture78## Source layout910The source code lives under `src/aiida/` with these main packages:1112| Package | Purpose |13|---------|---------|14| `brokers/` | Message broker interface (RabbitMQ via [`kiwipy`](https://github.com/aiidateam/kiwipy)) |15| `calculations/` | Built-in calculations |16| `cmdline/` | CLI (`verdi` command) built with `click` |17| `common/` | Shared utilities, exceptions, warnings, constants |18| `engine/` | Workflow engine: process runner, daemon, persistence, transport tasks (with [`plumpy`](https://github.com/aiidateam/plumpy) dependency) |19| `manage/` | Configuration management, manager singleton |20| `orm/` | Object-relational mapping: nodes, groups, users, computers, querybuilder |21| `parsers/` | Built-in parser plugins |22| `plugins/` | Plugin entry point system and factories |23| `repository/` | File repository abstraction layer |24| `restapi/` | Flask-based REST API (soon to be replaced by `aiida-restapi`) |25| `schedulers/` | Built-in HPC scheduler plugins (SLURM, PBS, SGE, LSF, etc.) |26| `storage/` | Storage backends (primarily `psql_dos` (`sqlite_dos`) for PostgreSQL (SQLite) + disk-objectstore) |27| `tools/` | Utility tools (graph visualization, archive operations, data dumping, etc.) |28| `transports/` | Built-in Transport plugins (SSH, local) |29| `workflows/` | Built-in workflows |3031## Key entry points3233| Area | Key file(s) | Purpose |34|------|------------|---------|35| Engine core | `src/aiida/engine/processes/process.py` | Base `Process` class |36| CalcJob | `src/aiida/engine/processes/calcjobs/calcjob.py` | `CalcJob` implementation |37| CalcJob file ops | `src/aiida/engine/daemon/execmanager.py` | File copying, job submission, retrieval |38| WorkChain | `src/aiida/engine/processes/workchains/workchain.py` | `WorkChain` implementation |39| ORM node | `src/aiida/orm/nodes/node.py` | Base `Node` class |40| QueryBuilder | `src/aiida/orm/querybuilder.py` | Query interface for the provenance graph |41| Process runner | `src/aiida/engine/runners.py` | `Runner` executes and submits processes |42| Plugin factories | `src/aiida/plugins/factories.py` | `DataFactory`, `CalculationFactory`, etc. |43| Storage ABC | `src/aiida/orm/implementation/storage_backend.py` | `StorageBackend` abstract base class |44| Transport ABC | `src/aiida/transports/transport.py` | `Transport`, `BlockingTransport`, `AsyncTransport` |45| Scheduler ABC | `src/aiida/schedulers/scheduler.py` | `Scheduler` base class |4647Other notable files: `ProcessBuilder` (`engine/processes/builder.py`), `Computer` (`orm/computers.py`), `Config` (`manage/configuration/config.py`), `Manager` (`manage/manager.py`), `DaemonClient` (`engine/daemon/client.py`), `Profile` (`manage/configuration/profile.py`), `psql_dos` backend (`storage/psql_dos/backend.py`), `RabbitmqBroker` (`brokers/rabbitmq/broker.py`), `Repository` (`repository/repository.py`).4849## Database and file storage5051- ORM: SQLAlchemy. File storage: disk-objectstore. Migrations: Alembic (under `src/aiida/storage/psql_dos/migrations/`).52- Main backend: `psql_dos` (PostgreSQL + disk-objectstore). Lightweight: `sqlite_dos` (SQLite + disk-objectstore).5354## Abstract base classes (ABCs)5556AiiDA defines ABCs for extensible components.57To create a plugin, implement the corresponding ABC and register it as an entry point.5859| ABC | Location | Purpose | Entry point |60|-----|----------|---------|-------------|61| `Transport` | `aiida.transports.transport` | File transfer and remote command execution | `aiida.transports` |62| `Scheduler` | `aiida.schedulers.scheduler` | HPC job scheduler interface | `aiida.schedulers` |63| `Parser` | `aiida.parsers.parser` | Parse calculation outputs | `aiida.parsers` |64| `StorageBackend` | `aiida.orm.implementation.storage_backend` | Database and file storage | `aiida.storage` |65| `AbstractCode` | `aiida.orm.nodes.data.code.abstract` | Code/executable representation | `aiida.data` |66| `CalcJobImporter` | `aiida.engine.processes.calcjobs.importer` | Import existing calculation results | `aiida.calculations.importers` |6768## Quick API overview via stubs6970To get a compact view of a module's public API without reading the full source (which can pollute context), generate type stubs:7172```bash73uv run stubgen -p aiida.orm -o /tmp/stubs # public API only74uv run stubgen -p aiida.orm -o /tmp/stubs --include-private # include _private members75```7677The generated `.pyi` files show only signatures, classes, and type annotations, useful for understanding an API surface quickly.78`stubgen` ships with `mypy`, which is part of the `pre-commit` optional dependencies (`uv sync --extra pre-commit` or just `uv sync` if already installed).7980## Project configuration8182`pyproject.toml` (dependencies, entry points, ruff/mypy config), `uv.lock`, `.pre-commit-config.yaml`, `.readthedocs.yml`, `.github/workflows/`, `.docker/`.
aiidateam