Database Patterns
Comprehensive patterns for database migrations, schema design, and version management. Each category has individual rule files in rules/ loaded on-demand.
Quick Reference
| Database Selection | 1 | HIGH | Choosing the right database, PostgreSQL vs MongoDB, cost analysis |
Total: 10 rules across 5 categories
This skill is a wrap around Alembic and PostgreSQL, not a replacement for their
docs. Read references/ork-delta.md first: it holds the
version floors, corrections and house conventions that upstream does not carry.
Everything in the table below was removed on purpose.
Upstream coverage (do not restate)
These topics are vendor documentation. Fetch them from the source instead of
re-teaching them here.
| Topic |
First-party source |
Alembic autogenerate, async env.py template, revision/upgrade/downgrade/history CLI |
https://alembic.sqlalchemy.org/en/latest/autogenerate.html (our one correction to the async template is in references/ork-delta.md) |
Migration branches, merge revisions, tuple down_revision, branch labels |
https://alembic.sqlalchemy.org/en/latest/branches.html |
Multi-database env.py, batched backfill recipes, migration hooks, environment-conditional migrations |
https://alembic.sqlalchemy.org/en/latest/cookbook.html |
| Rollback and data-integrity test harnesses |
references/migration-testing.md |
| JSONB operators, indexing and storage tradeoffs |
https://www.postgresql.org/docs/current/datatype-json.html (normal forms and the house denormalization call stay in rules/schema-normalization.md) |
Full index-type reference and syntax (B-tree, GIN, partial, covering, CREATE INDEX CONCURRENTLY, REINDEX) |
https://www.postgresql.org/docs/current/sql-createindex.html (the house subset we actually apply stays in rules/schema-indexing.md) |
lock_timeout, statement_timeout, advisory locks during migration |
https://www.postgresql.org/docs/current/runtime-config-client.html and rules/versioning-drift.md |
| Enum type changes |
https://www.postgresql.org/docs/current/datatype-enum.html |
| Table partitioning |
https://www.postgresql.org/docs/current/ddl-partitioning.html |
| Trigger functions |
https://www.postgresql.org/docs/current/plpgsql-trigger.html |
| Foreign-key cascade semantics |
https://www.postgresql.org/docs/current/ddl-constraints.html |
| Temporal and audit-trail tables, CDC change logs, stored-procedure and view versioning |
https://www.postgresql.org/docs/18/sql-createtable.html (read references/ork-delta.md before assuming these give row history) |
HNSW and vector index tuning (m, ef_construction, hnsw.ef_search) |
https://github.com/pgvector/pgvector |
| Generic pre-deployment, backup and schema-review checklists |
https://alembic.sqlalchemy.org/en/latest/tutorial.html |
| Async SQLAlchemy sessions, FastAPI wiring, connection pool tuning |
ork:python-backend skill |
Quick Start
# Alembic: Auto-generate migration from model changes
# alembic revision --autogenerate -m "add user preferences"
def upgrade() -> None:
op.add_column('users', sa.Column('org_id', UUID(as_uuid=True), nullable=True))
op.execute("UPDATE users SET org_id = 'default-org-uuid' WHERE org_id IS NULL")
def downgrade() -> None:
op.drop_column('users', 'org_id')
-- Schema: Normalization to 3NF with proper indexing
-- PG18: prefer uuidv7() (time-ordered, better B-tree locality) over gen_random_uuid() (random v4)
CREATE TABLE orders (
id UUID PRIMARY KEY DEFAULT uuidv7(),
customer_id UUID NOT NULL REFERENCES customers(id),
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX idx_orders_customer_id ON orders(customer_id);
Alembic Migrations
Migration management with Alembic for SQLAlchemy 2.0 async applications.
| Rule |
File |
Key Pattern |
| Data Migration |
rules/alembic-data-migration.md |
Batch backfill, two-phase NOT NULL, zero-downtime |
| Branching |
rules/alembic-branching.md |
Feature branches, merge migrations, conflict resolution |
Autogenerate setup is upstream. Our one deviation from Alembic's async env.py
template (the in_greenlet() guard) is in references/ork-delta.md.
Schema Design
SQL and NoSQL schema design with normalization, indexing, and constraint patterns.
| Rule |
File |
Key Pattern |
| Normalization |
rules/schema-normalization.md |
1NF-3NF, when to denormalize, JSON vs normalized |
| Indexing |
rules/schema-indexing.md |
B-tree, GIN, HNSW, partial/covering indexes |
| NoSQL Patterns |
rules/schema-nosql.md |
Embed vs reference, document design, sharding |
Versioning
Database version control and change management across environments.
| Rule |
File |
Key Pattern |
| Changelog |
rules/versioning-changelog.md |
Schema version table, semantic versioning, audit trails |
| Drift Detection |
rules/versioning-drift.md |
Environment sync, checksum verification, migration locks |
Rollback testing lives in references/migration-testing.md;
the docstring convention for lossy downgrades is in
references/ork-delta.md.
Database Selection
Decision frameworks for choosing the right database. Default: PostgreSQL.
| Rule |
File |
Key Pattern |
| Selection Guide |
rules/db-selection.md |
PostgreSQL-first, tier-based matrix, anti-patterns |
Key Decisions
| Decision |
Recommendation |
Rationale |
| Async dialect |
postgresql+asyncpg |
Native async support for SQLAlchemy 2.0 |
| NOT NULL column |
Two-phase: nullable first, then alter |
Avoids locking, backward compatible |
| Large table index |
CREATE INDEX CONCURRENTLY |
Zero-downtime, no table locks |
| Normalization target |
3NF for OLTP |
Reduces redundancy while maintaining query performance |
| Primary key strategy |
UUID for distributed, INT for single-DB |
Context-appropriate key generation |
| Soft deletes |
deleted_at timestamp column |
Preserves audit trail, enables recovery |
| Migration granularity |
One logical change per file |
Easier rollback and debugging |
| Production deployment |
Generate SQL, review, then apply |
Never auto-run in production |
Anti-Patterns (FORBIDDEN)
# NEVER: Add NOT NULL without default or two-phase approach
op.add_column('users', sa.Column('org_id', UUID, nullable=False)) # LOCKS TABLE!
# NEVER: Use blocking index creation on large tables
op.create_index('idx_large', 'big_table', ['col']) # Use CONCURRENTLY
# NEVER: Skip downgrade implementation
def downgrade():
pass # WRONG - implement proper rollback
# NEVER: Modify migration after deployment - create new migration instead
# NEVER: Run migrations automatically in production
# Use: alembic upgrade head --sql > review.sql
# NEVER: Run CONCURRENTLY inside transaction
op.execute("BEGIN; CREATE INDEX CONCURRENTLY ...; COMMIT;") # FAILS
# NEVER: Delete migration history
command.stamp(alembic_config, "head") # Loses history
# NEVER: Skip environments (Always: local -> CI -> staging -> production)
Detailed Documentation
| Resource |
Description |
references/ork-delta.md |
Our corrections and house conventions. Read this first |
references/migration-testing.md |
Upgrade/downgrade cycle and data-integrity test harnesses |
references/postgres-vs-mongodb.md |
Head-to-head comparison behind the PostgreSQL-first default |
references/db-migration-paths.md |
Cross-engine migration risk matrix |
references/cost-comparison.md |
Managed database cost analysis |
references/storage-and-cms.md |
Object storage and CMS selection |
scripts/ |
Migration template, model change detector |
Zero-Downtime Migration
Safe database schema changes without downtime using expand-contract pattern and online schema changes.
| Rule |
File |
Key Pattern |
| Expand-Contract |
rules/migration-zero-downtime.md |
Expand phase, backfill, contract phase, pgroll automation |
| Rollback & Monitoring |
rules/migration-rollback.md |
pgroll rollback, lock monitoring, replication lag, backfill progress |
Related Skills
sqlalchemy-2-async - Async SQLAlchemy session patterns
ork:testing-integration - Integration testing patterns including migration testing
caching - Cache layer design to complement database performance
ork:performance - Performance optimization patterns
1---2name: database-patterns3license: MIT4compatibility: "Claude Code 2.1.277+."5description: "Schema design, Alembic migrations and database versioning. Use when creating migrations, designing or normalizing schemas, or fixing schema drift."6user-invocable: false7disable-model-invocation: false8targets:9 - library: sqlalchemy10 version: ">=2.1.1"11metadata:12 source: https://skillmd.com/skills/yonatangross/database-patterns13 owner-agent: "database-engineer"14 category: document-asset-creation15 version: "2.0.0"16 author: "OrchestKit"17 complexity: "medium"18 tags: "database, migrations, alembic, schema-design, versioning, postgresql, sql, nosql"19allowed-tools: "Read Glob Grep WebFetch WebSearch"20paths: ["**/migrations/**", "**/models/**", "alembic.ini", "**/schema*"]21path_patterns: ["*.sql", "**/migrations/**", "**/alembic/**", "**/prisma/**"]22---2324<!-- directive-density: intentional (teaches migration anti-patterns; NEVER markers describe real production-break conditions, not aspirational guidance) -->2526# Database Patterns2728Comprehensive patterns for database migrations, schema design, and version management. Each category has individual rule files in `rules/` loaded on-demand.2930## Quick Reference3132| Category | Rules | Impact | When to Use |33|----------|-------|--------|-------------|34| [Alembic Migrations](#alembic-migrations) | 2 | CRITICAL | Data migrations, branch management |35| [Schema Design](#schema-design) | 3 | HIGH | Normalization, indexing strategies, NoSQL patterns |36| [Versioning](#versioning) | 2 | HIGH | Changelogs, schema drift detection |37| [Zero-Downtime Migration](#zero-downtime-migration) | 2 | CRITICAL | Expand-contract, pgroll, rollback monitoring |3839| [Database Selection](#database-selection) | 1 | HIGH | Choosing the right database, PostgreSQL vs MongoDB, cost analysis |4041**Total: 10 rules across 5 categories**4243This skill is a wrap around Alembic and PostgreSQL, not a replacement for their44docs. Read `references/ork-delta.md` first: it holds the45version floors, corrections and house conventions that upstream does not carry.46Everything in the table below was removed on purpose.4748## Upstream coverage (do not restate)4950These topics are vendor documentation. Fetch them from the source instead of51re-teaching them here.5253| Topic | First-party source |54|-------|--------------------|55| Alembic autogenerate, async `env.py` template, `revision`/`upgrade`/`downgrade`/`history` CLI | https://alembic.sqlalchemy.org/en/latest/autogenerate.html (our one correction to the async template is in `references/ork-delta.md`) |56| Migration branches, merge revisions, tuple `down_revision`, branch labels | https://alembic.sqlalchemy.org/en/latest/branches.html |57| Multi-database `env.py`, batched backfill recipes, migration hooks, environment-conditional migrations | https://alembic.sqlalchemy.org/en/latest/cookbook.html |58| Rollback and data-integrity test harnesses | `references/migration-testing.md` |59| JSONB operators, indexing and storage tradeoffs | https://www.postgresql.org/docs/current/datatype-json.html (normal forms and the house denormalization call stay in `rules/schema-normalization.md`) |60| Full index-type reference and syntax (B-tree, GIN, partial, covering, `CREATE INDEX CONCURRENTLY`, `REINDEX`) | https://www.postgresql.org/docs/current/sql-createindex.html (the house subset we actually apply stays in `rules/schema-indexing.md`) |61| `lock_timeout`, `statement_timeout`, advisory locks during migration | https://www.postgresql.org/docs/current/runtime-config-client.html and `rules/versioning-drift.md` |62| Enum type changes | https://www.postgresql.org/docs/current/datatype-enum.html |63| Table partitioning | https://www.postgresql.org/docs/current/ddl-partitioning.html |64| Trigger functions | https://www.postgresql.org/docs/current/plpgsql-trigger.html |65| Foreign-key cascade semantics | https://www.postgresql.org/docs/current/ddl-constraints.html |66| Temporal and audit-trail tables, CDC change logs, stored-procedure and view versioning | https://www.postgresql.org/docs/18/sql-createtable.html (read `references/ork-delta.md` before assuming these give row history) |67| HNSW and vector index tuning (`m`, `ef_construction`, `hnsw.ef_search`) | https://github.com/pgvector/pgvector |68| Generic pre-deployment, backup and schema-review checklists | https://alembic.sqlalchemy.org/en/latest/tutorial.html |69| Async SQLAlchemy sessions, FastAPI wiring, connection pool tuning | `ork:python-backend` skill |7071## Quick Start7273```python74# Alembic: Auto-generate migration from model changes75# alembic revision --autogenerate -m "add user preferences"7677def upgrade() -> None:78 op.add_column('users', sa.Column('org_id', UUID(as_uuid=True), nullable=True))79 op.execute("UPDATE users SET org_id = 'default-org-uuid' WHERE org_id IS NULL")8081def downgrade() -> None:82 op.drop_column('users', 'org_id')83```8485```sql86-- Schema: Normalization to 3NF with proper indexing87-- PG18: prefer uuidv7() (time-ordered, better B-tree locality) over gen_random_uuid() (random v4)88CREATE TABLE orders (89 id UUID PRIMARY KEY DEFAULT uuidv7(),90 customer_id UUID NOT NULL REFERENCES customers(id),91 created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()92);93CREATE INDEX idx_orders_customer_id ON orders(customer_id);94```9596## Alembic Migrations9798Migration management with Alembic for SQLAlchemy 2.0 async applications.99100| Rule | File | Key Pattern |101|------|------|-------------|102| Data Migration | `rules/alembic-data-migration.md` | Batch backfill, two-phase NOT NULL, zero-downtime |103| Branching | `rules/alembic-branching.md` | Feature branches, merge migrations, conflict resolution |104105Autogenerate setup is upstream. Our one deviation from Alembic's async `env.py`106template (the `in_greenlet()` guard) is in `references/ork-delta.md`.107108## Schema Design109110SQL and NoSQL schema design with normalization, indexing, and constraint patterns.111112| Rule | File | Key Pattern |113|------|------|-------------|114| Normalization | `rules/schema-normalization.md` | 1NF-3NF, when to denormalize, JSON vs normalized |115| Indexing | `rules/schema-indexing.md` | B-tree, GIN, HNSW, partial/covering indexes |116| NoSQL Patterns | `rules/schema-nosql.md` | Embed vs reference, document design, sharding |117118## Versioning119120Database version control and change management across environments.121122| Rule | File | Key Pattern |123|------|------|-------------|124| Changelog | `rules/versioning-changelog.md` | Schema version table, semantic versioning, audit trails |125| Drift Detection | `rules/versioning-drift.md` | Environment sync, checksum verification, migration locks |126127Rollback testing lives in `references/migration-testing.md`;128the docstring convention for lossy downgrades is in129`references/ork-delta.md`.130131## Database Selection132133Decision frameworks for choosing the right database. Default: PostgreSQL.134135| Rule | File | Key Pattern |136|------|------|-------------|137| Selection Guide | `rules/db-selection.md` | PostgreSQL-first, tier-based matrix, anti-patterns |138139## Key Decisions140141| Decision | Recommendation | Rationale |142|----------|----------------|-----------|143| Async dialect | `postgresql+asyncpg` | Native async support for SQLAlchemy 2.0 |144| NOT NULL column | Two-phase: nullable first, then alter | Avoids locking, backward compatible |145| Large table index | `CREATE INDEX CONCURRENTLY` | Zero-downtime, no table locks |146| Normalization target | 3NF for OLTP | Reduces redundancy while maintaining query performance |147| Primary key strategy | UUID for distributed, INT for single-DB | Context-appropriate key generation |148| Soft deletes | `deleted_at` timestamp column | Preserves audit trail, enables recovery |149| Migration granularity | One logical change per file | Easier rollback and debugging |150| Production deployment | Generate SQL, review, then apply | Never auto-run in production |151152## Anti-Patterns (FORBIDDEN)153154```python155# NEVER: Add NOT NULL without default or two-phase approach156op.add_column('users', sa.Column('org_id', UUID, nullable=False)) # LOCKS TABLE!157158# NEVER: Use blocking index creation on large tables159op.create_index('idx_large', 'big_table', ['col']) # Use CONCURRENTLY160161# NEVER: Skip downgrade implementation162def downgrade():163 pass # WRONG - implement proper rollback164165# NEVER: Modify migration after deployment - create new migration instead166167# NEVER: Run migrations automatically in production168# Use: alembic upgrade head --sql > review.sql169170# NEVER: Run CONCURRENTLY inside transaction171op.execute("BEGIN; CREATE INDEX CONCURRENTLY ...; COMMIT;") # FAILS172173# NEVER: Delete migration history174command.stamp(alembic_config, "head") # Loses history175176# NEVER: Skip environments (Always: local -> CI -> staging -> production)177```178179## Detailed Documentation180181| Resource | Description |182|----------|-------------|183| `references/ork-delta.md` | Our corrections and house conventions. Read this first |184| `references/migration-testing.md` | Upgrade/downgrade cycle and data-integrity test harnesses |185| `references/postgres-vs-mongodb.md` | Head-to-head comparison behind the PostgreSQL-first default |186| `references/db-migration-paths.md` | Cross-engine migration risk matrix |187| `references/cost-comparison.md` | Managed database cost analysis |188| `references/storage-and-cms.md` | Object storage and CMS selection |189| `scripts/` | Migration template, model change detector |190191## Zero-Downtime Migration192193Safe database schema changes without downtime using expand-contract pattern and online schema changes.194195| Rule | File | Key Pattern |196|------|------|-------------|197| Expand-Contract | `rules/migration-zero-downtime.md` | Expand phase, backfill, contract phase, pgroll automation |198| Rollback & Monitoring | `rules/migration-rollback.md` | pgroll rollback, lock monitoring, replication lag, backfill progress |199200## Related Skills201202- `sqlalchemy-2-async` - Async SQLAlchemy session patterns203- `ork:testing-integration` - Integration testing patterns including migration testing204- `caching` - Cache layer design to complement database performance205- `ork:performance` - Performance optimization patterns