Database Patterns
Comprehensive patterns for database migrations, schema design, and version management. Each category has individual rule files in rules/ loaded on-demand.
Quick Reference
| Database Selection | 1 | HIGH | Choosing the right database, PostgreSQL vs MongoDB, cost analysis |
Total: 10 rules across 5 categories
This skill is a wrap around Alembic and PostgreSQL, not a replacement for their
docs. Read references/ork-delta.md first: it holds the
version floors, corrections and house conventions that upstream does not carry.
Everything in the table below was removed on purpose.
Upstream coverage (do not restate)
These topics are vendor documentation. Fetch them from the source instead of
re-teaching them here.
| Topic |
First-party source |
Alembic autogenerate, async env.py template, revision/upgrade/downgrade/history CLI |
https://alembic.sqlalchemy.org/en/latest/autogenerate.html (our one correction to the async template is in references/ork-delta.md) |
Migration branches, merge revisions, tuple down_revision, branch labels |
https://alembic.sqlalchemy.org/en/latest/branches.html |
Multi-database env.py, batched backfill recipes, migration hooks, environment-conditional migrations |
https://alembic.sqlalchemy.org/en/latest/cookbook.html |
| Rollback and data-integrity test harnesses |
references/migration-testing.md |
| JSONB operators, indexing and storage tradeoffs |
https://www.postgresql.org/docs/current/datatype-json.html (normal forms and the house denormalization call stay in rules/schema-normalization.md) |
Full index-type reference and syntax (B-tree, GIN, partial, covering, CREATE INDEX CONCURRENTLY, REINDEX) |
https://www.postgresql.org/docs/current/sql-createindex.html (the house subset we actually apply stays in rules/schema-indexing.md) |
lock_timeout, statement_timeout, advisory locks during migration |
https://www.postgresql.org/docs/current/runtime-config-client.html and rules/versioning-drift.md |
| Enum type changes |
https://www.postgresql.org/docs/current/datatype-enum.html |
| Table partitioning |
https://www.postgresql.org/docs/current/ddl-partitioning.html |
| Trigger functions |
https://www.postgresql.org/docs/current/plpgsql-trigger.html |
| Foreign-key cascade semantics |
https://www.postgresql.org/docs/current/ddl-constraints.html |
| Temporal and audit-trail tables, CDC change logs, stored-procedure and view versioning |
https://www.postgresql.org/docs/18/sql-createtable.html (read references/ork-delta.md before assuming these give row history) |
HNSW and vector index tuning (m, ef_construction, hnsw.ef_search) |
https://github.com/pgvector/pgvector |
| Generic pre-deployment, backup and schema-review checklists |
https://alembic.sqlalchemy.org/en/latest/tutorial.html |
| Async SQLAlchemy sessions, FastAPI wiring, connection pool tuning |
ork:python-backend skill |
Quick Start
# Alembic: Auto-generate migration from model changes
# alembic revision --autogenerate -m "add user preferences"
def upgrade() -> None:
op.add_column('users', sa.Column('org_id', UUID(as_uuid=True), nullable=True))
op.execute("UPDATE users SET org_id = 'default-org-uuid' WHERE org_id IS NULL")
def downgrade() -> None:
op.drop_column('users', 'org_id')
-- Schema: Normalization to 3NF with proper indexing
-- PG18: prefer uuidv7() (time-ordered, better B-tree locality) over gen_random_uuid() (random v4)
CREATE TABLE orders (
id UUID PRIMARY KEY DEFAULT uuidv7(),
customer_id UUID NOT NULL REFERENCES customers(id),
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX idx_orders_customer_id ON orders(customer_id);
Alembic Migrations
Migration management with Alembic for SQLAlchemy 2.0 async applications.
| Rule |
File |
Key Pattern |
| Data Migration |
rules/alembic-data-migration.md |
Batch backfill, two-phase NOT NULL, zero-downtime |
| Branching |
rules/alembic-branching.md |
Feature branches, merge migrations, conflict resolution |
Autogenerate setup is upstream. Our one deviation from Alembic's async env.py
template (the in_greenlet() guard) is in references/ork-delta.md.
Schema Design
SQL and NoSQL schema design with normalization, indexing, and constraint patterns.
| Rule |
File |
Key Pattern |
| Normalization |
rules/schema-normalization.md |
1NF-3NF, when to denormalize, JSON vs normalized |
| Indexing |
rules/schema-indexing.md |
B-tree, GIN, HNSW, partial/covering indexes |
| NoSQL Patterns |
rules/schema-nosql.md |
Embed vs reference, document design, sharding |
Versioning
Database version control and change management across environments.
| Rule |
File |
Key Pattern |
| Changelog |
rules/versioning-changelog.md |
Schema version table, semantic versioning, audit trails |
| Drift Detection |
rules/versioning-drift.md |
Environment sync, checksum verification, migration locks |
Rollback testing lives in references/migration-testing.md;
the docstring convention for lossy downgrades is in
references/ork-delta.md.
Database Selection
Decision frameworks for choosing the right database. Default: PostgreSQL.
| Rule |
File |
Key Pattern |
| Selection Guide |
rules/db-selection.md |
PostgreSQL-first, tier-based matrix, anti-patterns |
Key Decisions
| Decision |
Recommendation |
Rationale |
| Async dialect |
postgresql+asyncpg |
Native async support for SQLAlchemy 2.0 |
| NOT NULL column |
Two-phase: nullable first, then alter |
Avoids locking, backward compatible |
| Large table index |
CREATE INDEX CONCURRENTLY |
Zero-downtime, no table locks |
| Normalization target |
3NF for OLTP |
Reduces redundancy while maintaining query performance |
| Primary key strategy |
UUID for distributed, INT for single-DB |
Context-appropriate key generation |
| Soft deletes |
deleted_at timestamp column |
Preserves audit trail, enables recovery |
| Migration granularity |
One logical change per file |
Easier rollback and debugging |
| Production deployment |
Generate SQL, review, then apply |
Never auto-run in production |
Anti-Patterns (FORBIDDEN)
# NEVER: Add NOT NULL without default or two-phase approach
op.add_column('users', sa.Column('org_id', UUID, nullable=False)) # LOCKS TABLE!
# NEVER: Use blocking index creation on large tables
op.create_index('idx_large', 'big_table', ['col']) # Use CONCURRENTLY
# NEVER: Skip downgrade implementation
def downgrade():
pass # WRONG - implement proper rollback
# NEVER: Modify migration after deployment - create new migration instead
# NEVER: Run migrations automatically in production
# Use: alembic upgrade head --sql > review.sql
# NEVER: Run CONCURRENTLY inside transaction
op.execute("BEGIN; CREATE INDEX CONCURRENTLY ...; COMMIT;") # FAILS
# NEVER: Delete migration history
command.stamp(alembic_config, "head") # Loses history
# NEVER: Skip environments (Always: local -> CI -> staging -> production)
Detailed Documentation
| Resource |
Description |
references/ork-delta.md |
Our corrections and house conventions. Read this first |
references/migration-testing.md |
Upgrade/downgrade cycle and data-integrity test harnesses |
references/postgres-vs-mongodb.md |
Head-to-head comparison behind the PostgreSQL-first default |
references/db-migration-paths.md |
Cross-engine migration risk matrix |
references/cost-comparison.md |
Managed database cost analysis |
references/storage-and-cms.md |
Object storage and CMS selection |
scripts/ |
Migration template, model change detector |
Zero-Downtime Migration
Safe database schema changes without downtime using expand-contract pattern and online schema changes.
| Rule |
File |
Key Pattern |
| Expand-Contract |
rules/migration-zero-downtime.md |
Expand phase, backfill, contract phase, pgroll automation |
| Rollback & Monitoring |
rules/migration-rollback.md |
pgroll rollback, lock monitoring, replication lag, backfill progress |
Related Skills
sqlalchemy-2-async - Async SQLAlchemy session patterns
ork:testing-integration - Integration testing patterns including migration testing
caching - Cache layer design to complement database performance
ork:performance - Performance optimization patterns
1---2name: database-patterns3description: Database design and migration patterns for Alembic migrations, schema design (SQL/NoSQL), and database versioning. Use when creating migrations, designing schemas, normalizing data, managing database versions, or handling schema drift.4license: MIT5---6
7<!-- directive-density: intentional (teaches migration anti-patterns; NEVER markers describe real production-break conditions, not aspirational guidance) -->
8
9# Database Patterns
10
11Comprehensive patterns for database migrations, schema design, and version management. Each category has individual rule files in `rules/` loaded on-demand.
12
13## Quick Reference
14
15| Category | Rules | Impact | When to Use |
16|----------|-------|--------|-------------|
17| [Alembic Migrations](#alembic-migrations) | 2 | CRITICAL | Data migrations, branch management |
18| [Schema Design](#schema-design) | 3 | HIGH | Normalization, indexing strategies, NoSQL patterns |
19| [Versioning](#versioning) | 2 | HIGH | Changelogs, schema drift detection |
20| [Zero-Downtime Migration](#zero-downtime-migration) | 2 | CRITICAL | Expand-contract, pgroll, rollback monitoring |
21
22| [Database Selection](#database-selection) | 1 | HIGH | Choosing the right database, PostgreSQL vs MongoDB, cost analysis |
23
24**Total: 10 rules across 5 categories**
25
26This skill is a wrap around Alembic and PostgreSQL, not a replacement for their
27docs. Read `references/ork-delta.md` first: it holds the
28version floors, corrections and house conventions that upstream does not carry.
29Everything in the table below was removed on purpose.
30
31## Upstream coverage (do not restate)
32
33These topics are vendor documentation. Fetch them from the source instead of
34re-teaching them here.
35
36| Topic | First-party source |
37|-------|--------------------|
38| Alembic autogenerate, async `env.py` template, `revision`/`upgrade`/`downgrade`/`history` CLI | https://alembic.sqlalchemy.org/en/latest/autogenerate.html (our one correction to the async template is in `references/ork-delta.md`) |
39| Migration branches, merge revisions, tuple `down_revision`, branch labels | https://alembic.sqlalchemy.org/en/latest/branches.html |
40| Multi-database `env.py`, batched backfill recipes, migration hooks, environment-conditional migrations | https://alembic.sqlalchemy.org/en/latest/cookbook.html |
41| Rollback and data-integrity test harnesses | `references/migration-testing.md` |
42| JSONB operators, indexing and storage tradeoffs | https://www.postgresql.org/docs/current/datatype-json.html (normal forms and the house denormalization call stay in `rules/schema-normalization.md`) |
43| Full index-type reference and syntax (B-tree, GIN, partial, covering, `CREATE INDEX CONCURRENTLY`, `REINDEX`) | https://www.postgresql.org/docs/current/sql-createindex.html (the house subset we actually apply stays in `rules/schema-indexing.md`) |
44| `lock_timeout`, `statement_timeout`, advisory locks during migration | https://www.postgresql.org/docs/current/runtime-config-client.html and `rules/versioning-drift.md` |
45| Enum type changes | https://www.postgresql.org/docs/current/datatype-enum.html |
46| Table partitioning | https://www.postgresql.org/docs/current/ddl-partitioning.html |
47| Trigger functions | https://www.postgresql.org/docs/current/plpgsql-trigger.html |
48| Foreign-key cascade semantics | https://www.postgresql.org/docs/current/ddl-constraints.html |
49| Temporal and audit-trail tables, CDC change logs, stored-procedure and view versioning | https://www.postgresql.org/docs/18/sql-createtable.html (read `references/ork-delta.md` before assuming these give row history) |
50| HNSW and vector index tuning (`m`, `ef_construction`, `hnsw.ef_search`) | https://github.com/pgvector/pgvector |
51| Generic pre-deployment, backup and schema-review checklists | https://alembic.sqlalchemy.org/en/latest/tutorial.html |
52| Async SQLAlchemy sessions, FastAPI wiring, connection pool tuning | `ork:python-backend` skill |
53
54## Quick Start
55
56```python
57# Alembic: Auto-generate migration from model changes
58# alembic revision --autogenerate -m "add user preferences"
59
60def upgrade() -> None:
61 op.add_column('users', sa.Column('org_id', UUID(as_uuid=True), nullable=True))
62 op.execute("UPDATE users SET org_id = 'default-org-uuid' WHERE org_id IS NULL")
63
64def downgrade() -> None:
65 op.drop_column('users', 'org_id')
66```
67
68```sql
69-- Schema: Normalization to 3NF with proper indexing
70-- PG18: prefer uuidv7() (time-ordered, better B-tree locality) over gen_random_uuid() (random v4)
71CREATE TABLE orders (
72 id UUID PRIMARY KEY DEFAULT uuidv7(),
73 customer_id UUID NOT NULL REFERENCES customers(id),
74 created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
75);
76CREATE INDEX idx_orders_customer_id ON orders(customer_id);
77```
78
79## Alembic Migrations
80
81Migration management with Alembic for SQLAlchemy 2.0 async applications.
82
83| Rule | File | Key Pattern |
84|------|------|-------------|
85| Data Migration | `rules/alembic-data-migration.md` | Batch backfill, two-phase NOT NULL, zero-downtime |
86| Branching | `rules/alembic-branching.md` | Feature branches, merge migrations, conflict resolution |
87
88Autogenerate setup is upstream. Our one deviation from Alembic's async `env.py`
89template (the `in_greenlet()` guard) is in `references/ork-delta.md`.
90
91## Schema Design
92
93SQL and NoSQL schema design with normalization, indexing, and constraint patterns.
94
95| Rule | File | Key Pattern |
96|------|------|-------------|
97| Normalization | `rules/schema-normalization.md` | 1NF-3NF, when to denormalize, JSON vs normalized |
98| Indexing | `rules/schema-indexing.md` | B-tree, GIN, HNSW, partial/covering indexes |
99| NoSQL Patterns | `rules/schema-nosql.md` | Embed vs reference, document design, sharding |
100
101## Versioning
102
103Database version control and change management across environments.
104
105| Rule | File | Key Pattern |
106|------|------|-------------|
107| Changelog | `rules/versioning-changelog.md` | Schema version table, semantic versioning, audit trails |
108| Drift Detection | `rules/versioning-drift.md` | Environment sync, checksum verification, migration locks |
109
110Rollback testing lives in `references/migration-testing.md`;
111the docstring convention for lossy downgrades is in
112`references/ork-delta.md`.
113
114## Database Selection
115
116Decision frameworks for choosing the right database. Default: PostgreSQL.
117
118| Rule | File | Key Pattern |
119|------|------|-------------|
120| Selection Guide | `rules/db-selection.md` | PostgreSQL-first, tier-based matrix, anti-patterns |
121
122## Key Decisions
123
124| Decision | Recommendation | Rationale |
125|----------|----------------|-----------|
126| Async dialect | `postgresql+asyncpg` | Native async support for SQLAlchemy 2.0 |
127| NOT NULL column | Two-phase: nullable first, then alter | Avoids locking, backward compatible |
128| Large table index | `CREATE INDEX CONCURRENTLY` | Zero-downtime, no table locks |
129| Normalization target | 3NF for OLTP | Reduces redundancy while maintaining query performance |
130| Primary key strategy | UUID for distributed, INT for single-DB | Context-appropriate key generation |
131| Soft deletes | `deleted_at` timestamp column | Preserves audit trail, enables recovery |
132| Migration granularity | One logical change per file | Easier rollback and debugging |
133| Production deployment | Generate SQL, review, then apply | Never auto-run in production |
134
135## Anti-Patterns (FORBIDDEN)
136
137```python
138# NEVER: Add NOT NULL without default or two-phase approach
139op.add_column('users', sa.Column('org_id', UUID, nullable=False)) # LOCKS TABLE!
140
141# NEVER: Use blocking index creation on large tables
142op.create_index('idx_large', 'big_table', ['col']) # Use CONCURRENTLY
143
144# NEVER: Skip downgrade implementation
145def downgrade():
146 pass # WRONG - implement proper rollback
147
148# NEVER: Modify migration after deployment - create new migration instead
149
150# NEVER: Run migrations automatically in production
151# Use: alembic upgrade head --sql > review.sql
152
153# NEVER: Run CONCURRENTLY inside transaction
154op.execute("BEGIN; CREATE INDEX CONCURRENTLY ...; COMMIT;") # FAILS
155
156# NEVER: Delete migration history
157command.stamp(alembic_config, "head") # Loses history
158
159# NEVER: Skip environments (Always: local -> CI -> staging -> production)
160```
161
162## Detailed Documentation
163
164| Resource | Description |
165|----------|-------------|
166| `references/ork-delta.md` | Our corrections and house conventions. Read this first |
167| `references/migration-testing.md` | Upgrade/downgrade cycle and data-integrity test harnesses |
168| `references/postgres-vs-mongodb.md` | Head-to-head comparison behind the PostgreSQL-first default |
169| `references/db-migration-paths.md` | Cross-engine migration risk matrix |
170| `references/cost-comparison.md` | Managed database cost analysis |
171| `references/storage-and-cms.md` | Object storage and CMS selection |
172| `scripts/` | Migration template, model change detector |
173
174## Zero-Downtime Migration
175
176Safe database schema changes without downtime using expand-contract pattern and online schema changes.
177
178| Rule | File | Key Pattern |
179|------|------|-------------|
180| Expand-Contract | `rules/migration-zero-downtime.md` | Expand phase, backfill, contract phase, pgroll automation |
181| Rollback & Monitoring | `rules/migration-rollback.md` | pgroll rollback, lock monitoring, replication lag, backfill progress |
182
183## Related Skills
184
185- `sqlalchemy-2-async` - Async SQLAlchemy session patterns
186- `ork:testing-integration` - Integration testing patterns including migration testing
187- `caching` - Cache layer design to complement database performance
188- `ork:performance` - Performance optimization patterns