PostgreSQL Operations
Comprehensive PostgreSQL skill covering schema design through production operations.
Quick Connection
# Standard connection
psql "postgresql://user:pass@localhost:5432/dbname"
# With SSL
psql "postgresql://user:pass@host:5432/dbname?sslmode=require"
# Environment variables (libpq)
export PGHOST=localhost PGPORT=5432 PGDATABASE=mydb PGUSER=myuser PGPASSWORD=secret
psql
# Connection pooling (pgBouncer default)
psql "postgresql://user:pass@localhost:6432/dbname"
-- Check current connection
SELECT current_database(), current_user, inet_server_addr(), inet_server_port();
-- Active connections
SELECT count(*) FROM pg_stat_activity WHERE state = 'active';
Index Type Selection
What query pattern are you optimizing?
│
├─ Equality (WHERE col = val)
│ └─ B-tree (default, almost always right)
│
├─ Range (WHERE col > val, ORDER BY, BETWEEN)
│ └─ B-tree
│
├─ Array/JSONB containment (@>, ?, ?|, ?&)
│ └─ GIN
│
├─ Full-text search (@@)
│ └─ GIN with tsvector
│
├─ Geometric/range overlap (&&, <->)
│ └─ GiST
│
├─ Pattern matching (LIKE '%text%', similarity)
│ └─ GIN with pg_trgm (gin_trgm_ops)
│
├─ Large table, few distinct values, append-only
│ └─ BRIN (tiny index, good for timestamps)
│
└─ Exact equality only, no range/sort needed
└─ Hash (rare - B-tree usually better)
Quick Index Reference
| Index |
Best For |
Size |
Write Cost |
| B-tree |
Equality, range, sort |
Medium |
Low |
| GIN |
Arrays, JSONB, FTS, trigrams |
Large |
High |
| GiST |
Geometry, ranges, FTS |
Medium |
Medium |
| BRIN |
Correlated data (timestamps) |
Tiny |
Very low |
| Hash |
Exact equality only |
Medium |
Low |
Deep dive: Load ./references/indexing.md for composite, partial, expression, and covering index strategies.
EXPLAIN ANALYZE Workflow
-- Step 1: Run with ANALYZE and BUFFERS
EXPLAIN (ANALYZE, BUFFERS, FORMAT TEXT) SELECT ...;
-- Step 2: Read bottom-up. Find the slowest node.
-- Step 3: Check estimates vs actuals
-- actual rows=10000, rows=100 -> bad estimate, run ANALYZE
-- Step 4: Look for these red flags:
| Red Flag |
Meaning |
Fix |
Seq Scan on large table |
No usable index |
Add index matching WHERE/JOIN |
actual rows >> estimated rows |
Stale statistics |
ANALYZE tablename |
Nested Loop with high rows |
O(n*m) join |
Check join conditions, add index |
Sort with external merge |
work_mem too small |
Increase work_mem for session |
Buffers: shared read >> hit |
Cold cache or table too large |
Check shared_buffers, add covering index |
Hash Batch > 1 |
Hash join spilling to disk |
Increase work_mem |
Deep dive: Load ./references/query-tuning.md for plan node reference and optimization patterns.
Workload Profiles
| Setting |
OLTP |
OLAP |
Notes |
shared_buffers |
25% RAM |
25% RAM |
Same baseline |
work_mem |
4-16 MB |
256 MB-1 GB |
OLAP needs big sorts |
effective_cache_size |
75% RAM |
75% RAM |
Planner hint |
random_page_cost |
1.1 (SSD) |
1.1 (SSD) |
Lower for SSD |
max_parallel_workers_per_gather |
2 |
4-8 |
OLAP benefits more |
checkpoint_completion_target |
0.9 |
0.9 |
Spread checkpoint I/O |
wal_buffers |
64 MB |
64 MB |
-1 for auto |
maintenance_work_mem |
512 MB |
1-2 GB |
For VACUUM, CREATE INDEX |
Deep dive: Load ./references/config-tuning.md for full postgresql.conf walkthrough and extension setup.
Common Operations
Backup & Restore
# Logical backup (single database)
pg_dump -Fc dbname > backup.dump
# Restore
pg_restore -d dbname backup.dump
# Parallel backup (faster for large DBs)
pg_dump -Fc -j4 dbname > backup.dump
# Base backup for PITR
pg_basebackup -D /backup/base -Ft -Xs -P
Vacuum & Maintenance
-- Manual vacuum (reclaim space, update stats)
VACUUM (VERBOSE, ANALYZE) tablename;
-- Full vacuum (rewrites table, exclusive lock)
VACUUM FULL tablename; -- CAUTION: locks table
-- Reindex without downtime
REINDEX INDEX CONCURRENTLY idx_name;
-- Update statistics only
ANALYZE tablename;
Monitor Key Metrics
-- Slow queries (requires pg_stat_statements)
SELECT query, calls, mean_exec_time, total_exec_time
FROM pg_stat_statements ORDER BY mean_exec_time DESC LIMIT 10;
-- Table bloat indicator
SELECT schemaname, relname, n_dead_tup, n_live_tup,
round(n_dead_tup::numeric / NULLIF(n_live_tup, 0) * 100, 1) AS dead_pct
FROM pg_stat_user_tables WHERE n_dead_tup > 1000
ORDER BY n_dead_tup DESC;
-- Lock contention
SELECT pid, relation::regclass, mode, granted, query
FROM pg_locks JOIN pg_stat_activity USING (pid)
WHERE NOT granted;
-- Cache hit ratio (should be > 99%)
SELECT sum(heap_blks_hit) / NULLIF(sum(heap_blks_hit) + sum(heap_blks_read), 0) AS ratio
FROM pg_statio_user_tables;
Deep dive: Load ./references/operations.md for WAL archiving, PITR, autovacuum tuning, connection pooling.
Data Types Quick Reference
| Type |
Use When |
Example |
JSONB |
Semi-structured data, flexible schema |
'{"tags": ["a","b"]}'::jsonb |
ARRAY |
Fixed-type lists |
ARRAY['a','b','c'] |
tsrange |
Time periods, scheduling |
'[2024-01-01, 2024-12-31)'::tsrange |
tsvector |
Full-text search |
to_tsvector('english', body) |
uuid |
Distributed IDs |
gen_random_uuid() |
inet/cidr |
IP addresses, networks |
'192.168.1.0/24'::cidr |
Deep dive: Load ./references/schema-design.md for normalization, constraints, RLS, generated columns, table inheritance.
Gotchas & Anti-Patterns
| Mistake |
Why It's Bad |
Fix |
SELECT * in production |
Wastes bandwidth, blocks covering index scans |
List columns explicitly |
Function on indexed column (WHERE UPPER(email) = ...) |
Prevents index use |
Expression index: CREATE INDEX ... ON (UPPER(email)) |
NOT IN (subquery) with NULLs |
Returns no rows if subquery has NULL |
Use NOT EXISTS |
Missing ANALYZE after bulk load |
Planner uses stale row estimates |
Run ANALYZE tablename |
VACUUM FULL in production |
Exclusive lock on entire table |
Regular VACUUM + pg_repack |
LIMIT without ORDER BY |
Non-deterministic results |
Always pair with ORDER BY |
| Offset pagination on large tables |
Scans and discards rows |
Keyset pagination: WHERE id > last_id |
| Too many indexes |
Slows writes, wastes space |
Audit with pg_stat_user_indexes |
| Single shared connection pool |
Contention across services |
Per-service pools via pgBouncer |
default_transaction_isolation = serializable |
Excessive serialization failures |
Keep read committed, use explicit SERIALIZABLE where needed |
Row-Level Security (RLS) Quick Start
-- Enable RLS on table
ALTER TABLE documents ENABLE ROW LEVEL SECURITY;
-- Policy: users see only their own rows
CREATE POLICY user_isolation ON documents
USING (owner_id = current_setting('app.current_user_id')::int);
-- Policy: admins see everything
CREATE POLICY admin_access ON documents
USING (current_setting('app.role') = 'admin');
-- Set context per request (from app layer)
SET app.current_user_id = '42';
SET app.role = 'user';
Full-Text Search Quick Start
-- Add search column
ALTER TABLE articles ADD COLUMN search_vector tsvector
GENERATED ALWAYS AS (to_tsvector('english', title || ' ' || body)) STORED;
-- Index it
CREATE INDEX idx_articles_fts ON articles USING gin(search_vector);
-- Search with ranking
SELECT title, ts_rank(search_vector, query) AS rank
FROM articles, to_tsquery('english', 'database & optimization') AS query
WHERE search_vector @@ query
ORDER BY rank DESC;
LISTEN/NOTIFY
-- Publisher
NOTIFY order_events, '{"order_id": 123, "status": "shipped"}';
-- Subscriber (in psql or app)
LISTEN order_events;
-- Check for notifications (app code)
-- Python: conn.poll(); conn.notifies
-- Node: client.on('notification', callback)
Reference Files
Load these for deep-dive topics. Each is self-contained.
| Reference |
When to Load |
./references/schema-design.md |
Designing tables, choosing types, constraints, RLS policies, JSONB modeling |
./references/indexing.md |
Choosing index types, composite/partial/expression indexes, index maintenance |
./references/query-tuning.md |
Reading EXPLAIN plans, pg_stat_statements, optimizing specific query patterns |
./references/operations.md |
Backup/restore, WAL/PITR, vacuum tuning, monitoring, connection pooling |
./references/replication.md |
Streaming/logical replication, failover, partitioning, FDW |
./references/config-tuning.md |
postgresql.conf settings, OLTP/OLAP profiles, extension setup |
See Also
sql-ops - Vendor-neutral SQL patterns (CTEs, window functions, JOINs)
sqlite-ops - SQLite-specific patterns and operations
python-database-ops - SQLAlchemy ORM and async database patterns
1---2name: postgres-ops3description: PostgreSQL operations, optimization, and administration. Use for: schema design, index selection, query tuning with EXPLAIN ANALYZE, postgresql.conf configuration, backup and restore (pg_dump, pg_basebackup, WAL, PITR), vacuum and autovacuum tuning, connection pooling (pgBouncer, pgPool), replication (streaming, logical), partitioning, monitoring (pg_stat_statements, pg_stat_activity), JSONB operations, full-text search (tsvector, tsquery), row-level security (RLS), extensions (PostGIS, pg_trgm, timescaledb), GiST/GIN/BRIN indexes, materialized views, foreign data wrappers, LISTEN/NOTIFY.4license: MIT5---67# PostgreSQL Operations89Comprehensive PostgreSQL skill covering schema design through production operations.1011## Quick Connection1213```bash14# Standard connection15psql "postgresql://user:pass@localhost:5432/dbname"1617# With SSL18psql "postgresql://user:pass@host:5432/dbname?sslmode=require"1920# Environment variables (libpq)21export PGHOST=localhost PGPORT=5432 PGDATABASE=mydb PGUSER=myuser PGPASSWORD=secret22psql2324# Connection pooling (pgBouncer default)25psql "postgresql://user:pass@localhost:6432/dbname"26```2728```sql29-- Check current connection30SELECT current_database(), current_user, inet_server_addr(), inet_server_port();3132-- Active connections33SELECT count(*) FROM pg_stat_activity WHERE state = 'active';34```3536## Index Type Selection3738```39What query pattern are you optimizing?40│41├─ Equality (WHERE col = val)42│ └─ B-tree (default, almost always right)43│44├─ Range (WHERE col > val, ORDER BY, BETWEEN)45│ └─ B-tree46│47├─ Array/JSONB containment (@>, ?, ?|, ?&)48│ └─ GIN49│50├─ Full-text search (@@)51│ └─ GIN with tsvector52│53├─ Geometric/range overlap (&&, <->)54│ └─ GiST55│56├─ Pattern matching (LIKE '%text%', similarity)57│ └─ GIN with pg_trgm (gin_trgm_ops)58│59├─ Large table, few distinct values, append-only60│ └─ BRIN (tiny index, good for timestamps)61│62└─ Exact equality only, no range/sort needed63 └─ Hash (rare - B-tree usually better)64```6566### Quick Index Reference6768| Index | Best For | Size | Write Cost |69|-------|----------|------|------------|70| B-tree | Equality, range, sort | Medium | Low |71| GIN | Arrays, JSONB, FTS, trigrams | Large | High |72| GiST | Geometry, ranges, FTS | Medium | Medium |73| BRIN | Correlated data (timestamps) | Tiny | Very low |74| Hash | Exact equality only | Medium | Low |7576**Deep dive**: Load `./references/indexing.md` for composite, partial, expression, and covering index strategies.7778## EXPLAIN ANALYZE Workflow7980```sql81-- Step 1: Run with ANALYZE and BUFFERS82EXPLAIN (ANALYZE, BUFFERS, FORMAT TEXT) SELECT ...;8384-- Step 2: Read bottom-up. Find the slowest node.85-- Step 3: Check estimates vs actuals86-- actual rows=10000, rows=100 -> bad estimate, run ANALYZE87-- Step 4: Look for these red flags:88```8990| Red Flag | Meaning | Fix |91|----------|---------|-----|92| `Seq Scan` on large table | No usable index | Add index matching WHERE/JOIN |93| `actual rows` >> `estimated rows` | Stale statistics | `ANALYZE tablename` |94| `Nested Loop` with high rows | O(n*m) join | Check join conditions, add index |95| `Sort` with `external merge` | work_mem too small | Increase `work_mem` for session |96| `Buffers: shared read` >> `hit` | Cold cache or table too large | Check `shared_buffers`, add covering index |97| `Hash Batch` > 1 | Hash join spilling to disk | Increase `work_mem` |9899**Deep dive**: Load `./references/query-tuning.md` for plan node reference and optimization patterns.100101## Workload Profiles102103| Setting | OLTP | OLAP | Notes |104|---------|------|------|-------|105| `shared_buffers` | 25% RAM | 25% RAM | Same baseline |106| `work_mem` | 4-16 MB | 256 MB-1 GB | OLAP needs big sorts |107| `effective_cache_size` | 75% RAM | 75% RAM | Planner hint |108| `random_page_cost` | 1.1 (SSD) | 1.1 (SSD) | Lower for SSD |109| `max_parallel_workers_per_gather` | 2 | 4-8 | OLAP benefits more |110| `checkpoint_completion_target` | 0.9 | 0.9 | Spread checkpoint I/O |111| `wal_buffers` | 64 MB | 64 MB | -1 for auto |112| `maintenance_work_mem` | 512 MB | 1-2 GB | For VACUUM, CREATE INDEX |113114**Deep dive**: Load `./references/config-tuning.md` for full postgresql.conf walkthrough and extension setup.115116## Common Operations117118### Backup & Restore119120```bash121# Logical backup (single database)122pg_dump -Fc dbname > backup.dump123124# Restore125pg_restore -d dbname backup.dump126127# Parallel backup (faster for large DBs)128pg_dump -Fc -j4 dbname > backup.dump129130# Base backup for PITR131pg_basebackup -D /backup/base -Ft -Xs -P132```133134### Vacuum & Maintenance135136```sql137-- Manual vacuum (reclaim space, update stats)138VACUUM (VERBOSE, ANALYZE) tablename;139140-- Full vacuum (rewrites table, exclusive lock)141VACUUM FULL tablename; -- CAUTION: locks table142143-- Reindex without downtime144REINDEX INDEX CONCURRENTLY idx_name;145146-- Update statistics only147ANALYZE tablename;148```149150### Monitor Key Metrics151152```sql153-- Slow queries (requires pg_stat_statements)154SELECT query, calls, mean_exec_time, total_exec_time155FROM pg_stat_statements ORDER BY mean_exec_time DESC LIMIT 10;156157-- Table bloat indicator158SELECT schemaname, relname, n_dead_tup, n_live_tup,159 round(n_dead_tup::numeric / NULLIF(n_live_tup, 0) * 100, 1) AS dead_pct160FROM pg_stat_user_tables WHERE n_dead_tup > 1000161ORDER BY n_dead_tup DESC;162163-- Lock contention164SELECT pid, relation::regclass, mode, granted, query165FROM pg_locks JOIN pg_stat_activity USING (pid)166WHERE NOT granted;167168-- Cache hit ratio (should be > 99%)169SELECT sum(heap_blks_hit) / NULLIF(sum(heap_blks_hit) + sum(heap_blks_read), 0) AS ratio170FROM pg_statio_user_tables;171```172173**Deep dive**: Load `./references/operations.md` for WAL archiving, PITR, autovacuum tuning, connection pooling.174175## Data Types Quick Reference176177| Type | Use When | Example |178|------|----------|---------|179| `JSONB` | Semi-structured data, flexible schema | `'{"tags": ["a","b"]}'::jsonb` |180| `ARRAY` | Fixed-type lists | `ARRAY['a','b','c']` |181| `tsrange` | Time periods, scheduling | `'[2024-01-01, 2024-12-31)'::tsrange` |182| `tsvector` | Full-text search | `to_tsvector('english', body)` |183| `uuid` | Distributed IDs | `gen_random_uuid()` |184| `inet`/`cidr` | IP addresses, networks | `'192.168.1.0/24'::cidr` |185186**Deep dive**: Load `./references/schema-design.md` for normalization, constraints, RLS, generated columns, table inheritance.187188## Gotchas & Anti-Patterns189190| Mistake | Why It's Bad | Fix |191|---------|-------------|-----|192| `SELECT *` in production | Wastes bandwidth, blocks covering index scans | List columns explicitly |193| Function on indexed column (`WHERE UPPER(email) = ...`) | Prevents index use | Expression index: `CREATE INDEX ... ON (UPPER(email))` |194| `NOT IN (subquery)` with NULLs | Returns no rows if subquery has NULL | Use `NOT EXISTS` |195| Missing `ANALYZE` after bulk load | Planner uses stale row estimates | Run `ANALYZE tablename` |196| `VACUUM FULL` in production | Exclusive lock on entire table | Regular `VACUUM` + `pg_repack` |197| `LIMIT` without `ORDER BY` | Non-deterministic results | Always pair with `ORDER BY` |198| Offset pagination on large tables | Scans and discards rows | Keyset pagination: `WHERE id > last_id` |199| Too many indexes | Slows writes, wastes space | Audit with `pg_stat_user_indexes` |200| Single shared connection pool | Contention across services | Per-service pools via pgBouncer |201| `default_transaction_isolation = serializable` | Excessive serialization failures | Keep `read committed`, use explicit `SERIALIZABLE` where needed |202203## Row-Level Security (RLS) Quick Start204205```sql206-- Enable RLS on table207ALTER TABLE documents ENABLE ROW LEVEL SECURITY;208209-- Policy: users see only their own rows210CREATE POLICY user_isolation ON documents211 USING (owner_id = current_setting('app.current_user_id')::int);212213-- Policy: admins see everything214CREATE POLICY admin_access ON documents215 USING (current_setting('app.role') = 'admin');216217-- Set context per request (from app layer)218SET app.current_user_id = '42';219SET app.role = 'user';220```221222## Full-Text Search Quick Start223224```sql225-- Add search column226ALTER TABLE articles ADD COLUMN search_vector tsvector227 GENERATED ALWAYS AS (to_tsvector('english', title || ' ' || body)) STORED;228229-- Index it230CREATE INDEX idx_articles_fts ON articles USING gin(search_vector);231232-- Search with ranking233SELECT title, ts_rank(search_vector, query) AS rank234FROM articles, to_tsquery('english', 'database & optimization') AS query235WHERE search_vector @@ query236ORDER BY rank DESC;237```238239## LISTEN/NOTIFY240241```sql242-- Publisher243NOTIFY order_events, '{"order_id": 123, "status": "shipped"}';244245-- Subscriber (in psql or app)246LISTEN order_events;247248-- Check for notifications (app code)249-- Python: conn.poll(); conn.notifies250-- Node: client.on('notification', callback)251```252253## Reference Files254255Load these for deep-dive topics. Each is self-contained.256257| Reference | When to Load |258|-----------|-------------|259| `./references/schema-design.md` | Designing tables, choosing types, constraints, RLS policies, JSONB modeling |260| `./references/indexing.md` | Choosing index types, composite/partial/expression indexes, index maintenance |261| `./references/query-tuning.md` | Reading EXPLAIN plans, pg_stat_statements, optimizing specific query patterns |262| `./references/operations.md` | Backup/restore, WAL/PITR, vacuum tuning, monitoring, connection pooling |263| `./references/replication.md` | Streaming/logical replication, failover, partitioning, FDW |264| `./references/config-tuning.md` | postgresql.conf settings, OLTP/OLAP profiles, extension setup |265266## See Also267268- `sql-ops` - Vendor-neutral SQL patterns (CTEs, window functions, JOINs)269- `sqlite-ops` - SQLite-specific patterns and operations270- `python-database-ops` - SQLAlchemy ORM and async database patterns