# System Design

> Architecture and system design skill based on donnemartin's System Design Primer. Teaches scalable architecture thinking: load balancers, caching, DB partitioning, microservices, CAP theorem, and trade-off analysis before writing backend code.

- Skill: `kok-o/system-design` (Agent Skill)
- Install (CLI): `npx skillmds@latest add kok-o/system-design`
- Raw SKILL.md: https://api.skillmd.com/api/skills/kok-o/system-design/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: kok-o (https://skillmd.com/u/kok-o)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/kok-o/system-design

---

# system-design

## Overview

Scalable system architecture blueprint based on the System Design Primer and DDIA. Enforces load balancing, multi-tier caching (Redis, CDN), database partitioning, CAP theorem tradeoffs, and rate limiting before code is written.

## When to Use

Activate during the PLAN phase of any backend service, API design, database schema creation, or scalability optimization.

## Rules & Patterns

Based on [donnemartin/system-design-primer](https://github.com/donnemartin/system-design-primer) — the most starred system design resource on GitHub.

## Core Principle

> **Everything is a trade-off.** Before writing a single line of backend code, reason through the system at scale. A flat monolith that works now fails at 10× load.

---

## Mandatory Pre-Design Checklist

Before architecting any backend system, answer these questions:

1. **Scale**: What is the expected QPS (queries per second)? Peak vs average?
2. **Data volume**: How much data? Growth rate? 1GB? 1TB? 1PB?
3. **Consistency vs Availability**: Can we tolerate eventual consistency? (CAP theorem)
4. **Read/Write ratio**: Is it read-heavy (cache it!) or write-heavy (shard it!)?
5. **Latency requirements**: Real-time (<100ms)? Near-real-time (<1s)? Batch?
6. **Global distribution**: Single region or multi-region?
7. **Deployment model**: Traditional servers, Serverless, or Edge functions?

---

## Core Architecture Patterns

### Load Balancing

```
Clients → Load Balancer → [App Server 1, App Server 2, App Server N]
```

- Use **Round Robin** for stateless services
- Use **Least Connections** for varying request times
- Use **IP Hash** for session affinity (or move sessions to Redis)
- Always add **health checks** — remove unhealthy nodes automatically

**Rule**: Any service expecting > 1000 RPS needs a load balancer. No exceptions.

### Caching Strategy

```
App → [Cache Layer: Redis/Memcached] → Database
```

Cache decision ladder (check in order):

1. Is it read > write? → Cache it
2. Is it expensive to compute? → Cache it  
3. Is it user-specific? → Cache with user key
4. Is it global? → Shared cache, shorter TTL

**Cache patterns**:

- **Cache-aside** (lazy loading): check cache → miss → load DB → write cache
- **Write-through**: write to DB AND cache simultaneously (consistency > performance)
- **Write-behind**: write to cache → async flush to DB (performance > consistency)

**Invalidation**: Use TTL + event-driven invalidation. Never stale-forever.

**Modern framework-native caching (Next.js App Router)**:  
Before spinning up a dedicated Redis instance for caching API responses, check if Next.js built-in mechanisms are sufficient:

- `revalidatePath('/dashboard')` — invalidate all cache for a route
- `revalidateTag('user-profile')` — fine-grained tagged cache invalidation
- `unstable_cache()` — server-side data caching with TTL

```typescript
// [GOOD] Use Next.js native caching first
import { revalidateTag } from 'next/cache'

const getUser = unstable_cache(
  async (id: string) => db.users.findById(id),
  ['user'],
  { tags: ['user-profile'], revalidate: 3600 }
)

// Invalidate on mutation:
await db.users.update(id, data)
revalidateTag('user-profile')

// [BAD] Don't add Redis for simple SSR caching when Next.js handles it
```

### Database Architecture

#### When to use SQL vs NoSQL

| Scenario | Use SQL | Use NoSQL |
| ---------- | --------- | ----------- |
| Complex joins, ACID transactions | [PASS] | [FAIL] |
| Flexible/evolving schema | [FAIL] | [PASS] |
| Horizontal scaling needed | Careful | [PASS] |
| Simple key-value lookup | Overkill | [PASS] |
| Full-text search | Use Elasticsearch | Use Elasticsearch |
| Time-series data | TimescaleDB | InfluxDB |

#### Scaling Databases

**Vertical scaling**: Bigger machine. Easy but has ceiling.  
**Read replicas**: Route SELECT to replicas, writes to primary.  
**Sharding (horizontal partitioning)**:

- Hash sharding: `user_id % N` — even distribution, hard to rebalance
- Range sharding: user_id 1-1M on shard 1 — easy range queries, hotspots risk
- Directory-based: lookup table maps key → shard — flexible, but lookup is overhead

**Denormalization**: For read-heavy systems, duplicate data to avoid joins.  
**Rule**: Don't shard until you've maxed out read replicas.

### Message Queues & Async Processing

```
Producer → [Queue: Redis/RabbitMQ/Kafka] → Consumer Workers
```

Use queues when:

- Operation takes > 200ms (email, PDF generation, ML inference)
- You need retry logic on failure
- You need to decouple services
- Traffic spikes need to be absorbed

**Kafka** = durability + replay + high throughput (events/analytics)  
**Redis Queue** = simplicity + low latency (jobs/tasks)  
**RabbitMQ** = complex routing + acknowledgements

### Microservices vs Monolith

**Start with a monolith** unless you have > 10 engineers or proven scale need.

When to split into microservices:

- Independent deployment cycles needed
- Different scaling requirements per service
- Team autonomy (Conway's Law)
- Clear service boundaries (DDD bounded contexts)

**Rule**: A microservice should be able to be rewritten in 2 weeks by 2 engineers.

Service communication:

- **Sync (REST/gRPC)**: when caller needs immediate response
- **Async (events/queue)**: when caller can tolerate delay, or decoupling is needed
- **BFF / Server Actions**: for web apps, prefer typed client-server contracts (see below)

---

## Modern Stack Patterns (Serverless, Edge, Next.js)

### Serverless & Edge Architecture

When deploying to serverless (Vercel Functions, AWS Lambda) or edge (Vercel Edge, Cloudflare Workers), the classical "App Server + Load Balancer" model changes:

**Cold Start Problem**:

- Serverless functions spin up from zero on first request — this can add 100–1000ms
- **Never** do heavy initialization at module level (DB connections, config loading, crypto keys)
- **Always** initialize lazily inside the handler, or use a connection pooling service

```typescript
// [BAD] Wrong: Module-level initialization (runs on cold start, hangs the function)
const db = new DatabaseClient({ ... })  // top of file

// [GOOD] Correct: Lazy initialization with caching
let db: DatabaseClient | null = null
function getDb() {
  if (!db) db = new DatabaseClient({ ... })
  return db
}
```

**DB Connection Pooling in Serverless**:

- Traditional in-process pools (pg-pool, knex) do NOT work in serverless — each invocation is ephemeral
- Use **Prisma Accelerate**, **PlanetScale**, **Neon** pooling, or **Supabase** — they handle pooling at the infrastructure level
- Rule: If deploying to Vercel/serverless, NEVER assume `max_connections` is managed by your app process

**Edge Functions limitations**:

- No Node.js APIs (no `fs`, no `crypto.randomBytes`, limited DNS)
- Latency must be < 50ms — no heavy DB queries
- Use edge for: auth token verification, A/B testing, geo-routing, lightweight transformations

### BFF Pattern & Server Actions (Type-Safe Client-Server)

When building web apps, prefer typed client-server communication over generic REST endpoints:

**Option 1: Server Actions (Next.js App Router)**  
For mutations that touch the database directly, skip the API route entirely:

```typescript
// [GOOD] Server Action: No API route needed, fully type-safe
"use server"
export async function updateUser(id: string, data: UpdateUserInput) {
  // Input validation (always!)
  const validated = UpdateUserSchema.parse(data)
  
  // Auth check (always before data access!)
  const session = await getSession()
  if (session.userId !== id && !session.isAdmin) {
    throw new Error("Forbidden")
  }
  
  return db.users.update(id, validated)
}

// [BAD] Over-engineering: Don't create /api/users/[id] + fetch wrapper for simple mutations
```

**Option 2: tRPC (Full-stack type safety)**  
For complex APIs with many routes, use tRPC to get end-to-end type safety from DB to UI without code generation.

**Option 3: REST (When appropriate)**  
When building a public API consumed by external clients or mobile apps — use REST with OpenAPI spec.

**Decision rule**:

- Internal web-to-DB mutation → **Server Action**
- Internal complex API → **tRPC**
- Public/mobile API → **REST + OpenAPI**

### Domain-Driven Design (DDD) — Business Logic Isolation

**Rule**: NEVER write business logic inside API route handlers, Server Actions, or controllers. Always extract to dedicated services/use-cases.

```
[FAIL] Wrong structure:
app/api/orders/route.ts  ← contains: validation + auth + business logic + DB query

[PASS] Correct structure:
app/api/orders/route.ts  ← only: parse request, call service, return response
src/services/order.service.ts  ← all business logic, testable without HTTP context
src/repositories/order.repo.ts  ← all DB queries
```

Example:

```typescript
// [BAD] Business logic in route (untestable, bloated)
export async function POST(req: Request) {
  const data = await req.json()
  if (data.quantity <= 0) return new Response("Invalid", { status: 400 })
  const inventory = await db.inventory.findById(data.productId)
  if (inventory.stock < data.quantity) return new Response("Out of stock", { status: 400 })
  const total = inventory.price * data.quantity
  // ... 40 more lines
}

// [GOOD] Thin route, fat service
export async function POST(req: Request) {
  const data = await req.json()
  const result = await orderService.createOrder(data)
  return Response.json(result)
}

// orderService.createOrder() — pure function, fully unit-testable without HTTP
```

---

## Scalability Design Patterns

### CDN (Content Delivery Network)

- Serve static assets (JS, CSS, images) from CDN edge nodes
- Cache API responses that don't change per-user
- Reduce origin server load by 80%+

### Rate Limiting

Always implement for public APIs:

```
- Token bucket: smooth bursts, allows brief spikes
- Leaky bucket: strict rate, no bursts
- Fixed window: simple, vulnerable to boundary spikes
- Sliding window: most accurate, slightly more complex
```

Store rate limit state in Redis (not in-process — it doesn't survive restarts).

### Circuit Breaker

Prevent cascade failures:

```
CLOSED (normal) → [failures > threshold] → OPEN (fail fast)
  ↑                                               ↓
  └────── [timeout] ← HALF-OPEN (test request) ──┘
```

### Database Connection Pooling

- **Traditional servers**: Use pg-pool, knex, Prisma connection pool
- **Serverless**: Use Prisma Accelerate, PlanetScale, Neon, or Supabase pooling — NOT in-process pools

---

## CAP Theorem in Practice

**You can only guarantee 2 of 3**: Consistency, Availability, Partition Tolerance

| System | Chooses | Example |
| -------- | --------- | --------- |
| Traditional SQL | CP | PostgreSQL |
| Distributed NoSQL | AP | DynamoDB, Cassandra |
| Cache | AP (tunable) | Redis with replication |

**For most apps**: Choose AP. Accept eventual consistency. Use optimistic locking for critical writes.

---

## Designing Data-Intensive Applications (DDIA) Patterns

Based on _Designing Data-Intensive Applications_ (Martin Kleppmann) and [ciembor/agent-rules-books](https://github.com/ciembor/agent-rules-books).

### 1. The Dual-Write Problem & Transactional Outbox

**The Anti-Pattern**: Updating the database and sending a message to a broker (Kafka, RabbitMQ, SQS) in two separate operations. If one fails, the system enters an inconsistent state.

**The Solution**: Write the business entity AND an event record to an `outbox` table in the SAME database transaction:

```sql
BEGIN TRANSACTION;
  UPDATE orders SET status = 'PAID' WHERE id = 'ord_123';
  INSERT INTO outbox_events (id, aggregate_type, aggregate_id, event_type, payload, created_at)
  VALUES ('evt_456', 'Order', 'ord_123', 'OrderPaid', '{"amount": 99.00}', NOW());
COMMIT;
```

A background process (polling worker or Debezium CDC) reads `outbox_events`, delivers them to the message broker, and marks them as published.

### 2. Idempotency Invariant for Mutations

All write operations exposed over HTTP or queues MUST support deduplication:

- Accept an `Idempotency-Key` header (UUID or client-generated hash).
- Store key with status in Redis or DB with a TTL (e.g., 24 hours).
- If the key is already `COMPLETED`, return the cached response immediately without re-executing.
- If `IN_PROGRESS`, return HTTP `409 Conflict` or queue retry.

### 3. Read-Your-Own-Writes Consistency

When using read replicas, replication lag (even 50ms) causes users to not see their own changes immediately after saving:

- **Rule**: Route user reads to the primary database for `N` seconds (e.g., 5s) following any mutation by that user.
- Route all other queries and background jobs to read replicas.

---

## Architecture Decision Template

When proposing any backend architecture, include:

```markdown
## System Design Decision

**Scale Target**: [X RPS, Y GB data, Z users]
**Deployment Model**: [Traditional servers | Serverless | Edge]
**CAP Choice**: [CP/AP] because [reason]
**Read/Write Ratio**: [X:Y]

### Components
- **API Layer**: [REST/tRPC/Server Actions] — [why this choice]
- **Cache**: [Next.js native | Redis] for [what] with [TTL/tags strategy]
- **Database**: [SQL/NoSQL] - [pooling solution for serverless if applicable]
- **Async**: [Queue tech] for [what operations]

### Business Logic Isolation
- Services: [list key service files]
- Repositories: [list key repo files]
- Routes/Actions: [thin handlers only]

### Trade-offs Accepted
- [Trade-off 1]: [Why acceptable]
- [Trade-off 2]: [Why acceptable]

### Scaling Path
1. Now (MVP): [simple setup]
2. At 10× load: [first scaling step]
3. At 100× load: [next scaling step]
```


## Code Examples

See `EXAMPLES.md` for detailed code examples.

## Validation Checklist

What to verify during the review phase before completing the task.

## Common Mistakes

Anti-patterns and things to explicitly avoid. See `TROUBLESHOOTING.md`.

## Integration Notes

How this skill interacts with other skills.


<!-- Source: system-design.md -->

# System Design — Patterns & Principles

## Architecture Patterns

### Monolith (Start Here)

- **When**: MVP, small team, < 100K users
- **Structure**: Modular monolith with clear boundaries
- **Rule**: You can always extract microservices later. You can't easily merge them back.

### Microservices

- **When**: Team > 10 engineers, independent deployment needed
- **Communication**: REST (sync), Message Queue (async)
- **Data**: Each service owns its database
- **Pitfalls**: Network complexity, distributed transactions, debugging difficulty

### Event-Driven

- **When**: Loose coupling between services, async processing
- **Patterns**: Event Sourcing, CQRS, Pub/Sub
- **Tools**: Kafka, RabbitMQ, AWS SQS, Redis Streams

## Scalability

### Horizontal vs Vertical

- **Vertical first** — upgrade your server before distributing
- **Horizontal when** — you hit single-machine limits or need redundancy

### Database Scaling

1. **Indexes** — most common fix for slow queries
2. **Read replicas** — for read-heavy workloads
3. **Connection pooling** — PgBouncer, connection limits
4. **Caching** — Redis for hot data
5. **Sharding** — last resort, adds massive complexity

### Caching Layers

```
Client Cache (browser) → CDN → API Cache (Redis) → Database
```

| Strategy | Description |
| --- | --- |
| Cache-Aside | App checks cache first, fetches from DB on miss |
| Write-Through | App writes to cache AND DB simultaneously |
| Write-Behind | App writes to cache, async write to DB |
| TTL-based | Set expiry, accept stale data |

## Load Balancing

- **Round Robin** — simple, default
- **Least Connections** — for varying request duration
- **IP Hash** — for sticky sessions (avoid if possible)
- **Health checks** — remove unhealthy instances

## API Design

### REST Conventions

```
GET    /users          → List users
GET    /users/:id      → Get user
POST   /users          → Create user
PUT    /users/:id      → Full update
PATCH  /users/:id      → Partial update
DELETE /users/:id      → Delete user
```

### Pagination

- Cursor-based for real-time data (recommended)
- Offset-based for static data

### Versioning

- URL-based (`/v1/`, `/v2/`) — simplest
- Header-based — more flexible

## Reliability

- **Circuit breaker** — stop calling failing services
- **Retry with exponential backoff** — for transient failures
- **Timeout** — every external call needs a timeout
- **Health checks** — `/health` endpoint for load balancers
- **Graceful degradation** — work with reduced functionality

## Data Storage

| Use Case | Technology |
| --- | --- |
| Relational data, ACID | PostgreSQL |
| Key-value, caching | Redis |
| Full-text search | Elasticsearch, Meilisearch |
| Document store | MongoDB (if truly schemaless) |
| Time series | TimescaleDB, InfluxDB |
| Blob storage | S3, R2 |
| Queue | Redis Streams, RabbitMQ, SQS |

## Decision Framework

Before choosing an architecture:

1. What's the expected load? (users, requests/sec)
2. What's the team size?
3. What's the timeline?
4. What are the consistency requirements?
5. What's the budget?

**Default answer**: Start with a monolith, PostgreSQL, Redis cache. Extract services only when you have data showing you need to.

<!-- Source: EXAMPLES.md -->

# system-design Examples — Anti-patterns vs ContextOS Standard

## Example 1: Database Caching Strategy

### Anti-pattern: Cache-Aside with Unbounded Thundering Herd

```typescript
// BAD: When cache expires, 10,000 concurrent requests hit PostgreSQL simultaneously
async function getUserProfile(id: string) {
  const cached = await redis.get(`user:${id}`);
  if (cached) return JSON.parse(cached);
  const user = await db.user.findUnique({ where: { id } });
  await redis.set(`user:${id}`, JSON.stringify(user), 'EX', 300);
  return user;
}
```

### Best practice: ContextOS Standard (Mutex Lock / Single-Flight Pattern)

```typescript
// GOOD: Only one worker fetches from DB on cache miss; others wait
import { singleflight } from './singleflight';

async function getUserProfile(id: string) {
  const cached = await redis.get(`user:${id}`);
  if (cached) return JSON.parse(cached);

  return singleflight.do(`user:${id}`, async () => {
    const fresh = await redis.get(`user:${id}`);
    if (fresh) return JSON.parse(fresh);

    const user = await db.user.findUnique({ where: { id } });
    if (user) {
      await redis.set(`user:${id}`, JSON.stringify(user), 'EX', 300);
    }
    return user;
  });
}
```

---

## Example 2: Outbox Pattern for Distributed Consistency

### Anti-pattern: Dual-Write Anti-pattern (Direct DB write + Kafka publish)

```typescript
// BAD: If Kafka publish fails, DB change is committed but event is lost forever
async function createOrder(data: OrderInput) {
  const order = await db.order.create({ data });
  await kafkaProducer.send({ topic: 'orders', messages: [{ value: JSON.stringify(order) }] });
  return order;
}
```

### Best practice: ContextOS Standard (Transactional Outbox)

```typescript
// GOOD: Order and Outbox record committed in a single atomic DB transaction
async function createOrder(data: OrderInput) {
  return await db.$transaction(async (tx) => {
    const order = await tx.order.create({ data });
    await tx.outbox.create({
      data: {
        aggregateType: 'Order',
        aggregateId: order.id,
        eventType: 'OrderCreated',
        payload: JSON.stringify(order),
        status: 'PENDING',
      },
    });
    return order;
  });
}
```

<!-- Source: TROUBLESHOOTING.md -->

# system-design Troubleshooting & Common Mistakes

## 1. Serverless Connection Exhaustion

- **Symptom**: "FATAL: remaining connection slots are reserved for non-replication superuser connections" under modest traffic.
- **Root Cause**: Serverless/Edge functions opening new DB connection pools per invoked instance.
- **Fix**: Use a connection pooler like PgBouncer or managed pooling (Supabase connection pool, AWS RDS Proxy, Prisma Accelerate).

## 2. Cache Invalidation Drift

- **Symptom**: Users see stale, outdated data after making updates.
- **Root Cause**: Updates to database do not invalidate related cache keys, or TTLs are set to infinite.
- **Fix**: Invalidate cache keys explicitly on write in the same transactional flow, and always set defensive TTLs.

## 3. Lack of Rate Limiting and Backpressure

- **Symptom**: Backend crashes or slows to a crawl during traffic spikes or bot scraping.
- **Root Cause**: Unthrottled public endpoints without token-bucket or sliding-window rate limiting.
- **Fix**: Add rate-limiting middleware (Redis-backed sliding window) at the API gateway / Edge layer.