System Design

system-design

Overview

Scalable system architecture blueprint based on the System Design Primer and DDIA. Enforces load balancing, multi-tier caching (Redis, CDN), database partitioning, CAP theorem tradeoffs, and rate limiting before code is written.

When to Use

Activate during the PLAN phase of any backend service, API design, database schema creation, or scalability optimization.

Rules & Patterns

Based on donnemartin/system-design-primer — the most starred system design resource on GitHub.

Core Principle

Everything is a trade-off. Before writing a single line of backend code, reason through the system at scale. A flat monolith that works now fails at 10× load.

Mandatory Pre-Design Checklist

Before architecting any backend system, answer these questions:

1. Scale: What is the expected QPS (queries per second)? Peak vs average?
2. Data volume: How much data? Growth rate? 1GB? 1TB? 1PB?
3. Consistency vs Availability: Can we tolerate eventual consistency? (CAP theorem)
4. Read/Write ratio: Is it read-heavy (cache it!) or write-heavy (shard it!)?
5. Latency requirements: Real-time (<100ms)? Near-real-time (<1s)? Batch?
6. Global distribution: Single region or multi-region?
7. Deployment model: Traditional servers, Serverless, or Edge functions?

Core Architecture Patterns

Load Balancing

Clients → Load Balancer → [App Server 1, App Server 2, App Server N]

• Use Round Robin for stateless services
• Use Least Connections for varying request times
• Use IP Hash for session affinity (or move sessions to Redis)
• Always add health checks — remove unhealthy nodes automatically

Rule: Any service expecting > 1000 RPS needs a load balancer. No exceptions.

Caching Strategy

App → [Cache Layer: Redis/Memcached] → Database

Cache decision ladder (check in order):

1. Is it read > write? → Cache it
2. Is it expensive to compute? → Cache it
3. Is it user-specific? → Cache with user key
4. Is it global? → Shared cache, shorter TTL

Cache patterns:

• Cache-aside (lazy loading): check cache → miss → load DB → write cache
• Write-through: write to DB AND cache simultaneously (consistency > performance)
• Write-behind: write to cache → async flush to DB (performance > consistency)

Invalidation: Use TTL + event-driven invalidation. Never stale-forever.

Modern framework-native caching (Next.js App Router):
Before spinning up a dedicated Redis instance for caching API responses, check if Next.js built-in mechanisms are sufficient:

• revalidatePath('/dashboard') — invalidate all cache for a route
• revalidateTag('user-profile') — fine-grained tagged cache invalidation
• unstable_cache() — server-side data caching with TTL

// [GOOD] Use Next.js native caching first
import { revalidateTag } from 'next/cache'

const getUser = unstable_cache(
  async (id: string) => db.users.findById(id),
  ['user'],
  { tags: ['user-profile'], revalidate: 3600 }
)

// Invalidate on mutation:
await db.users.update(id, data)
revalidateTag('user-profile')

// [BAD] Don't add Redis for simple SSR caching when Next.js handles it

Database Architecture

When to use SQL vs NoSQL

Scaling Databases

Vertical scaling: Bigger machine. Easy but has ceiling.
Read replicas: Route SELECT to replicas, writes to primary.
Sharding (horizontal partitioning):

• Hash sharding: user_id % N — even distribution, hard to rebalance
• Range sharding: user_id 1-1M on shard 1 — easy range queries, hotspots risk
• Directory-based: lookup table maps key → shard — flexible, but lookup is overhead

Denormalization: For read-heavy systems, duplicate data to avoid joins.
Rule: Don't shard until you've maxed out read replicas.

Message Queues & Async Processing

Producer → [Queue: Redis/RabbitMQ/Kafka] → Consumer Workers

Use queues when:

• Operation takes > 200ms (email, PDF generation, ML inference)
• You need retry logic on failure
• You need to decouple services
• Traffic spikes need to be absorbed

Kafka = durability + replay + high throughput (events/analytics)
Redis Queue = simplicity + low latency (jobs/tasks)
RabbitMQ = complex routing + acknowledgements

Microservices vs Monolith

Start with a monolith unless you have > 10 engineers or proven scale need.

When to split into microservices:

• Independent deployment cycles needed
• Different scaling requirements per service
• Team autonomy (Conway's Law)
• Clear service boundaries (DDD bounded contexts)

Rule: A microservice should be able to be rewritten in 2 weeks by 2 engineers.

Service communication:

• Sync (REST/gRPC): when caller needs immediate response
• Async (events/queue): when caller can tolerate delay, or decoupling is needed
• BFF / Server Actions: for web apps, prefer typed client-server contracts (see below)

Modern Stack Patterns (Serverless, Edge, Next.js)

Serverless & Edge Architecture

When deploying to serverless (Vercel Functions, AWS Lambda) or edge (Vercel Edge, Cloudflare Workers), the classical "App Server + Load Balancer" model changes:

Cold Start Problem:

• Serverless functions spin up from zero on first request — this can add 100–1000ms
• Never do heavy initialization at module level (DB connections, config loading, crypto keys)
• Always initialize lazily inside the handler, or use a connection pooling service

// [BAD] Wrong: Module-level initialization (runs on cold start, hangs the function)
const db = new DatabaseClient({ ... })  // top of file

// [GOOD] Correct: Lazy initialization with caching
let db: DatabaseClient | null = null
function getDb() {
  if (!db) db = new DatabaseClient({ ... })
  return db
}

DB Connection Pooling in Serverless:

• Traditional in-process pools (pg-pool, knex) do NOT work in serverless — each invocation is ephemeral
• Use Prisma Accelerate, PlanetScale, Neon pooling, or Supabase — they handle pooling at the infrastructure level
• Rule: If deploying to Vercel/serverless, NEVER assume max_connections is managed by your app process

Edge Functions limitations:

• No Node.js APIs (no fs, no crypto.randomBytes, limited DNS)
• Latency must be < 50ms — no heavy DB queries
• Use edge for: auth token verification, A/B testing, geo-routing, lightweight transformations

BFF Pattern & Server Actions (Type-Safe Client-Server)

When building web apps, prefer typed client-server communication over generic REST endpoints:

Option 1: Server Actions (Next.js App Router)
For mutations that touch the database directly, skip the API route entirely:

// [GOOD] Server Action: No API route needed, fully type-safe
"use server"
export async function updateUser(id: string, data: UpdateUserInput) {
  // Input validation (always!)
  const validated = UpdateUserSchema.parse(data)
  
  // Auth check (always before data access!)
  const session = await getSession()
  if (session.userId !== id && !session.isAdmin) {
    throw new Error("Forbidden")
  }
  
  return db.users.update(id, validated)
}

// [BAD] Over-engineering: Don't create /api/users/[id] + fetch wrapper for simple mutations

Option 2: tRPC (Full-stack type safety)
For complex APIs with many routes, use tRPC to get end-to-end type safety from DB to UI without code generation.

Option 3: REST (When appropriate)
When building a public API consumed by external clients or mobile apps — use REST with OpenAPI spec.

Decision rule:

• Internal web-to-DB mutation → Server Action
• Internal complex API → tRPC
• Public/mobile API → REST + OpenAPI

Domain-Driven Design (DDD) — Business Logic Isolation

Rule: NEVER write business logic inside API route handlers, Server Actions, or controllers. Always extract to dedicated services/use-cases.

[FAIL] Wrong structure:
app/api/orders/route.ts  ← contains: validation + auth + business logic + DB query

[PASS] Correct structure:
app/api/orders/route.ts  ← only: parse request, call service, return response
src/services/order.service.ts  ← all business logic, testable without HTTP context
src/repositories/order.repo.ts  ← all DB queries

Example:

// [BAD] Business logic in route (untestable, bloated)
export async function POST(req: Request) {
  const data = await req.json()
  if (data.quantity <= 0) return new Response("Invalid", { status: 400 })
  const inventory = await db.inventory.findById(data.productId)
  if (inventory.stock < data.quantity) return new Response("Out of stock", { status: 400 })
  const total = inventory.price * data.quantity
  // ... 40 more lines
}

// [GOOD] Thin route, fat service
export async function POST(req: Request) {
  const data = await req.json()
  const result = await orderService.createOrder(data)
  return Response.json(result)
}

// orderService.createOrder() — pure function, fully unit-testable without HTTP

Scalability Design Patterns

CDN (Content Delivery Network)

• Serve static assets (JS, CSS, images) from CDN edge nodes
• Cache API responses that don't change per-user
• Reduce origin server load by 80%+

Rate Limiting

Always implement for public APIs:

- Token bucket: smooth bursts, allows brief spikes
- Leaky bucket: strict rate, no bursts
- Fixed window: simple, vulnerable to boundary spikes
- Sliding window: most accurate, slightly more complex

Store rate limit state in Redis (not in-process — it doesn't survive restarts).

Circuit Breaker

Prevent cascade failures:

CLOSED (normal) → [failures > threshold] → OPEN (fail fast)
  ↑                                               ↓
  └────── [timeout] ← HALF-OPEN (test request) ──┘

Database Connection Pooling

• Traditional servers: Use pg-pool, knex, Prisma connection pool
• Serverless: Use Prisma Accelerate, PlanetScale, Neon, or Supabase pooling — NOT in-process pools

CAP Theorem in Practice

You can only guarantee 2 of 3: Consistency, Availability, Partition Tolerance

For most apps: Choose AP. Accept eventual consistency. Use optimistic locking for critical writes.

Designing Data-Intensive Applications (DDIA) Patterns

Based on Designing Data-Intensive Applications (Martin Kleppmann) and ciembor/agent-rules-books.

1. The Dual-Write Problem & Transactional Outbox

The Anti-Pattern: Updating the database and sending a message to a broker (Kafka, RabbitMQ, SQS) in two separate operations. If one fails, the system enters an inconsistent state.

The Solution: Write the business entity AND an event record to an outbox table in the SAME database transaction:

BEGIN TRANSACTION;
  UPDATE orders SET status = 'PAID' WHERE id = 'ord_123';
  INSERT INTO outbox_events (id, aggregate_type, aggregate_id, event_type, payload, created_at)
  VALUES ('evt_456', 'Order', 'ord_123', 'OrderPaid', '{"amount": 99.00}', NOW());
COMMIT;

A background process (polling worker or Debezium CDC) reads outbox_events, delivers them to the message broker, and marks them as published.

2. Idempotency Invariant for Mutations

All write operations exposed over HTTP or queues MUST support deduplication:

• Accept an Idempotency-Key header (UUID or client-generated hash).
• Store key with status in Redis or DB with a TTL (e.g., 24 hours).
• If the key is already COMPLETED, return the cached response immediately without re-executing.
• If IN_PROGRESS, return HTTP 409 Conflict or queue retry.

3. Read-Your-Own-Writes Consistency

When using read replicas, replication lag (even 50ms) causes users to not see their own changes immediately after saving:

• Rule: Route user reads to the primary database for N seconds (e.g., 5s) following any mutation by that user.
• Route all other queries and background jobs to read replicas.

Architecture Decision Template

When proposing any backend architecture, include:

## System Design Decision

**Scale Target**: [X RPS, Y GB data, Z users]
**Deployment Model**: [Traditional servers | Serverless | Edge]
**CAP Choice**: [CP/AP] because [reason]
**Read/Write Ratio**: [X:Y]

### Components
- **API Layer**: [REST/tRPC/Server Actions] — [why this choice]
- **Cache**: [Next.js native | Redis] for [what] with [TTL/tags strategy]
- **Database**: [SQL/NoSQL] - [pooling solution for serverless if applicable]
- **Async**: [Queue tech] for [what operations]

### Business Logic Isolation
- Services: [list key service files]
- Repositories: [list key repo files]
- Routes/Actions: [thin handlers only]

### Trade-offs Accepted
- [Trade-off 1]: [Why acceptable]
- [Trade-off 2]: [Why acceptable]

### Scaling Path
1. Now (MVP): [simple setup]
2. At 10× load: [first scaling step]
3. At 100× load: [next scaling step]

Code Examples

See EXAMPLES.md for detailed code examples.

Validation Checklist

What to verify during the review phase before completing the task.

Common Mistakes

Anti-patterns and things to explicitly avoid. See TROUBLESHOOTING.md.

Integration Notes

How this skill interacts with other skills.

system-design Examples — Anti-patterns vs ContextOS Standard

Example 1: Database Caching Strategy

Anti-pattern: Cache-Aside with Unbounded Thundering Herd

// BAD: When cache expires, 10,000 concurrent requests hit PostgreSQL simultaneously
async function getUserProfile(id: string) {
  const cached = await redis.get(`user:${id}`);
  if (cached) return JSON.parse(cached);
  const user = await db.user.findUnique({ where: { id } });
  await redis.set(`user:${id}`, JSON.stringify(user), 'EX', 300);
  return user;
}

Best practice: ContextOS Standard (Mutex Lock / Single-Flight Pattern)

// GOOD: Only one worker fetches from DB on cache miss; others wait
import { singleflight } from './singleflight';

async function getUserProfile(id: string) {
  const cached = await redis.get(`user:${id}`);
  if (cached) return JSON.parse(cached);

  return singleflight.do(`user:${id}`, async () => {
    const fresh = await redis.get(`user:${id}`);
    if (fresh) return JSON.parse(fresh);

    const user = await db.user.findUnique({ where: { id } });
    if (user) {
      await redis.set(`user:${id}`, JSON.stringify(user), 'EX', 300);
    }
    return user;
  });
}

Example 2: Outbox Pattern for Distributed Consistency

Anti-pattern: Dual-Write Anti-pattern (Direct DB write + Kafka publish)

// BAD: If Kafka publish fails, DB change is committed but event is lost forever
async function createOrder(data: OrderInput) {
  const order = await db.order.create({ data });
  await kafkaProducer.send({ topic: 'orders', messages: [{ value: JSON.stringify(order) }] });
  return order;
}

Best practice: ContextOS Standard (Transactional Outbox)

// GOOD: Order and Outbox record committed in a single atomic DB transaction
async function createOrder(data: OrderInput) {
  return await db.$transaction(async (tx) => {
    const order = await tx.order.create({ data });
    await tx.outbox.create({
      data: {
        aggregateType: 'Order',
        aggregateId: order.id,
        eventType: 'OrderCreated',
        payload: JSON.stringify(order),
        status: 'PENDING',
      },
    });
    return order;
  });
}

system-design Troubleshooting & Common Mistakes

1. Serverless Connection Exhaustion

• Symptom: "FATAL: remaining connection slots are reserved for non-replication superuser connections" under modest traffic.
• Root Cause: Serverless/Edge functions opening new DB connection pools per invoked instance.
• Fix: Use a connection pooler like PgBouncer or managed pooling (Supabase connection pool, AWS RDS Proxy, Prisma Accelerate).

2. Cache Invalidation Drift

• Symptom: Users see stale, outdated data after making updates.
• Root Cause: Updates to database do not invalidate related cache keys, or TTLs are set to infinite.
• Fix: Invalidate cache keys explicitly on write in the same transactional flow, and always set defensive TTLs.

3. Lack of Rate Limiting and Backpressure

• Symptom: Backend crashes or slows to a crawl during traffic spikes or bot scraping.
• Root Cause: Unthrottled public endpoints without token-bucket or sliding-window rate limiting.
• Fix: Add rate-limiting middleware (Redis-backed sliding window) at the API gateway / Edge layer.