System Design

System Design

Design Process (Always Follow This Order)

1. Clarify requirements — functional AND non-functional
2. Estimate scale — users, requests/sec, data volume, read/write ratio
3. Define the API — what does the system expose?
4. Design the data model — what data, how stored, how accessed?
5. High-level architecture — major components and their relationships
6. Deep dive — zoom into the hardest or most critical parts
7. Identify bottlenecks — and how to address them

Requirements Clarification

Before designing anything, answer:

Functional: What does the system do? Scale: How many users? Requests/sec? Data volume? Growth rate? Availability: What's the acceptable downtime? (99.9% = 8.7h/yr, 99.99% = 52m/yr) Latency: What's the acceptable response time? (p99) Consistency: Strong vs eventual consistency? (CAP theorem) Read/Write ratio: Read-heavy? Write-heavy? Mixed? Geography: Single region? Global? Data sovereignty requirements?

Scalability Patterns

Horizontal vs Vertical Scaling

Default: design for horizontal scaling from the start.

Load Balancing

• Round Robin — equal distribution, simple, doesn't account for load
• Least Connections — routes to server with fewest active connections
• Consistent Hashing — for distributed caches, minimizes redistribution on scaling

Caching Strategy

Cache Invalidation Strategies:

• TTL (Time-to-Live) — expire after N seconds. Simple, may serve stale data.
• Write-through — update cache on every write. Strong consistency, slower writes.
• Write-behind — async cache update after write. Fast writes, eventual consistency.
• Cache-aside — app manages cache explicitly. Most flexible.

Data Architecture

SQL vs NoSQL Decision

Database Sharding

Partition data across multiple databases:

• Horizontal sharding — split rows (e.g., users A-M on DB1, N-Z on DB2)
• Vertical sharding — split tables by feature (users DB, orders DB, payments DB)
• Hash-based — shard key hashed to determine partition

Read Replicas

• Primary handles writes
• Replicas handle reads
• Good for read-heavy workloads (social feeds, analytics)
• Trade-off: replication lag = eventual consistency

Architecture Patterns

Monolith vs Microservices

Rule: Start monolith, extract services when you feel the pain. Premature microservices is a common mistake.

Event-Driven Architecture

Use when:

• Services need to be decoupled
• High throughput, async processing
• Audit trail / event sourcing needed

Tools: Kafka (high throughput), RabbitMQ (complex routing), AWS SQS (simple, managed)

CQRS (Command Query Responsibility Segregation)

Separate read and write models:

• Write model — optimized for consistency and transactions
• Read model — optimized for query performance (denormalized, pre-computed)

Good for: high read/write ratio disparity, complex domain logic

Common System Design Components

API Gateway

• Single entry point for all clients
• Handles auth, rate limiting, SSL termination, routing
• Tools: AWS API Gateway, Kong, Nginx

Message Queue

• Decouples producers from consumers
• Enables async processing, load leveling
• Provides buffer for traffic spikes

CDN (Content Delivery Network)

• Serve static assets from edge locations close to users
• Reduce latency, reduce origin server load
• Cache invalidation is the hard part

Search

• Don't use your primary DB for full-text search
• Elasticsearch / OpenSearch for complex text search
• Typesense for simpler, faster search

Non-Functional Requirements Cheatsheet

Trade-Off Framework

For every design decision, explicitly state the trade-off:

Decision: [what you chose]
Why: [what problem it solves]
Trade-off: [what you give up]
Alternative: [what you considered]
When to reconsider: [conditions that would change this decision]

Back-of-Envelope Estimation

Useful numbers:

• 1M users × 10 req/day = ~120 requests/second
• 1KB × 1M requests/day = ~1GB/day of data
• SSD read: ~0.1ms | Network: ~1ms | HDD: ~10ms
• Single server handles ~10K-50K requests/sec (depends on complexity)
• RAM: 1 server ≈ 64-256GB | Typical process ≈ 100MB-1GB