Backend

Backend Architecture

Overview

Backend architecture encompasses the server-side decisions that determine how a system stores data, exposes functionality, handles security, and scales under load. The choices made at this level -- API style, database type, caching strategy, authentication mechanism -- ripple through every layer of the application and are difficult to change once established.

This skill provides a decision-making framework drawn from Martin Kleppmann's Designing Data-Intensive Applications and industry-proven patterns for building reliable, scalable, and maintainable backend systems.

Knowledge Map

┌─────────────────────────────────────────────────────────────────┐
│  API Layer                                                       │
│  REST, GraphQL, gRPC, WebSocket                                 │
│  → How clients communicate with the backend                     │
├─────────────────────────────────────────────────────────────────┤
│  Data Storage                     │  Caching                    │
│  Relational, Document, Graph,     │  In-memory, Distributed,   │
│  Key-Value, Time-Series           │  CDN, HTTP caching          │
│  → How data is persisted          │  → How hot data is served   │
├─────────────────────────────────────────────────────────────────┤
│  Authentication & Authorization   │  Background Processing      │
│  OAuth 2.0, JWT, RBAC, ABAC,     │  Job queues, schedulers,   │
│  Multi-tenancy                    │  event-driven workers       │
├─────────────────────────────────────────────────────────────────┤
│  Rate Limiting & Throttling       │  Observability              │
│  Token bucket, sliding window,    │  Logging, metrics, tracing, │
│  API quotas, circuit breakers     │  health checks, alerting    │
└─────────────────────────────────────────────────────────────────┘

Choosing an API Style

Decision heuristic:

• Default to REST for public APIs and simple CRUD services.
• Choose GraphQL when clients need flexible, aggregated queries across multiple resources (especially mobile).
• Choose gRPC for internal service-to-service communication where latency and throughput matter.
• Choose WebSocket when you need real-time, bidirectional data flow (chat, live dashboards, collaborative editing).
• Many systems combine styles: REST for public API, gRPC internally, WebSocket for real-time features.

Choosing a Database Type

Decision heuristic:

• Default to relational (PostgreSQL) when data is structured and relationships matter.
• Choose document when schema flexibility and developer velocity are priorities, and joins are rare.
• Choose graph when the primary queries traverse relationships (social networks, recommendations, fraud detection).
• Choose key-value for caching, sessions, and simple lookup-by-key workloads.
• Choose time-series for append-heavy, time-stamped data with downsampling and retention needs.
• Polyglot persistence is common: use the right database for each bounded context.

Backend Architecture Concerns

Rate Limiting & Throttling

Protect backend services from abuse and overload:

• Token Bucket -- allows bursts up to a configured capacity, refills at a steady rate.
• Sliding Window -- counts requests in a rolling time window for smoother limiting.
• Fixed Window -- simple counter per time window (risk of burst at window boundaries).
• Leaky Bucket -- processes requests at a constant rate, queuing excess.
• Implement at the API gateway layer for consistency across services.

Background Processing

Offload long-running or non-urgent work from the request/response cycle:

• Job queues (Sidekiq, Celery, Hangfire, BullMQ) -- enqueue work, process asynchronously.
• Scheduled jobs (cron, Quartz, Hangfire recurring) -- time-triggered processing.
• Event-driven workers -- react to domain events from a message broker.
• Batch processing -- periodic bulk operations (ETL, report generation).
• Always design for idempotency -- workers may process the same job more than once.

Observability

The three pillars of observability, plus health monitoring:

• Logging -- structured logs (JSON) with correlation IDs for request tracing.
• Metrics -- counters, gauges, histograms (request rate, error rate, latency percentiles).
• Distributed Tracing -- end-to-end trace across services (OpenTelemetry, Jaeger, Zipkin).
• Health checks -- liveness (is the process running?) and readiness (can it serve traffic?).
• Alerting -- thresholds on key metrics (error rate > 1%, p99 latency > 500ms).

Canonical Reference

• Designing Data-Intensive Applications by Martin Kleppmann -- the definitive guide to data storage, replication, partitioning, encoding, and distributed system trade-offs. Essential reading for any backend architect.

Sub-Skills

• dev/backend/data-modeling -- Data modeling and database architecture patterns
• dev/backend/api-design -- API design patterns for REST, GraphQL, gRPC, WebSocket
• dev/backend/caching -- Caching strategies and patterns
• dev/backend/authentication -- Authentication and authorization patterns

Best Practices

• Start with a monolith and extract services only when complexity demands it -- premature microservices add coordination cost without proportional benefit.
• Choose boring technology by default. PostgreSQL, Redis, and a well-designed REST API solve the vast majority of backend problems.
• Design for failure: every network call can fail, every database can be slow. Use timeouts, retries with backoff, circuit breakers, and fallbacks.
• Make operations idempotent wherever possible -- especially for writes, background jobs, and event handlers.
• Instrument everything from day one. Adding observability retroactively is far more expensive than building it in.
• Treat API contracts as public commitments: version explicitly, deprecate gracefully, never break existing clients without a migration path.