Caching

Caching Strategies & Patterns

Overview

Caching is the practice of storing copies of data in a faster storage layer so that future requests for that data are served more quickly. Effective caching can reduce database load by orders of magnitude, cut response latency dramatically, and improve system resilience -- but incorrect caching introduces stale data, consistency bugs, and operational complexity. Choosing the right caching strategy is a critical backend architecture decision.

Core Caching Patterns

Cache-Aside (Lazy Loading)

The application manages the cache explicitly. On a read, the application checks the cache first. On a miss, it loads from the database, stores the result in cache, and returns it.

Read path:
  1. App checks cache for key
  2. Cache HIT → return cached value
  3. Cache MISS → query database
  4. Store result in cache with TTL
  5. Return result

Write path:
  1. App writes to database
  2. App invalidates (deletes) the cache key

Pros: Simple to implement; cache only contains data that is actually requested; works with any database. Cons: First request always hits the database (cold start); potential for stale data between write and invalidation.

Best for: General-purpose caching where the application can tolerate brief staleness.

Read-Through

The cache sits in front of the database and loads data transparently on a miss. The application always reads from the cache -- it never talks to the database directly for reads.

Read path:
  1. App reads from cache
  2. Cache HIT → return cached value
  3. Cache MISS → cache loads from database automatically
  4. Cache stores result, returns to app

Pros: Cleaner application code (no cache miss handling); cache warms itself. Cons: Requires a cache layer that supports read-through (or a wrapper); first request still slow.

Best for: Workloads where you want the caching logic decoupled from application code.

Write-Through

Writes go to the cache first, and the cache synchronously writes to the database before confirming the write to the application.

Write path:
  1. App writes to cache
  2. Cache writes to database synchronously
  3. Cache confirms write to app

Pros: Cache is always consistent with the database; no stale reads after writes. Cons: Higher write latency (two writes in series); cache may contain data that is never read.

Best for: Read-heavy workloads where consistency is critical and write volume is moderate.

Write-Behind (Write-Back)

Writes go to the cache immediately, and the cache asynchronously flushes changes to the database in the background.

Write path:
  1. App writes to cache
  2. Cache confirms write to app immediately
  3. Cache asynchronously flushes to database (batched, periodic)

Pros: Very low write latency; writes can be batched for efficiency; absorbs write spikes. Cons: Risk of data loss if the cache fails before flushing; eventual consistency with the database; complex to implement correctly.

Best for: Write-heavy workloads where write latency matters more than durability guarantees (e.g., analytics counters, session updates).

Write-Around

Writes go directly to the database, bypassing the cache entirely. The cache is populated only on subsequent reads (via cache-aside or read-through).

Write path:
  1. App writes directly to database (cache not involved)

Read path:
  1. App reads from cache
  2. Cache MISS → load from database, populate cache

Pros: Avoids polluting the cache with data that may never be read; simple write path. Cons: Recently written data always misses the cache on first read.

Best for: Write-heavy workloads where most written data is rarely read immediately (e.g., log ingestion, audit trails).

Choosing a Caching Strategy

Decision heuristic:

• Start with cache-aside -- it is the simplest and most widely applicable pattern.
• Use write-through when you need strong consistency between cache and database.
• Use write-behind when write latency is critical and you can tolerate potential data loss.
• Use write-around when most writes are not read back immediately.
• Use read-through when you want to keep caching logic out of your application code.

Cache Invalidation Strategies

Cache invalidation is one of the two hard problems in computer science (along with naming things and off-by-one errors).

Recommendation: Use TTL as a safety net on every cache entry (even with event-based invalidation). This ensures that stale data eventually expires even if an invalidation event is lost.

Cache Stampede / Thundering Herd

When a popular cache key expires, many concurrent requests simultaneously miss the cache and hit the database, potentially overwhelming it.

Solutions

# Probabilistic early expiration (XFetch algorithm)
import random, time

def get_with_early_expiration(cache, key, ttl, beta=1.0):
    entry = cache.get(key)
    if entry is None:
        return recompute_and_cache(cache, key, ttl)

    value, expiry, delta = entry
    # delta = time it took to recompute last time
    # Probabilistically refresh before actual expiry
    if time.time() - delta * beta * random.random() >= expiry:
        return recompute_and_cache(cache, key, ttl)

    return value

CDN Caching

Content Delivery Networks cache responses at edge locations close to users.

Cache-Control Headers

Cache-Control: public, max-age=3600, s-maxage=86400, stale-while-revalidate=60

Edge Caching Strategy

• Cache static assets with long TTLs and content-hash filenames (app.a1b2c3.js with immutable).
• Cache API responses at the CDN with s-maxage and stale-while-revalidate.
• Use cache tags or surrogate keys for targeted invalidation (supported by Fastly, CloudFront, Cloudflare).

Application Caching

Redis vs. Memcached

Recommendation: Use Redis unless you need only simple string caching and prefer Memcached's multi-threaded model for pure throughput at scale.

In-Memory / Local Cache

• Fastest possible access (no network hop).
• Limited by process memory; not shared across instances.
• Use for extremely hot data with short TTLs (e.g., config, feature flags, rate limit counters).
• Examples: Caffeine (Java), MemoryCache (.NET), node-cache (Node.js), lru-cache (Python).

HTTP Caching

ETag and Last-Modified

Response:
  ETag: "abc123"
  Last-Modified: Tue, 15 Jan 2024 10:00:00 GMT

Subsequent request:
  If-None-Match: "abc123"
  If-Modified-Since: Tue, 15 Jan 2024 10:00:00 GMT

Server response if unchanged:
  304 Not Modified (no body, saves bandwidth)

• ETag -- opaque identifier for a specific version of a resource (hash of content or version number).
• Last-Modified -- timestamp of last change.
• Both enable conditional requests that save bandwidth when content has not changed.

Multi-Tier Caching

┌─────────────┐    ┌─────────────────┐    ┌──────────┐    ┌──────────┐
│   Client     │───>│  CDN (L3)       │───>│ Redis    │───>│ Database │
│   Browser    │    │  Edge cache     │    │ (L2)     │    │          │
│   cache (L0) │    │                 │    │ Distrib. │    │          │
└─────────────┘    └─────────────────┘    │ cache    │    │          │
                                          └──────────┘    └──────────┘
                                               ▲
                                          ┌──────────┐
                                          │ In-proc  │
                                          │ cache(L1)│
                                          └──────────┘

Strategy: Check L1 first, then L2, then L3, then origin. Write-through from origin to L2; let L1 fill on demand with short TTLs. CDN serves public, cacheable content.

Best Practices

• Always set a TTL on every cache entry -- even if you also use event-based invalidation. TTL is your safety net against stale data from missed events.
• Monitor cache hit rates. A hit rate below 80% suggests the cache is not well-tuned for actual access patterns. Investigate and adjust.
• Design for cache failure gracefully: the system must function (possibly with degraded performance) when the cache is unavailable.
• Never cache sensitive data (credentials, tokens, PII) without encryption and strict TTLs.
• Use consistent hashing for distributed cache clusters to minimize key redistribution when nodes are added or removed.
• Prefer cache-aside as the starting pattern. Only move to more complex patterns (write-through, write-behind) when you have measured evidence that they are needed.
• Implement cache stampede protection (locking or probabilistic refresh) for any high-traffic cache keys with expensive recomputation.