# Caching Strategies

> Deciding what to cache, where, and how to keep it coherent — trading freshness for speed and load reduction across the cache layers (client, CDN/edge, application in-memory, distributed cache, database). Covers the two hard problems (cache invalidation and choosing the layer), the write/read patterns (cache-aside/lazy, read-through, write-through, write-behind, refresh-ahead), expiry and TTL design, eviction policies (LRU/LFU/FIFO/TTL), staleness budgets and tolerated inconsistency, cache coherence and invalidation strategies (TTL expiry, explicit invalidation, key/tag versioning, write-through), the thundering-herd / cache-stampede problem and its mitigations (request coalescing, locks, jitter, stale-while-revalidate), HTTP caching semantics (Cache-Control, ETag, conditional requests, stale-while-revalidate/stale-if-error), CDN and edge caching, multi-layer cache hierarchies, cache key design and cardinality, negative caching, and measuring hit ratio and the cost of a miss. Stack-agnostic across Redis/Memcac

- Skill: `jacob-balslev/caching-strategies` (Agent Skill, multi-file: 3 files)
- Install (CLI): `npx skillmds@latest add jacob-balslev/caching-strategies`
- Raw SKILL.md: https://api.skillmd.com/api/skills/jacob-balslev/caching-strategies/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- License: MIT
- Author: jacob-balslev (https://skillmd.com/u/jacob-balslev)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/jacob-balslev/caching-strategies

---


# Caching Strategies

## Concept of the skill

A cache stores a copy of data closer to the consumer so repeated reads avoid the expensive recompute or refetch. The win is real — an expensive query becomes a fast lookup — but the copy creates a **coherence problem**: it can drift from the source of truth. Caching is the discipline of capturing the speed-and-load win while controlling the staleness it introduces.

Every caching decision is two coupled choices made for a specific **layer**:

1. **Staleness budget** — how out-of-date may this data be? (A price quote: seconds. A user's avatar: hours. A blog post: minutes.)
2. **Invalidation strategy** — how does the copy get corrected when the source changes? (Expiry, explicit invalidation, versioned keys, write-through.)

The two famously hard problems, per the field's oldest joke, are **cache invalidation** and naming things — and caching forces you to confront the first one directly.

## Coverage

- **The cache layers** — client/browser, CDN/edge, application in-memory, distributed cache (Redis/Memcached), and the database's own caches — and what each is good for.
- **Read/write patterns** — cache-aside (lazy loading), read-through, write-through, write-behind (write-back), refresh-ahead — with consistency and failure tradeoffs.
- **TTL and expiry** design and the **staleness budget**.
- **Eviction policies** — LRU, LFU, FIFO, TTL — and sizing.
- **Cache coherence and invalidation** — time-based expiry, explicit/event-driven invalidation, key/tag versioning, write-through coherence.
- **Cache stampede / thundering herd** — and mitigations: request coalescing (single-flight), locks/leases, TTL jitter, stale-while-revalidate, probabilistic early refresh.
- **HTTP caching semantics** — Cache-Control, ETag and conditional requests, stale-while-revalidate / stale-if-error — and CDN/edge caching.
- **Cache key design** — cardinality, namespacing, and the per-user-in-a-shared-cache trap.
- **Negative caching** of misses.
- **Measurement** — hit ratio and the cost of a miss.

## Philosophy of the skill

**Caching is a correctness risk traded for a performance win — treat it as such.** A cache that serves stale or cross-tenant data is worse than no cache: it is fast and wrong. Before adding a cache, state the staleness budget and the invalidation strategy explicitly. If you cannot say how the cache gets invalidated, you are not ready to add it.

**The miss path is the design.** A cache makes the hot path fast, but the system's behavior under misses — cold start, mass expiry, a hot key invalidated — is where caches cause outages. Design the miss path (coalescing, jitter, stale-while-revalidate) with as much care as the hit path.

## The cache layers

```
client/browser ─► CDN/edge ─► app in-memory ─► distributed cache ─► database
   (per user)      (shared,     (fastest,        (shared across       (source of
                    geo)         per-instance)     instances, Redis)     truth)
```

Choosing the layer is half the skill. Per-user data must not sit in a shared CDN (you'll leak one user's response to another). Cross-instance consistency needs a distributed cache, not per-instance memory. Static assets belong at the edge. The closer to the consumer, the faster — and the harder to invalidate.

## Read/write patterns

- **Cache-aside (lazy loading):** the app checks the cache; on a miss it loads from the source, populates the cache, and returns. Simple and resilient (cache down ≠ app down), but the first request after expiry pays the miss, and there's a small write-skew window.
- **Read-through:** the cache itself loads from the source on a miss (the app only talks to the cache). Cleaner app code; couples app to the cache.
- **Write-through:** writes go to the cache and the source synchronously — strong coherence, higher write latency.
- **Write-behind (write-back):** writes hit the cache and flush to the source asynchronously — fast writes, risk of loss if the cache dies before flush.
- **Refresh-ahead:** proactively refresh hot keys before they expire — hides miss latency, wastes work on cold keys.

Pick by the consistency you need and the failure you can tolerate.

## Invalidation strategies

- **TTL / time-based expiry** — simplest; data is correct-enough within the staleness budget, wrong after a source change until expiry.
- **Explicit / event-driven invalidation** — on a write to the source, delete or update the cached key. Coherent, but you must find *every* key affected by a change.
- **Key/tag versioning** — embed a version (or a tag) in the key; bump it to invalidate a whole class at once without enumerating keys.
- **Write-through coherence** — the write updates the cache, so it never goes stale (at write-latency cost).

There is no free invalidation: TTL is easy but stale-prone; explicit is coherent but hard to get complete; versioning trades memory for simplicity.

## Cache stampede (thundering herd)

When a hot key expires (or many keys share an expiry), the next requests all miss simultaneously and slam the backend with identical work — a self-inflicted load spike that can cascade into an outage. Mitigations:

- **Request coalescing / single-flight** — only one request recomputes the key; the rest wait and share the result.
- **Locks / leases** — the first miss takes a lease to recompute; others serve stale or wait.
- **TTL jitter** — randomize expiry so keys don't all expire together.
- **stale-while-revalidate** — serve the stale value immediately and refresh in the background.
- **Probabilistic early refresh** — refresh a key slightly before expiry with rising probability, smoothing the miss.

A caching design that ignores the miss-storm is incomplete.

## HTTP caching

For cacheable HTTP responses, the protocol gives you the tools: `Cache-Control` (max-age, s-maxage, public/private, no-store), `ETag` + conditional `If-None-Match` for cheap revalidation (304 Not Modified), and `stale-while-revalidate` / `stale-if-error` for graceful freshness. CDNs honor these directives at the edge. `private` vs `public` is a correctness control — never let a `private` (per-user) response be cached by a shared CDN.

## Verification

A caching design is sound when you can answer yes to:

- Is the **staleness budget** stated for each cached datum?
- Is there a real **invalidation strategy** (not just a TTL) for data that changes?
- Is the **layer** correct for the data's sharing scope (no per-user data in a shared cache)?
- Is the **read/write pattern** chosen for the consistency you actually need?
- Is the **miss storm** handled (coalescing, jitter, or stale-while-revalidate)?
- Do you **measure hit ratio** and know the **cost of a miss**?
- For HTTP, are `private`/`public` and validation headers correct?

If any answer is no, name the gap before shipping the cache.

## Do NOT Use When

- The task is making the **source query itself faster** via an index — that is `indexing-strategy`. A cache fronts the query; it does not optimize it.
- The task is **rewriting the query** (N+1 → join, removing a full scan) — that is `query-optimization`.
- The task is **HTTP protocol semantics in general** (methods, status codes, content negotiation) — that is `http-semantics`; caching uses only its Cache-Control/ETag subset.
- The task is **connection pooling** or **CDN/infrastructure provisioning** — out of scope.

## References

- RFC 9111 — HTTP Caching: https://www.rfc-editor.org/rfc/rfc9111
- Cache-Aside pattern (Azure): https://learn.microsoft.com/en-us/azure/architecture/patterns/cache-aside
- Redis patterns: https://redis.io/docs/latest/develop/use/patterns/
- stale-while-revalidate: https://web.dev/articles/stale-while-revalidate
- Scaling Memcache at Facebook (NSDI 2013): https://www.usenix.org/system/files/conference/nsdi13/nsdi13-final197.pdf
- Cloudflare cache concepts: https://developers.cloudflare.com/cache/concepts/cache-control/