Load Testing

Load Testing (Verbose)

Core Patterns

The Four Shapes, and What Only Each One Finds

Most teams run one test — expected peak for ten minutes — and believe they have load tested. Each shape isolates a different failure mode, and the others stay invisible until production finds them.

A leak of 4 MB/hour is invisible in a 20-minute run and takes down a pod in three days. A ten-minute ramp to peak gives an autoscaler nine minutes of warning it will never get when a push notification lands. The shape is not cosmetic — it determines which bugs are reachable.

Spike tests also expose the recovery half of the curve, which is usually worse than the spike: after the drop, retried requests and rebuilt caches keep the system saturated long after offered load is normal.

Open vs Closed Workload Models

This is the single most consequential choice in the harness, and the default in most tools is the wrong one.

A closed model holds a fixed population of virtual users. Each sends a request, waits for the response, thinks, sends again. Throughput is therefore users / (latency + think time) — a feedback loop. When the server slows, offered load automatically drops.

An open model holds arrival rate fixed. Requests arrive on a schedule regardless of whether prior ones have returned. If the server slows, work piles up, exactly as it does when real users hit refresh and upstream services retry.

import http from 'k6/http';
import { check } from 'k6';

export const options = {
  discardResponseBodies: true,
  scenarios: {
    ramp_arrival: {
      executor: 'ramping-arrival-rate',
      startRate: 200,
      timeUnit: '1s',
      preAllocatedVUs: 200,
      maxVUs: 3000,          // headroom, or the generator becomes the bottleneck
      stages: [
        { target: 200, duration: '3m' },   // warm-up, excluded from analysis
        { target: 800, duration: '5m' },
        { target: 1600, duration: '5m' },
        { target: 2400, duration: '5m' },
        { target: 3200, duration: '5m' },
      ],
    },
  },
  thresholds: {
    http_req_failed: ['rate<0.01'],
    http_req_duration: ['p(95)<500', 'p(99)<1500'],
  },
};

export default function () {
  const res = http.get('https://api.example.com/products?page=1');
  check(res, { 'status 200': (r) => r.status === 200 });
}

The closed-model failure is subtle and common: a team runs 500 VUs, sees stable throughput and rising-but-bounded latency, and concludes the system degrades gracefully. In production the same system collapses at the same load, because real arrivals do not back off. The closed model measured a self-throttling version of the workload that does not exist.

Use a closed model only when it matches reality — a fixed pool of worker processes calling your service, or a batch job with bounded concurrency.

Why Averages Lie

The mean is dominated by the bulk of fast requests and is nearly blind to the tail that users actually notice.

Reported as an average, build B is a 3 ms regression. Reported honestly, build B has a GC pause or a lock convoy and one request in a hundred takes four seconds.

Tail latency compounds with fan-out. If a page issues 40 backend calls and each independently has a 1% chance of exceeding 4 s, then 1 - 0.99^40 ≈ 33% of page renders contain one. The p99 of a dependency becomes the p66 of the page.

Two aggregation errors to avoid:

• Averaging percentiles. The mean of twenty per-minute p95 values is not the p95 of the run. Merge histograms, or compute over the whole window.
• Including the ramp. Percentiles over a period where load was changing describe no particular load level.

Warming Up

The first minute of any run measures initialisation, not steady state:

• JIT-compiled runtimes (JVM, .NET, V8) interpret bytecode until methods pass their compilation threshold — early requests can be 10–50× slower.
• Caches are empty, so hit rate starts at zero and climbs.
• Connection pools are cold; every early request pays TCP and TLS setup.
• Lazily-initialised singletons, ORM metadata, and class loading fire once.
• Autoscalers sit at minimum replica count.

Ramp for 2–5 minutes at low rate, confirm the system reached steady state (flat p99, stable cache hit rate, stable RSS), then start the measurement window. In k6, express this as a separate warm-up scenario with startTime on the measured one, so the warm-up traffic never enters the reported metrics.

Production-Like Data

Query plans are chosen from statistics, so behaviour changes with cardinality.

• A nested-loop join that is optimal for 1,000 rows becomes catastrophic at 10M when the planner still estimates low and the loop actually runs 10M times.
• An index that fits in shared_buffers at small scale spills to disk at large scale, converting memory reads into random IO.
• Uniform synthetic keys give a flat access distribution, so every cache looks effective. Real traffic is Zipfian: a small hot set plus a long cold tail. The cold tail is what causes cache misses and slow queries.

Match, at minimum: row counts within an order of magnitude, realistic key skew, realistic value sizes (a 2 KB JSON blob behaves differently from "x"), and the same indexes. Feed a parameter corpus rather than one hardcoded id:

import { SharedArray } from 'k6/data';

// Init context: parsed once, shared across all VUs
const skus = new SharedArray('skus', () => JSON.parse(open('./skus.json')));

export default function () {
  const sku = skus[Math.floor(Math.random() * skus.length)];
  http.get(`https://api.example.com/products/${sku}`, {
    tags: { name: 'GET /products/:sku' },   // group dynamic paths
  });
}

Finding the Knee

The goal of a stress test is a curve, not a verdict. Step the arrival rate and record, at each step, throughput, p50/p95/p99, error rate, and saturation signals from the system under test.

Three things are readable here and none of them is pass/fail. The knee is near 2,900. Past 3,200, completed throughput falls while offered load rises — the signature of retry amplification and timeout waste. And pool wait time tracks p99 almost exactly, which names the bottleneck: the connection pool, not CPU.

That last column is the point. Latency alone tells you it broke; resource telemetry tells you what to fix. Always collect CPU, memory, GC pause, pool utilisation and queue depth alongside the client-side numbers. See performance-optimization for what to do once the bottleneck is named.

Not Measuring Your Own Load Generator

A load generator that is itself saturated silently under-delivers and adds its own latency to every sample. Before trusting any run, confirm:

• Generator CPU below ~70% and no run-queue backlog.
• Enough source ports and file descriptors (ulimit -n); ephemeral port exhaustion looks exactly like server-side connection refusal.
• k6 dropped_iterations at zero — nonzero means maxVUs was too low and the intended arrival rate was never actually offered.
• Generator and target in the same region unless you are deliberately testing WAN latency; 80 ms of cross-continent RTT drowns the signal you want.

Common Anti-Patterns

❌ Reporting a single average response time. It hides the tail entirely and makes tail regressions invisible. ✅ Report p50/p95/p99/max, and set thresholds on p95 and p99.

❌ A fixed VU count with sleep(), presented as realistic traffic. It is a closed model; it self-throttles and cannot produce queue collapse. ✅ Use an arrival-rate executor unless a closed model genuinely matches the caller.

❌ Including ramp-up and cold-start in the reported window. Every run starts with its worst numbers and the comparison between builds becomes noise. ✅ Warm to steady state, then measure a separate, clearly bounded window.

❌ Testing against a dev database with 5,000 rows. Query plans, cache hit rates, and index residency are all different from production. ✅ Restore an anonymised production-scale dataset with realistic key skew.

❌ Replaying one hardcoded user id or product id. Cache hit rate approaches 100% and you measure the cache, not the system. ✅ Drive from a parameter corpus with a production-like distribution.

❌ A pass/fail number with no curve behind it. "Handles 2,000 req/s" says nothing about what happens at 2,100. ✅ Publish the load-versus-latency curve and name the bottleneck resource.

❌ Load testing only before launch. The knee moves with every schema change and dependency upgrade. ✅ Run a short load test on a schedule, and trend the results across builds.

❌ Ignoring the generator's own health. Ephemeral port exhaustion and dropped iterations are read as server failures. ✅ Assert on generator saturation metrics before accepting any result.

Load Testing Checklist

• Test shape chosen to match the failure class in question (load/stress/soak/spike)
• Soak run long enough to reveal leaks (≥ 4 h) before any major release
• Spike test covers both the surge and the recovery period after it
• Workload model is open (arrival-rate) unless a closed model matches reality
• Warm-up period defined and excluded from reported metrics
• Steady state confirmed (flat p99, stable RSS, stable cache hit rate) before measuring
• Dataset within an order of magnitude of production row counts
• Request parameters drawn from a corpus with realistic key skew
• p50/p95/p99/max reported; no bare averages
• Percentiles computed from histograms, not averaged across time buckets
• Load-versus-latency curve produced and the knee identified
• Server-side resource telemetry captured alongside client latency
• Bottleneck resource named in the result, not just the failing load level
• Load generator CPU, fd, and port headroom verified; dropped iterations at zero
• Thresholds expressed against SLOs rather than invented round numbers
• Results trended across builds so regressions surface as a slope