Load Testing

Load Testing

Purpose

Design an experiment whose result applies to a declared production question. Precise percentiles can still describe the wrong workload, a saturated generator, unrepresentative data, a changing JVM, censored failures, or an arrival process that slows itself when the service slows.

Distinguish offered work, started/admitted work, attempts including retries and fan-out, successful useful work, and rejected/abandoned/timed-out work. They answer different questions and diverge under overload.

Workflow

1. State the claim and experimental unit

Define population, system boundary, configuration, response variable, SLO window and intended generalization. Decide whether one run, node, build, cluster or time block is the independent unit. Requests within one run are correlated and do not make one deployment thousands of independent replications.

2. Choose the workload model from causality

Use an open model when arrivals occur independently of prior completion: public requests, scheduled messages or an externally imposed rate. Use a closed model when a fixed population genuinely waits or thinks before its next operation.

For a closed population in equilibrium:

[ X=\frac{N}{R+Z} ]

where (N) is population, (R) mean response time and (Z) mean think time. This is the interactive response-time law, not a ceiling on individual latency. A closed test can show saturation, but its offered rate falls as response time grows.

An open system with unbounded backlog and (\lambda>\mu) has no steady state. Real systems often have finite queues, deadlines, rejection or shedding and can reach a lossy steady state; measure those outcomes.

3. Model workload and state

Derive operation mix, payload/data/key and tenant distributions, workflows, locality, session/think times, retries and background work from a relevant production window. Preserve meaningful correlation and bursts.

Define cold-start, cold-cache and warm steady state separately. End warmup when relevant signals stabilize—compilation, cache hit rate, connection establishment, allocation/GC and response distribution—not after a universal duration. Do not discard startup when startup is the subject.

Predeclare the stability criterion and maximum warmup budget. If stability never arrives, report that state rather than extending warmup until a favorable interval can be selected.

4. Control environment and generator

Pin artifact, JDK, JVM, resources, dependencies and placement. Prefer isolated generators and a representative network path. Colocation is valid only when it reproduces production or shared contention is deliberately under test.

Capacity-plan generator CPU, memory, network, connections, ephemeral ports and result output. Little's Law supplies an initial arrival-executor concurrency estimate from iteration rate and duration; a pilot must verify scheduled starts and generator headroom.

5. Define validity before the headline run

Verify business correctness, data isolation, output schema, clock alignment and telemetry overhead. Predeclare:

• reconciliation of scheduled/offered and started arrivals;
• generator CPU, pauses, network and connection bounds;
• target identity and dependency state;
• missing/dropped-start classification;
• timeout, graceful-stop and incomplete-work treatment;
• workload-fidelity checks.

A dropped scheduled iteration limits a claim about the configured arrival schedule, but the run can still reveal overload behavior. Occupied VUs may reflect slow target responses, slow scenario code or insufficient allocation, not necessarily saturated generator hardware. Record start lateness as well as missing starts: zero drops alone does not prove timing fidelity. The missing attempts are unobserved target outcomes; do not invent their latencies or silently classify the executor as a closed loop.

6. Repeat and quantify uncertainty

Choose repetitions from between-run variability and the decision-relevant effect, not a fixed count. Randomize or block version/order comparisons. Report run-level distributions, uncertainty and environment drift; do not pool all requests as independent observations.

7. Diagnose through a causal chain

Correlate arrival/admission, queues, useful throughput, latency/error, resources, JVM events and dependencies. A plateau alone does not identify its cause. Collect discriminating evidence, change one causal factor, and reproduce.

Measurement rules

• Use the mean for total-work, conservation and resource-demand questions; use relevant quantiles/tails for latency objectives. No statistic is universally forbidden.
• Publish sample count, duration, histogram precision/range, timeout/censoring policy and aggregation dimensions with percentiles.
• Do not average or add percentiles across replicas, windows or path components.
• Do not treat error responses as fast successes. Report outcome and latency jointly.
• Separate client- and server-observed latency.
• Name each timer's start/end and included phases. Report scheduled-to-start delay separately from started-request latency; a tool's request-duration metric may omit DNS, connection or TLS time. Client timeout ends the client wait, not necessarily server work.
• Explicitly start JFR and control its overhead. Native Memory Tracking requires startup configuration and does not enumerate Java objects.

Decision table

Failure modes

Anti-patterns

Validity equals SLO pass. A stress run can be valid while deliberately violating its SLO. Validity asks whether the scheduled experiment occurred; acceptance asks whether the system met its objective.

One request equals one iteration. An iteration can execute a workflow, parallel resources or retries. Calibrate rate to the declared work unit.

Precision without replication. Millions of requests in one run estimate that run's distribution, not build/node/time variability.

Universal generator rules. Separate hosts and generous preallocation are common good choices, but topology and allocation must follow the production claim and measured generator behavior.

Cross-skill routing

• Read test plan and validity for the executable contract.
• Use load-testing-advanced for breakpoint, burst, stress and soak profiles.
• Use coordinated-omission for scheduled-arrival loss and correction limits.
• Use latency-statistics for uncertainty and comparisons.
• Use capacity-planning for provisioning decisions.

Authoritative references

• Grafana k6: open and closed models
• Grafana k6: arrival-rate VU allocation
• Gatling: workload models
• Apache JMeter: Open Model Thread Group
• JDK Mission Control
• k6 dropped iterations and their causes