Load Balancing Standards

Load balancing standards — sharing traffic is easy; failing well is not

Criteria verified as of August 2026. Re-verify on the web before committing to anything (§8).

1. Scope and triggers

Applies to designing, configuring and operating a load balancer or reverse proxy: choice of layer, balancing algorithm, health checking, draining and zero-downtime deployment, TLS termination, HTTP protocols, high availability of the balancer itself, stateless balancing, and protection of the service edge.

Triggers: haproxy.cfg, nginx.conf (upstream, proxy_pass), Envoy bootstrap/xDS, traefik.yml, Caddyfile (reverse_proxy), keepalived.conf, ipvsadm, "target group", "backend service", "health check", "readiness", "drain", "sticky session", "consistent hashing", "maglev", "PROXY protocol", "X-Forwarded-For", "SNI passthrough", "ECMP", "anycast", "outlier detection".

Not applicable: the catalogue already splits this up: networking-standards is the trunk (proxies and balancing as a principle, VLAN, MTU, addressing) and already delegates the depth, while routing-switching-standards owns the campus, BGP policy and control-plane security —including the BGP announcement that makes anycast possible—, datacenter-fabric-standards owns the fabric, VXLAN/EVPN and lossless Ethernet, and network-automation-standards configuration as code. Outwards: caching and CDN belong to caching-cdn-standards, the service mesh and discovery to microservices-architecture-standards, Service, Ingress and Gateway API to kubernetes-standards, filtering to firewall-policy-standards, identity and OIDC to identity-access-management-standards, TLS and PKI to cryptography-pki-standards, metrics and dashboards to observability-standards, the SLO to sre-practice-standards, the measurement methodology and the load model to performance-engineering-standards, the reactive method to network-troubleshooting-standards, match-server session assignment to gaming-infrastructure-standards (a game server is stateful and non-interruptible: assigning a session is not balancing, and applying draining or per-request health to it breaks it), and the umbrella of onprem-standards (alongside datacenter-facilities-standards and hpc-standards). Among the three sibling skills of this batch: wireless-standards is the access network, this one the service network and high-speed-interconnect-standards the compute network; the domain's mistake is applying the same criteria to all three.

Guiding principle: balancing is deciding what happens when something fails. Sharing traffic is the trivial part; almost all the value is in the health check and in the draining. A balancer with a good algorithm and a bad health check sends traffic towards broken servers with exemplary precision.

2. Default decisions

Verify version, maintenance and raw licence before pinning any of these (§8).

3. Design criteria

L4 versus L7 — what you gain and what you lose

• L4 shares connections without understanding them: minimal cost per byte, any protocol, encryption intact. You lose routing by path or header, per-request retry, health with application semantics and all HTTP observability. L7 gives you all of that, but you pay CPU, latency and a component that is part of your application's semantics (and of its attack surface).
• Rule: L4 for non-HTTP traffic and raw throughput; L7 as soon as the decision depends on the content. Mixing layers in cascade (L4 in front, L7 behind) is legitimate and often the right thing.

Algorithms

• Round-robin: only with identical backends and requests of similar cost; otherwise it concentrates the expensive ones in the same place. Least connections: a reasonable default because it approximates "who is least busy", with mandatory slow start (a freshly restarted backend has zero connections and takes an avalanche). Two random choices is cheap and very good with large lists.
• Consistent hashing (Maglev, ketama): mandatory when the backend holds useful state per key —local cache, partitions, long sessions, per-tenant connections— and when the set changes frequently: a modulo-N hash reassigns all keys when a node is added or removed; the consistent one only its fraction. It is what allows several balancers to share the same way.

Session persistence: it is a smell

• Affinity ties a user to a server: it breaks draining, skews the sharing, turns every deployment into session loss and hides the real bug, which is state in the process's memory. If it exists, it is debt with an owner and a date; the solution is to move the session out. Legitimate exception: long-lived connections (WebSocket, SSE), where the affinity is of the connection, not a cookie.

Health checks — the section that decides everything

• Shallow (/healthz, no external dependencies) versus deep (are my dependencies responding?): useful, but with different consequences.
• The capital danger: a check that queries the database means that, when the database hiccups, the whole pool is marked unhealthy at once and the balancer withdraws 100% of the service for a problem that only degraded a part —the total outage is caused by the check, not the failure—. Mitigations: separate liveness from readiness, do not put shared dependencies into the check that governs the sharing, and a panic threshold (if more than X% of the pool is unhealthy, ignore health and share to all: degraded is better than switched off).
• Threshold asymmetry: go down fast (few failures), come back up slowly (several successes), so as not to oscillate. Intervals, timeouts and thresholds are declared, not inherited from the default; the timeout must be smaller than the interval or they overlap and falsify the state; and the check goes by the same path as the real traffic (same port, same TLS).
• The application decides when it is ready: /readyz must start failing before the process begins shutting down. That is the real mechanism of draining.

Draining and zero-downtime deployment

• Mandatory sequence: mark as unavailable → wait for the balancer to notice (≥ one full check cycle) → stop accepting new connections → finish the in-flight ones within a deadline → close. Skipping the wait is the usual cause of 502s during deployments. The grace period exceeds the longest legitimate request; long-lived connections are closed with an orderly signal (GOAWAY in H2). Retries only on idempotent things and with a budget: retrying everything under load turns a degradation into a storm.

TLS

• Terminating at the balancer simplifies certificates and gives visibility; re-encrypting towards the backend is the default when the internal leg is not physically trusted —"it's the internal network" is not an argument—, and mTLS when the balancer must prove who it is.
• Passthrough when the backend needs the client certificate or compliance forbids decrypting: it is routed by SNI and everything else is lost. It is a decision, not a default. On terminating you lose the real IP: PROXY protocol at L4, headers at L7 (§5).

HTTP/2 and HTTP/3 on the balancer

• H2 towards the client is the default. H2 towards the backend changes the balancing: it multiplexes many requests onto few connections, so balancing per connection stops sharing —you need per-request balancing, and even then a few persistent connections concentrate load—. It is the classic cause of "uneven balancing" after enabling internal H2.
• H3/QUIC runs over UDP: it changes the firewall, ECMP (hashing over UDP and the Connection ID), connection accounting and migration between networks. Verify the exact state per proxy and version (§8): maturity differs between the client-proxy and the proxy-backend direction.

HA of the balancer itself

• The balancer is the single point of failure par excellence: it concentrates all traffic and all connection state. A pair with VRRP (RFC 9568, which obsoletes 5798) or equivalent, with exercised failover, knowing that it cuts in-flight connections unless there is state synchronisation.
• ECMP + anycast scales better: N identical balancers announcing the same VIP, with no active-passive pair, no shared state and capacity that grows by adding nodes. The price: a change in the set reshuffles the ECMP hash and breaks connections, unless the nodes use consistent hashing towards the backends. It is what you need to know before buying a bigger appliance.

4. Quality gates

• Validate the configuration before applying it (haproxy -c -f, nginx -t, envoy --mode validate or the product's equivalent). A config that does not validate does not even reach staging.
• Backend failure test with real traffic: kill a backend and measure how many requests are lost and how long it takes to be withdrawn. If nobody has measured it, the number is unknown, not zero.
• Zero-downtime deployment test under load with zero 5xx as the acceptance criterion (it detects badly done draining), and a negative health test: degrade the shared dependency and verify that the whole pool is not withdrawn — the gate that prevents the total outage.
• Balancer failover test, including the return (it fails more than the outbound leg), and a forwarding header test: a spoofed X-Forwarded-For from outside and the application does not believe it. Config in the repo and applied by automation; a balancer that differs from its peer is a finding.

5. Security

• X-Forwarded-For without trimming is a vulnerability. It is a list that anyone can prepend to: if the application takes the first value, the attacker chooses their own IP and evades block lists, rate limits, geolocation and auditing. Rule: the edge balancer overwrites (does not append), or a fixed and known number of trusted proxies is counted from the right. The same goes for Forwarded (RFC 7239), X-Forwarded-Proto/-Host and X-Real-IP.
• Strip at the edge every internal header the application uses to decide (roles, "is internal", already-authenticated identity): an X-Authenticated-User that survives from outside is a complete authentication bypass.
• Rate limiting at the balancer by real IP and by credential/path, with 429 and limit headers: it protects even with the application saturated. Per-IP limits are bypassed behind NAT or CGNAT; combine them with per-identity limits.
• Flood protection: SYN cookies, per-source connection limits, aggressive handshake and header timeouts (Slowloris is killed with a header read timeout), maximum body and header size, and frame/stream limits in H2 (rapid reset is exhaustion, not throughput).
• The balancer's own surface: statistics and admin API never exposed; certificates with automatic renewal and an expiry alert (ciphers and TLS, in cryptography-pki-standards). And against request desynchronisation (request smuggling), which arises from proxy and backend interpreting Content-Length/Transfer-Encoding differently: reject ambiguous requests, normalise at the proxy and keep versions up to date at both ends.

6. Performance and operability

• Signals that are always watched: requests per second and per code, latency at high percentiles separating balancer queue from backend time, healthy versus configured backends, active connections and their real distribution per backend, retries and 502/503/504 by cause.
• Overflow and queues: the accept queue (backlog) and its kernel limit turn a burst into lost connections; it is sized and its overflow is monitored. A large backlog with no capacity behind it only swaps errors for latency.
• Ephemeral port exhaustion: opening a new connection per request towards few backends exhausts the source port range and fails intermittently. Solution: keepalive towards the backend with a sized pool, several source IPs if needed, and watch TIME_WAIT.
• Timeout coherence: the balancer's idle timeout must be smaller than the backend's keepalive; if the backend closes first, a dead connection gets reused and sporadic 502s appear —the hardest failure in the domain to reproduce. Write them in a table (client, balancer, backend, database) and verify they decrease. Capacity: size by real percentiles and by concurrent connections; TLS and H2 consume memory per connection.

7. Sustainability and prohibitions

• Cadence: LTS/stable branches versus the latest minor; quarterly review and on any CVE with relevant KEV/EPSS. The balancer is exposed: it is among the first to patch.
• nginx ecosystem (a datum that decides): nginx has belonged to F5 since 2019; in 2024 its lead developer forked it into freenginx over governance disagreements, and since 2022 there is Angie, from former core developers. All three share configuration and both forks are BSD-2-Clause; nginx remains active (copyright through 2026), so it is not an emergency, but if you choose nginx also decide whom you follow and verify each one's release cadence.
• Deprecation: every retired backend, rule and certificate disappears from the configuration and from the inventory. A commented-out server is not documentation.

FORBIDDEN

• ❌ A balancer with no active health check, or with TCP connect as the only signal for HTTP.
• ❌ A health check that queries the database or another shared dependency and can mark the whole pool unhealthy at once, with no panic threshold.
• ❌ Deploying without draining: withdrawing from the pool and killing the process without waiting a health cycle.
• ❌ Permanent session affinity with no owner or retirement date; source-IP affinity.
• ❌ Trusting X-Forwarded-For or Forwarded received from the client without overwriting or trimming.
• ❌ Letting internal trust or identity headers through to the backend.
• ❌ A single balancer for something that matters; or a pair whose failover has never been tested.
• ❌ Retrying non-idempotent requests, or retrying with no maximum budget.
• ❌ Enabling HTTP/3 without verifying its state in the specific version or adjusting firewall and ECMP for UDP.
• ❌ Backends without keepalive (and then blaming the network for ephemeral port exhaustion), or a balancer idle timeout greater than the backend keepalive (phantom 502s).
• ❌ Exposing the balancer's statistics page or admin API.
• ❌ Modulo-N hashing over a backend set that changes; consistent or nothing.
• ❌ Terminating TLS and speaking in the clear to the backend "because it's the internal network", with no written decision.
• ❌ Going to production without having measured how many requests are lost when a backend goes down.

8. Mandatory web verification

Methodology: the RFCs, one by one against the rfc-editor.org JSON; the licences, reading the raw file from the repository, not the GitHub label.

RFCs verified Aug 2026: HTTP/3 = RFC 9114 (Jun 2022, Proposed Standard, nothing obsoleted); HTTP/2 = RFC 9113 (Jun 2022, Proposed Standard, obsoletes 7540 and 8740 — citing RFC 7540 today is a factual error); the Forwarded header = RFC 7239 (Jun 2014, Proposed Standard); VRRPv3 = RFC 9568 (May 2024, Proposed Standard, obsoletes RFC 5798).

Licences verified raw Aug 2026: HAProxy — its LICENSE declares the core GPL v2 with the explicit intent of allowing external modules, and develops the headers scheme under LGPL: it is neither "plain GPL" nor MIT. nginx — 2-clause BSD, notice "Copyright (C) 2011-2026 Nginx, Inc." (active repository). Angie — the same BSD-2 text plus "Copyright (C) 2022-2026 Web Server LLC": a fork with an identical licence, and its LICENSE is on the master branch, not main (the usual path 404s and looks like a missing licence). Envoy and Caddy — Apache-2.0. Traefik — MIT, in LICENSE.md, not LICENSE.

Declared discrepancy: sources diverge on HTTP/3 maturity in Envoy — the downstream direction (client→proxy) is described as ready and the upstream one (proxy→backend) as alpha. Treat it as unresolved and verify the documentation of your exact version.

Declared gaps — DO NOT fill from memory:

1. Current stable versions and support windows of HAProxy, nginx, Angie, freenginx, Envoy, Traefik and Caddy: deliberately not pinned here; in particular, freenginx's release cadence and health are not verified.
2. Exact HTTP/3 state per proxy and version, with its TLS-with-QUIC library dependencies and connection migration: not verified product by product.
3. Managed cloud balancers (ALB/NLB, Application Gateway/Front Door, backend services): limits, draining, health semantics and HTTP/3 not verified here; they belong to aws/azure/gcp.
4. Katran is active but low-cadence and a mirror of Meta's internal code; Cilium/XDP and IPVS are not pinned by version. And the concrete values of backlog, health thresholds, grace periods and pool sizes are engineering criteria, not measurements: they are derived by measuring.

If the web contradicts this document, the web wins — flag the discrepancy.