HTTP Semantics

HTTP Semantics

Concept of the skill

HTTP semantics is the IETF-standardized contract layer (RFC 9110/9111/9112) consisting of three orthogonal axes — method semantics (what an action means: GET/POST/PUT/DELETE/PATCH and their safe/idempotent/cacheable properties), status code families (1xx informational / 2xx success / 3xx redirect / 4xx client-error / 5xx server-error with first-digit-as-contract), and conditional and caching metadata (ETag/If-Match/Cache-Control/Vary). Each axis is composable: a single response combines a method choice + status code + caching headers + conditional preconditions. Critically, RFC 9110 deliberately SEPARATES semantics from wire syntax — the same semantics are shared by HTTP/1.1 (RFC 9112), HTTP/2 (RFC 9113), and HTTP/3 (RFC 9114), so a method/status/header decision is version-independent. The wire-level meaning every HTTP API inherits, independent of any specific framework or protocol version.

Replaces framework-specific defaults with a portable spec contract. Without HTTP semantics as discipline, every API expresses meaning through framework conventions that do not match across stacks (Express does X, Rails does Y, Next.js does Z) — and the moment a request leaves the codebase (a CDN, a corporate proxy, a client library in another language, an HTTP/3 edge), it hits infrastructure that follows the RFC, not your framework. The alternative — "just whatever the framework does by default" — fails because most "weird HTTP behavior" stories come down to a framework default that does not match the spec. Knowing the spec lets you ground HTTP decisions in the wire-level contract, and lets you reach for the standardized header fields (Problem Details, Idempotency-Key, RateLimit, Content-Digest, Deprecation) instead of inventing per-API conventions that no off-the-shelf client, proxy, or gateway understands.

Distinct from api-design, which owns the API surface shape (routes, request/response JSON schemas, pagination, versioning) — http-semantics owns the underlying protocol contract that any HTTP API inherits, vendor-neutral, including the standardized media types and header fields that carry that contract on the wire. Distinct from webhook-integration, which owns inbound provider mechanics (signature verification, retry policy, idempotency keys, vendor-specific topics) — http-semantics owns vendor-neutral method/status/header semantics. Distinct from streaming-architecture, which owns bidirectional or long-running transport (WebSocket, SSE, HTTP/2 push) — http-semantics owns the unidirectional request-response contract. Distinct from system-interface-contracts, which is transport-agnostic (works for HTTP, gRPC, GraphQL, message bus) — http-semantics is HTTP-specific. HTTP semantics is to web APIs what the rules of grammar are to written language — the framework provides the dictionary (routes, handlers, middleware) but the grammar is non-negotiable, and a sentence that follows the dictionary while violating the grammar is technically intelligible but causes downstream tooling (CDNs, proxies, client libraries) to misinterpret it. The wrong mental model is that the framework's defaults ARE the HTTP contract. They are not — frameworks expose convenience APIs over the underlying spec, and those conveniences often diverge from the spec in subtle ways (Next.js's default cache headers, Express's connection-keep-alive behavior, the way different frameworks interpret 405 vs 404). Treating the framework as authoritative produces APIs that work locally but break at infrastructure boundaries — a CDN sees a 200 with an error body and caches it; a corporate proxy sees a POST as non-idempotent and retries it; a client library in a different language sees a missing Vary header and serves stale content; a gateway returns a 401 with no WWW-Authenticate and the client cannot tell how to authenticate. The discipline is to ground HTTP decisions in RFC 9110, not in the framework's documentation.

Coverage

The IETF contract layer for HTTP-based systems: version-independent semantics (RFC 9110), method semantics (§ 9), status code families (§ 15), representation metadata and validators (§ 8), request/response metadata (§ 6-8, 12), conditional requests (§ 13), content negotiation (§ 12), range requests (§ 14), request preferences (RFC 7240), structured field design (RFC 9651), and the caching model (RFC 9111). Extends to the modern header-field ecosystem built on these foundations: standardized error bodies (RFC 9457 Problem Details), payload integrity (RFC 9530 Digest Fields), retry-safety at scale (Idempotency-Key, RateLimit fields), cache observability (RFC 9211 Cache-Status, RFC 9213 Targeted Cache-Control, RFC 9875 Cache Groups), and API lifecycle signaling (RFC 9745 Deprecation, RFC 8594 Sunset). Applies to any system using HTTP as a transport: REST APIs, GraphQL-over-HTTP endpoints, webhook receivers, browser fetches, proxy behavior, CDN configuration, API clients, and agent/tool calls over HTTP — across HTTP/1.1, HTTP/2, and HTTP/3 identically.

This skill answers "what does this HTTP message mean to generic HTTP software?" It does not own the product-facing shape of an API, but it must be used to verify that the chosen API surface still obeys method, status, caching, negotiation, retry, lifecycle, integrity, and metadata semantics.

Philosophy of the skill

HTTP is older and more carefully specified than most of the frameworks running on top of it. RFC 9110 is the consolidated semantics reference (June 2022, obsoletes RFC 7230-7235); it represents 25+ years of distilled experience from browsers, proxies, CDNs, and load balancers. Most "weird HTTP behavior" stories come down to a framework default that doesn't match the spec.

The discipline is to ground HTTP decisions in the RFC, not the framework. When the framework default conflicts with the spec, the spec wins for portability: anything outside your codebase (a browser cache, a CDN, a corporate proxy, a retrying client, a crawler, an API gateway, a generated SDK, a client library in a different language) will follow the RFC, not your framework's documentation.

Status codes and headers are machine-readable control flow, not decoration. A client, cache, proxy, or agent decides whether to retry, revalidate, authenticate, follow a redirect, reuse a cached representation, or stop based entirely on the method, status family, and headers. If you return 200 with an error body, omit Vary, attach a weak ETag to an If-Match concurrency path, or use POST for a safe query because the framework made that easy, you have changed the protocol contract whether or not your handler code "worked."

Semantics are version-independent. A deliberate goal of the 2022 revision was to split semantics (RFC 9110) and caching (RFC 9111) away from wire syntax so each protocol version can evolve independently: HTTP/1.1 syntax is RFC 9112, HTTP/2 is RFC 9113, HTTP/3 (over QUIC) is RFC 9114. RFC 9110 states it plainly: "All three major versions of HTTP rely on the semantics defined by this document." A method/status/header decision is therefore a decision about meaning that holds identically whether the bytes travel over HTTP/1.1 text framing or an HTTP/3 QUIC stream. This is why this skill is about semantics, and transport syntax (RFC 9112/9113/9114) is out of scope.

Reach for the standard header field before inventing one. Most "API conventions" teams invent — error envelopes, idempotency tokens, rate-limit headers, integrity checksums, deprecation notices — now have a registered, off-the-shelf standard that generic clients, gateways, and proxies already understand. Inventing X-RateLimit-Remaining or {"error": ...} when RateLimit and application/problem+json exist creates a contract only your own client can read. The standard-field-first sections below name the current standard for each.

Method Semantics (RFC 9110 § 9)

The method token is the primary source of request semantics. Standard method names are case-sensitive (§ 9.1) and, once standardized, carry the same semantics on every resource — though each resource decides whether that method is implemented or allowed. The registry is closed-by-convention: inventing methods (PURGE, BAN) means only your own cache understands them.

Idempotency rule (RFC 9110 § 9.2.2): repeated requests have the same effect on resource state as a single request. The response code may differ across invocations (204 → 404 for DELETE); idempotency is about the post-condition, not the post-response.

Safe-method rule (§ 9.2.1): a method is safe when the client does not request, and does not expect, any state change from applying it. The spec is careful here: it does not forbid the server from having incidental side effects while serving a safe request — access logging, an ad-account charge triggered by a GET, metrics counters are all explicitly allowed. What matters is that the client did not request that behavior and cannot be held accountable for it. So "GET never changes anything on the server" is the wrong reading; the correct, load-bearing consequence is narrower: a state change must never be reachable only through a safe method, because crawlers, link prefetchers, and <a>-tag preloaders issue GETs unprompted — the classic "Googlebot deleted our records via GET links" failure. If a URI parameter selects an unsafe action (?do=delete), the resource owner MUST disallow that action under a safe method.

QUERY, and the GET-with-body trap. A recurring need is a safe, idempotent read whose parameters are too large or too structured for a URL query string (a complex search filter, a list of IDs). The wrong reach is "GET with a request body": RFC 9110 § 9.3.1 states content in a GET "has no generally defined semantics," cannot change the meaning of the request, and "might lead some implementations to reject the request and close the connection" (it reads like a request-smuggling vector to some intermediaries). So a GET body is non-portable — proxies, CDNs, and client libraries may strip or reject it; never put load-bearing data there. The standardized fix is the QUERY method (draft-ietf-httpbis-safe-method-w-body): a method that is safe and idempotent like GET but carries a request body, with cacheability keyed off the request content. As of this review QUERY is registered in the IANA HTTP Method Registry and IESG-approved with its defining document in the RFC Editor queue, but it is not yet a published RFC and ecosystem support (frameworks, proxies, gateways, generated SDKs, browsers) is still emerging — so for a public API today, prefer POST for a body-carrying "query" (accepting that it is not cacheable) and adopt QUERY once your stack and intermediaries support it. When you do adopt it, advertise it (Allow / Accept-Query) and check CORS-preflight impact. The portability rule to remember regardless: data the request depends on goes in the URL or in a POST/QUERY body, never in a GET body.

PUT vs PATCH, and the patch-document formats. PUT replaces the entire target representation; a field omitted from a PUT body is being set to absent. PATCH applies a partial change described by a patch document, and the format of that document is itself negotiated — PATCH is not "partial PUT," and the patch media type defines the operation, conflict model, and idempotency properties. The two registered JSON patch formats:

• JSON Merge Patch (RFC 7396, media type application/merge-patch+json) — the body looks like the resource; null means "delete this member." Simple, but cannot express array edits or set a value to null.
• JSON Patch (RFC 6902, media type application/json-patch+json) — an ordered array of add/remove/replace/move/copy/test operations with JSON Pointer paths. Expressive enough for array splices and conditional (test) edits.

A server advertises which patch formats it accepts with the Accept-Patch response header (RFC 5789 § 3.1), typically on the OPTIONS or 405 response, and rejects an unsupported patch format with 415 Unsupported Media Type. PATCH SHOULD be applied atomically — partial application is a spec violation. PATCH is not idempotent by default: some patch documents are idempotent by construction, but do not assume retry safety unless the format, target, and concurrency contract make it so. For state-changing PATCH, prefer a strong ETag + If-Match and return 412 Precondition Failed when the validator no longer matches.

Representation Metadata and Validators (RFC 9110 § 8)

Representation metadata says what the payload is and which selected representation it corresponds to. It is not interchangeable with transport framing, and it is the substrate conditional requests, range requests, and caching all build on.

ETag strength matters. A strong ETag ("abc") means byte-for-byte identical; a weak ETag (W/"abc") means semantically equivalent but possibly byte-different (e.g., a differently-whitespaced but equivalent body). Range requests and If-Match require a strong validator; If-None-Match cache validation accepts a weak one. Validator rule: send an ETag for representations where change detection can be consistently determined, and when content negotiation produces multiple simultaneous representations, validators must distinguish the variants that matter to cache updates and range requests — if gzip and identity encodings produce different bytes, their strong ETags must differ.

Status Code Selection

Use the first digit as the contract; use the last two digits as refinement. A proxy that doesn't understand a specific 4xx will treat it as a generic 4xx. A response body can add application detail, but it must not contradict the status family.

Companion headers — know which are MUST and which are SHOULD. Several status codes are incomplete without a paired header. The requirement level matters: for some of them omitting the header is an outright spec violation (a MUST); the rest are SHOULD / context-dependent — strongly recommended because real clients and proxies depend on them, but not a conformance failure. Do not flatten the distinction.

Auth rule. 401 means the request lacks valid authentication credentials and requires a WWW-Authenticate challenge ("who are you"). 403 means the server understood the request and refuses it ("you can't do that") — if credentials were provided, they are insufficient or the refusal is unrelated to credentials. A server may use 404 to hide a forbidden resource's existence, but that is a deliberate information-disclosure decision, not a substitute for authentication semantics. The two are not interchangeable.

Standard error body — RFC 9457 Problem Details (application/problem+json). When you return a 4xx/5xx with a body, the registered standard shape is a Problem Details object (RFC 9457, July 2023, obsoletes RFC 7807): type (a URI identifying the problem class), title, status, detail, instance, plus extension members. RFC 9457 added an IANA registry of common type URIs and support for reporting multiple problems. The protocol-level fact — that this standard media type exists and what its members mean — is in scope here; the decision of whether and how to adopt it for a given API's error contract is api-design's surface. The load-bearing rule this skill owns: an error must use a 4xx/5xx status as its primary signal, and the body is supplementary detail — Problem Details enriches the status code, it never replaces it, and never a 200 carrying {"error": ...}.

Rate-limit rule. For 429, pair the status with Retry-After when the client should wait a known duration. The IETF RateLimit / RateLimit-Policy fields are active Internet-Draft work as of this review — useful for emerging clients, but do not cite them as ratified RFC fields yet (see Rate Limiting below).

Conditional Requests (RFC 9110 § 13)

Use conditional requests to guard against lost updates (a concurrent overwrite of a representation you have already read) and to enable cache validation. Note the precise scope up front: a precondition protects a representation you already observed — it is not a general dedup mechanism for a brand-new POST whose outcome you never saw (see the Idempotency-Key section for that case).

A PUT/PATCH/DELETE made conditional with If-Match against a validator you already hold becomes safe to re-send: the server applies it once while the precondition holds and returns 412 Precondition Failed on every subsequent attempt. (A server MAY return success instead of 412 only when it can determine the state-changing operation already succeeded, and only when concurrent requests cannot corrupt state.) The boundary worth stating clearly: this works only because you have a validator for a representation you already fetched — it prevents clobbering a version you did not expect. It does not deduplicate the "I sent a POST /payments, the connection dropped, did it land?" case, where you hold no validator and the resource may not yet exist. For that, use an Idempotency-Key (below), not a conditional request.

Strong vs weak, and precedence. For lost-update protection, prefer strong ETags with If-Match; a weak ETag can validate a cache entry but is not enough to prove the representation data is unchanged for a write. Precondition evaluation has a defined precedence (RFC 9110 § 13.2.2): If-Match / If-Unmodified-Since (the "don't clobber" guards) are evaluated before If-None-Match / If-Modified-Since (the "don't re-send" guards). Use 428 Precondition Required when a resource requires clients to send a precondition before a state-changing request, making the concurrency contract explicit instead of allowing blind overwrites.

Range Requests (RFC 9110 § 14)

Range requests let a client fetch part of a selected representation — the foundation of resumable downloads, video seeking, and parallel chunked fetches.

A server that ignores Range and returns the whole body with 200 is spec-compliant but defeats resumable downloads; a server that returns 206 must include Content-Range. A range response is about the selected representation, not arbitrary pagination — do not use 206 for application-level list pagination; that stays 200 with explicit pagination fields or links.

Caching (RFC 9111)

Caching has four separate questions, not one — and the first is not reducible to "safe + 2xx":

1. May a cache store the response? A cache may store a response only when the conditions in RFC 9111 § 3 all hold: the request method and the response status code are defined as cacheable (200/203/204/206/300/301/308/404/405/410/414/501 are heuristically cacheable by default; POST only with explicit freshness), no-store is absent on both request and response, and — for a shared cache — private is absent and any Authorization header is accompanied by an explicit directive (public, must-revalidate, or s-maxage) permitting it. The cache must also understand status-code-specific rules for 206/304/must-understand. On top of that, the response needs a storage basis: an explicit freshness signal (max-age/s-maxage/Expires), public, no-cache, an extension permitting storage, or a heuristically cacheable status. no-store opts out entirely.
2. May the cache reuse the stored response for this request? Target URI, method semantics, Vary-selected request headers, no-cache, freshness/staleness, and validation all participate.
3. For how long is it fresh? s-maxage wins for shared caches, then max-age, then Expires; otherwise a heuristic may apply only in allowed cases. The Age response header reports how long a cached response has already been sitting in caches.
4. What is the cache key? At minimum method + target URI; Vary expands the key by request headers; QUERY cache keys must include the request content.

Cache-Control directive cheat sheet:

no-cache ≠ no-store. no-cache means "store it, but revalidate before each reuse"; no-store means "never write it to disk/memory at all." Reaching for no-cache when you meant no-store (e.g., for a response with PII) is a real privacy leak — the response is cached.

Shared-cache + Authorization. A response to a request carrying Authorization must not be reused by a shared cache unless the response explicitly allows shared caching (public, s-maxage, no-cache, or must-revalidate under RFC 9111's authenticated-response rules). Forgetting this leaks one user's authenticated response to another from a shared cache.

Targeted Cache-Control (RFC 9213). Lets an origin address a specific cache tier without affecting others, using named fields like CDN-Cache-Control (shared CDN) vs Cache-Control (everything downstream). Use it to give a CDN a long TTL while telling browsers no-store — reasoning carefully about Age, downstream freshness, and sensitive data.

Cache observability — Cache-Status (RFC 9211). A standardized response header that explains how each cache in the chain handled the request — Cache-Status: ExampleCDN; hit; ttl=300. Each cache appends its own member, so you can read the full hit/miss path of a layered CDN. This replaces the zoo of vendor-specific X-Cache: HIT headers with one parseable, multi-hop standard — reach for it when debugging "why is this stale / why didn't this cache." Vendor headers can still help local debugging, but Cache-Status is the interoperable signal.

Targeted invalidation — Cache Groups (RFC 9875). TTLs answer "how long is this fresh"; Cache Groups answer "how do I purge a set of related responses at once" with a standard instead of a vendor purge API. A response declares membership with the Cache-Groups header (a list of case-sensitive group names), and a later response to an unsafe method (POST/PUT/DELETE) carries Cache-Group-Invalidation to invalidate every stored response in the named groups. Scope is one cache, same-origin only; implementations support at least 32 groups per field. RFC 9875 (Standards Track, Oct 2025) is the portable replacement for the per-CDN tag/surrogate-key purge mechanisms teams currently lock into — reach for it when a single mutation must invalidate a fan-out of related cached resources.

Content Negotiation (RFC 9110 § 12)

Server-driven negotiation uses request Accept-* preference fields and response metadata. The response's Vary header is the cache contract: it lists the request fields that influenced representation selection.

Request:
  Accept: application/json, text/html;q=0.9
  Accept-Language: en, fr;q=0.8
  Accept-Encoding: br, gzip

Response:
  Content-Type: application/json
  Content-Language: en
  Content-Encoding: br
  ETag: "home-json-en-br-v7"
  Vary: Accept, Accept-Language, Accept-Encoding

The Vary response header is the contract for downstream caches: it lists the request headers the cache must match on. Forgetting Vary causes language-A users to see language-B responses cached by a CDN. For negotiated resources, validators must match the selected representation: if gzip and identity encodings produce different bytes, their strong ETags must differ.

Use 406 Not Acceptable when the target resource cannot provide a representation acceptable under the request's Accept-* constraints and the server is unwilling to choose a default. Use 415 Unsupported Media Type when request content or content coding is unsupported.

Vary pitfalls. Vary: * means "this response is uncacheable by a shared cache because it depends on factors beyond request headers." A frequent real bug: compressing responses but omitting Vary: Accept-Encoding, so a cache serves a gzip body to a client that did not send Accept-Encoding: gzip. Over-broad Vary (e.g., on the highly-unique User-Agent) destroys the cache hit rate by fragmenting the key — vary only on headers that genuinely change the representation.

Request Preferences (RFC 7240)

Content negotiation chooses a representation; the Prefer request header lets a client ask for optional processing behavior without changing the target resource or making the preference a hard precondition. Registered tokens include respond-async (process asynchronously — pairs with a 202 + a status resource), return=minimal vs return=representation (after a mutation, send back just headers or the full updated body), wait=N (bound how long the server holds an async request open), and handling=strict / handling=lenient (how aggressively to validate). Preferences are hints, not commands — the server MAY ignore them — and a server that honors one echoes it in the Preference-Applied response header so the client can tell. Prefer is not content negotiation, authentication, a validator, or a substitute for status codes; if the server ignores a preference, the response status still describes the actual HTTP outcome. Reach for Prefer instead of inventing ?async=true or X-Return-Minimal query params/headers.

Idempotency and Retry-Safety at Scale

HTTP method idempotency and idempotency keys solve different problems. Method idempotency is part of the method definition: PUT, DELETE, and safe methods can be retried because the requested effect is the same after one or many identical requests. Conditional requests (If-Match) make non-idempotent operations retry-safe when the client already holds a validator. When it does not — the classic "did my POST /payments succeed before the connection dropped?" — the de-facto industry pattern is the Idempotency-Key request header (used by Stripe, PayPal, and others).

Note its standardization status precisely: the IETF document (draft-ietf-httpapi-idempotency-key-header) reached draft-07 (Oct 2025) but is now an expired Internet-Draft, never published as an RFC — so Idempotency-Key is a widely-deployed convention, not a ratified standard. Treat it as the best available pattern (and useful pattern vocabulary) while knowing there is no RFC to point a client library at; do not describe it as an HTTP core standard. The client sends a unique key with a POST/PATCH; the server records the first outcome against that key and replays the stored response for any retry carrying the same key, instead of processing twice.

Server obligations and the response contract:

Scope the key to a fingerprint of the request, persist the outcome with a defined retention window (after expiry, the server cannot promise duplicate suppression), and never process twice for the same key. This is the protocol-level mechanism; per-vendor key semantics (e.g., a specific provider's required prefix) belong to webhook-integration / that vendor's docs.

Rate Limiting

When you return 429, the response should tell the client when to retry and ideally what the policy is:

• Retry-After (RFC 9110) — the minimum back-off, as seconds or an HTTP-date. The single most important header on a 429/503.
• RateLimit and RateLimit-Policy (draft-ietf-httpapi-ratelimit-headers, active Internet-Draft; not yet an RFC) — the standardizing replacement for the ad-hoc X-RateLimit-* family. RateLimit-Policy advertises the server's quota policy (windows, quotas, optionally concurrency limits); RateLimit reports the remaining quota for a policy. The draft consolidated to this two-field shape from the earlier three-header (X-RateLimit-Limit/-Remaining/-Reset) design. Prefer these standardized fields over inventing X--prefixed ones so generic client libraries and gateways can read your limits — but label them with their current IETF status before describing them as standards.

Payload Integrity (RFC 9530 Digest Fields)

When the bytes matter — large uploads, mirror reconstruction, content addressing — RFC 9530 (Feb 2024, obsoletes RFC 3230's Digest/Want-Digest) defines integrity fields with a clearer split. They are integrity, not authentication, authorization, privacy, or a full-message signature.

• Content-Digest — integrity of the message content actually transferred (after any range/encoding).
• Repr-Digest — integrity of the full representation (lets you validate a resource reassembled from multiple range requests or connections).
• Want-Content-Digest / Want-Repr-Digest — a receiver advertising that it wants a digest and which algorithms it prefers.

The old single Digest field was ambiguous about what it covered; the content-vs-representation split is the fix. Use these instead of inventing an X-Checksum header when you need end-to-end payload verification distinct from transport-level TLS integrity. If message metadata also needs integrity, combine digest fields with a signing mechanism; do not imply the digest field protects headers, identity, or transport.

API Lifecycle Signaling

HTTP has standardized fields to tell clients a resource is going away — far better than a changelog nobody reads. They communicate timing without changing the resource's current method or status semantics.

• Deprecation (RFC 9745, March 2025) — a response header signaling the resource is or will be deprecated, carrying a Structured-Field date (@<unix-timestamp>) for when. It is a hint and does not change the resource's behavior; it announces deprecation of the specific resource the response came from.
• Sunset (RFC 8594) — an HTTP-date for when the resource is expected to become unresponsive. Deprecation says "don't rely on this"; Sunset says "it stops working at time T."
• The deprecation and sunset link relations (Link: header) point at human docs describing the migration path.

Pairing Deprecation + Sunset + a Link; rel="deprecation" to migration docs is the standard, machine-readable way to retire an endpoint without breaking clients silently. Do not use deprecation headers as a substitute for accurate status codes: a deprecated-but-working resource still returns 200; a permanently removed one commonly returns 410 Gone; a hidden/unknown one may still return 404. Make the scope explicit — lifecycle fields on one response can be misread as applying to a wider API surface.

Structured Field Values (RFC 9651)

Nearly every modern header above (RateLimit, Cache-Status, Deprecation, Digest fields) is defined as a Structured Field (RFC 9651, Sept 2024, obsoletes RFC 8941) — a small set of typed data structures with one canonical parsing/serialization algorithm. The payoff: these headers parse deterministically across implementations instead of each one inventing its own comma/semicolon grammar.

Designing a new HTTP field. First check whether an existing registered field already carries the semantics. If a new

…(truncated)