API Content Negotiation

Content Negotiation

CONTENT NEGOTIATION IS THE HTTP MECHANISM BY WHICH CLIENTS AND SERVERS AGREE ON THE FORMAT, LANGUAGE, AND ENCODING OF A RESPONSE — ENABLING A SINGLE ENDPOINT TO SERVE JSON, XML, CSV, OR VERSIONED MEDIA TYPES WITHOUT SEPARATE URLS. IGNORING CONTENT NEGOTIATION FORCES VERSIONING THROUGH URLS OR QUERY PARAMETERS AND MAKES FORMAT DISCOVERY OPAQUE.

When to Use

• Designing an API endpoint that must serve multiple response formats (JSON, XML, CSV)
• Implementing media-type-based API versioning (application/vnd.myapp.v2+json)
• Diagnosing a 406 Not Acceptable error from a client or proxy
• Deciding between URL versioning (/v2/users) and header versioning (Accept: application/vnd.api.v2+json)
• Supporting internationalized responses where language selection matters
• Building a public API where clients may request compressed or alternative encodings
• Reviewing a PR that hardcodes Content-Type: application/json without honoring the Accept header
• Configuring a reverse proxy or CDN to vary caching by Accept header

Instructions

Key Concepts

1. Accept Header (Client-Driven Negotiation) — The client advertises acceptable response media types in order of preference using quality factors (q=). The server selects the best match and responds with the chosen type in Content-Type. If no acceptable type is available, the server returns 406 Not Acceptable.

   GET /reports/q1-2024
   Accept: text/csv;q=0.9, application/json;q=1.0, */*;q=0.1
   

   The server reads this as: JSON preferred (q=1.0), CSV acceptable (q=0.9), anything else as last resort.

2. Content-Type Header — Declares the media type of the request body (on POST/PUT/PATCH) or response body. The client sets it on requests with bodies; the server sets it on responses. Mismatch between declared and actual type causes parsing failures.

   POST /events
   Content-Type: application/json
   
   { "type": "order.completed", "orderId": "ord_123" }
   

3. Media Types and Vendor Types — Media types follow the pattern type/subtype[+suffix][;parameter]. Vendor types (application/vnd.*) allow APIs to declare version-specific or format-specific contracts. For example, application/vnd.github.v3+json is GitHub's versioned JSON type. The +json suffix tells generic parsers they can treat the body as JSON even without specific type knowledge.

4. Quality Factors (q values) — Values from 0.0 to 1.0 indicating relative preference. Default is 1.0. q=0 means "not acceptable." Used in Accept, Accept-Language, Accept-Encoding, and Accept-Charset headers. Servers must implement negotiation logic that respects q-value ordering.

   Accept: application/json;q=1.0, application/xml;q=0.8, text/plain;q=0.5
   

5. Vary Header — Tells downstream caches (CDNs, proxies, browsers) which request headers were used in content negotiation. A response that varies by Accept must include Vary: Accept. Without this, a CDN may serve a JSON response to a client requesting CSV if both requests hit the same cache key.

   HTTP/1.1 200 OK
   Content-Type: application/json
   Vary: Accept, Accept-Language
   

6. Accept-Encoding and Compression — Clients declare supported compression algorithms; servers respond with compressed bodies and Content-Encoding headers. gzip and br (Brotli) are the most common. Compression negotiation is separate from format negotiation.

   GET /large-dataset
   Accept-Encoding: br, gzip;q=0.8
   

   HTTP/1.1 200 OK
   Content-Encoding: br
   Content-Type: application/json
   

Worked Example

GitHub's API demonstrates media-type versioning through content negotiation. GitHub uses Accept headers both for version selection and for enabling preview features:

Request the default v3 JSON response:

GET /repos/octocat/hello-world
Authorization: Bearer ghp_...
Accept: application/vnd.github.v3+json

HTTP/1.1 200 OK
Content-Type: application/vnd.github.v3+json
Vary: Accept, Authorization
X-GitHub-Media-Type: github.v3; format=json

{
  "id": 1296269,
  "name": "hello-world",
  "full_name": "octocat/hello-world",
  ...
}

Request raw file content (format negotiation, same endpoint):

GET /repos/octocat/hello-world/contents/README.md
Accept: application/vnd.github.raw+json

HTTP/1.1 200 OK
Content-Type: text/plain
Vary: Accept

# Hello World
...

Enable a preview feature via Accept header (GitHub Reaction preview):

GET /repos/octocat/hello-world/issues/1
Accept: application/vnd.github.squirrel-girl-preview+json

The same URL returns an augmented response with reactions field when the preview media type is requested. This is GitHub's mechanism for progressive feature rollout without URL proliferation.

406 Not Acceptable — requesting an unsupported type:

GET /repos/octocat/hello-world
Accept: application/x-yaml

HTTP/1.1 406 Not Acceptable
Content-Type: application/json

{ "message": "Must accept 'application/vnd.github.v3+json'" }

Anti-Patterns

1. Ignoring the Accept header and always returning JSON. A server that returns Content-Type: application/json regardless of the Accept header breaks negotiation. If the client requests Accept: application/xml and receives JSON, it either rejects the response or silently parses wrong data. Fix: check the Accept header, return the negotiated type, and return 406 Not Acceptable if no acceptable type is available.

2. URL-based format selection instead of content negotiation. Adding /users.json and /users.xml as separate endpoints duplicates routing, skips the Vary header (breaking CDN cache correctness), and adds URL surface area. HTTP already provides the mechanism: use Accept headers and vary cache responses accordingly.

3. Omitting the Vary header on negotiated responses. A CDN that caches a JSON response without seeing Vary: Accept will serve that cached JSON to all subsequent requests for the same URL — including clients requesting CSV. The Vary header is mandatory whenever response content differs based on request headers.

4. Media-type versioning without a default. If an API requires Accept: application/vnd.myapp.v2+json but provides no fallback for plain Accept: application/json, existing clients that omit the vendor type receive a 406. Always define a default version for generic JSON requests, documented in the API contract.

Details

Media-Type Versioning vs. URL Versioning

Media-type versioning via Accept is the most RESTful but requires CDN and proxy configuration for correct Vary handling. Most public APIs (Stripe, GitHub, Twilio) choose URL versioning for its simplicity and developer experience.

Real-World Case Study: Twilio Content Negotiation

Twilio's REST API accepts both application/json and application/x-www-form-urlencoded on request bodies (via Content-Type) and returns JSON by default. When Twilio added support for CSV exports on call logs, they used content negotiation rather than a separate /export endpoint:

GET /2010-04-01/Accounts/{AccountSid}/Calls.json
Accept: text/csv

Returns a CSV download of the same resource. The Vary: Accept header ensures CDN caches do not mix JSON and CSV responses. This avoided a URL proliferation problem that had plagued the earlier /Calls.json vs /Calls.xml pattern (which duplicated the file-extension suffix hack).

Source

• MDN — Content Negotiation
• RFC 9110 — HTTP Semantics, Section 12
• RFC 6838 — Media Type Specifications and Registration Procedures
• MDN — Accept Header
• MDN — Vary Header

Process

1. Identify which dimensions of content negotiation are needed: format (JSON/XML/CSV), version (vendor type), language, and encoding.
2. Implement Accept header parsing in the server: parse quality factors, find the best match against supported types, return 406 if no match.
3. Set Content-Type in every response to the exact negotiated media type (including vendor type if applicable).
4. Add Vary headers listing all request headers used in negotiation (Accept, Accept-Language, Accept-Encoding).
5. Run harness validate to confirm skill files are well-formed.

Harness Integration

• Type: knowledge -- this skill is a reference document, not a procedural workflow.
• No tools or state -- consumed as context by other skills and agents.
• related_skills: api-versioning-header, api-http-caching, api-openapi-design

Success Criteria

• The server parses the Accept header and returns the best-match media type in Content-Type.
• 406 Not Acceptable is returned when no client-acceptable type is available.
• Every response whose content varies by a negotiated header includes an accurate Vary header listing those headers.
• Media-type versioning uses vendor types (application/vnd.*+json) and documents a default for generic application/json requests.
• Accept-Encoding is honored for compression, with Content-Encoding set in compressed responses.