# API Webhook Design > API Webhook Design - Skill: `intense-visions/api-webhook-design` (Agent Skill, multi-file: 2 files) - Install (CLI): `npx skillmds@latest add intense-visions/api-webhook-design` - Raw SKILL.md: https://api.skillmd.com/api/skills/intense-visions/api-webhook-design/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Integrations & APIs - Author: Intense-Visions (https://skillmd.com/u/intense-visions) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/intense-visions/api-webhook-design --- # API Webhook Design > WEBHOOKS ARE A PUSH-BASED CONTRACT — REGISTRATION, PAYLOAD SCHEMA, DELIVERY GUARANTEES, AND RETRY POLICIES ARE ALL CONSUMER-FACING COMMITMENTS THAT DETERMINE WHETHER INTEGRATIONS REMAIN RELIABLE UNDER PARTIAL FAILURES, AND DESIGNING THESE PROPERTIES EXPLICITLY UPFRONT PREVENTS AN AD-HOC SYSTEM THAT FAILS SILENTLY AT THE WORST POSSIBLE MOMENT. ## When to Use - Designing the webhook system for a new platform API that needs to notify external consumers of events - Auditing an existing webhook implementation for missing retry logic, payload versioning, or delivery guarantees - Deciding between at-least-once and exactly-once delivery and documenting the implications for consumers - Implementing fan-out from a single internal event to multiple registered subscriber endpoints - Writing the webhook reference section of a developer portal or API style guide - Responding to a consumer report of missed webhook deliveries or ordering anomalies - Adding webhook support to an existing REST API without disrupting current consumers ## Instructions ### Key Concepts 1. **Webhook registration and lifecycle** — Consumers register a publicly reachable HTTPS endpoint (the "webhook URL") with the provider, along with an event type filter (e.g., `payment.succeeded`, `order.created`). The registration should store: the endpoint URL, the event types subscribed, a shared signing secret, and metadata for auditing (created by, created at). Provide a management API or dashboard UI to create, list, test, pause, and delete registrations. GitHub's webhook management API (`POST /repos/{owner}/{repo}/hooks`) is the canonical reference: consumers can subscribe per-repository, per-organization, or at the App level. 2. **Payload design and versioning** — Every webhook payload must include: an `id` (unique delivery ID for deduplication), an `event` field (event type string), a `created` timestamp (ISO 8601), an `api_version` field (the schema version that generated the payload), and a `data` object (the event subject). Never omit the `api_version` — consumers need it to route the payload to the correct handler when you introduce breaking schema changes. Stripe's payload envelope is the industry standard: `{ "id": "evt_xxx", "object": "event", "api_version": "2024-04-10", "created": 1712800000, "type": "payment_intent.succeeded", "data": { "object": { ... } } }`. 3. **At-least-once delivery and consumer idempotency** — Webhook systems guarantee at-least-once delivery by design: a delivery is considered successful only when the consumer returns a 2xx response within the timeout window (typically 5–30 seconds). Any non-2xx response or timeout triggers a retry. Consequently, consumers will occasionally receive the same event more than once. Document this guarantee explicitly, and instruct consumers to use the delivery `id` field as an idempotency key to deduplicate retried deliveries. Never promise exactly-once delivery unless you have a distributed transaction mechanism backing it. 4. **Retry policy with exponential backoff** — Failed deliveries (non-2xx or timeout) must be retried with exponential backoff: attempt 1 at T+1 min, attempt 2 at T+5 min, attempt 3 at T+30 min, attempt 4 at T+2 h, attempt 5 at T+5 h. After the maximum retry count (commonly 5–10 attempts over 24–72 hours), mark the delivery as failed and optionally surface an alert to the webhook owner. GitHub retries for 3 days with exponential backoff. Stripe retries for 3 days across up to 17 attempts with escalating intervals. 5. **Ordering guarantees and fan-out** — Webhooks are not guaranteed to arrive in the order events occurred. Network and retry interactions can invert order. Tell consumers to use the `created` timestamp to establish event ordering rather than arrival order. For fan-out (one internal event dispatched to N registered endpoints), use an async queue per subscriber so a slow or failing subscriber cannot block delivery to other subscribers. Each subscriber's queue is independent; backpressure on one does not affect others. 6. **Endpoint health and circuit breaking** — Track consecutive delivery failures per endpoint. After a configurable threshold (e.g., 5 consecutive failures or >50% failure rate over 1 hour), automatically pause the endpoint and notify the webhook owner. A paused endpoint stops receiving new deliveries; the owner must investigate and re-enable it. Sending to a consistently failing endpoint wastes resources and can cause queue buildup. GitHub auto-disables webhooks that fail for 7 consecutive days. ### Worked Example **GitHub Webhooks — repository push event** Register a webhook on a repository: ```http POST /repos/acme/payments/hooks Authorization: Bearer ghp_xxx Content-Type: application/json { "name": "web", "active": true, "events": ["push", "pull_request"], "config": { "url": "https://ci.acme.com/hooks/github", "content_type": "json", "secret": "s3cr3t_abc123", "insecure_ssl": "0" } } → HTTP/1.1 201 Created { "id": 12345678, "type": "Repository", "active": true, "events": ["push", "pull_request"], "config": { "url": "https://ci.acme.com/hooks/github", "content_type": "json" }, "created_at": "2024-04-10T12:00:00Z" } ``` Incoming delivery payload: ```json { "ref": "refs/heads/main", "before": "abc123", "after": "def456", "repository": { "id": 99887766, "name": "payments", "full_name": "acme/payments" }, "pusher": { "name": "alice", "email": "alice@acme.com" }, "commits": [{ "id": "def456", "message": "fix: handle nil pointer in checkout" }] } ``` Delivery headers: ```http X-GitHub-Event: push X-GitHub-Delivery: 72d3162e-cc78-11e3-81ab-4c9367dc0958 X-Hub-Signature-256: sha256=abc123... ``` Consumer responds with `200 OK` and an empty body — GitHub marks the delivery as successful. The `X-GitHub-Delivery` UUID is the idempotency key consumers use to deduplicate retries. **Fan-out architecture:** ``` Internal Event Bus │ ▼ Dispatcher Service │ ┌───┴───────────────────────┐ ▼ ▼ Queue: endpoint-A Queue: endpoint-B │ │ ▼ ▼ Worker → POST /hook/A Worker → POST /hook/B ``` Each subscriber queue is independent. A 30-second timeout on endpoint-B does not delay endpoint-A's delivery. ### Anti-Patterns 1. **Synchronous delivery in the request path.** Firing the webhook HTTP call synchronously during the transaction that generated the event couples event delivery latency to the consumer's endpoint response time. A slow consumer endpoint (or network partition) can cause your transaction to time out or roll back. Always dispatch webhook deliveries asynchronously via a queue after the originating transaction commits. 2. **No delivery `id` in the payload.** Without a unique delivery identifier, consumers cannot implement idempotency. Every retry looks identical to the first delivery. The consumer has no reliable way to detect or handle duplicates. Include a UUID delivery ID in every payload regardless of whether the consumer currently uses it. 3. **Fixed retry interval (no backoff).** Retrying at a fixed 1-minute interval after a consumer endpoint goes down hammers the endpoint with traffic exactly when it is least able to respond — during an outage or deployment. Exponential backoff with jitter reduces thundering-herd behavior and gives the consumer time to recover before the next attempt. 4. **Delivering to HTTP endpoints.** Allowing webhook registrations to plain HTTP (non-TLS) URLs exposes the payload — which may contain sensitive business data — to network interception. Require HTTPS for all webhook URLs and reject registrations that specify HTTP. Surface a clear validation error: `"config.url must use HTTPS"`. 5. **No test delivery mechanism.** Consumers integrating a webhook system need a way to trigger a test delivery without generating a real event. Without it, they must create real objects in production to test their handler. Provide a `POST /hooks/{id}/test` endpoint that sends a sample payload and returns the delivery result. ## Details ### Payload Envelope Standardization A consistent payload envelope across all event types reduces the cognitive load on consumers who handle multiple event types. The envelope fields (`id`, `event`, `created`, `api_version`, `data`) are stable; only `data` varies by event type. This pattern enables consumers to write generic middleware that extracts the envelope before routing to event-specific handlers. Consider publishing a JSON Schema or OpenAPI schema for each event type's `data` payload. Stripe publishes event object schemas in their OpenAPI spec; consumers can generate typed client models from them. When you evolve a payload (rename a field, add a required property), bump `api_version` and publish a changelog. ### Real-World Case Study: Stripe Webhook Reliability Stripe processes billions of webhook deliveries per month. Their published reliability figures show that >99.9% of events are delivered within 30 seconds on the first attempt under normal conditions. The remaining <0.1% are retried across up to 17 attempts over 72 hours. Stripe's key design decisions that achieve this: 1. Deliveries are enqueued after the originating database write commits — no synchronous delivery. 2. Per-endpoint queues prevent a single slow consumer from causing head-of-line blocking. 3. The `api_version` field in every payload is the API version at the time the event was created, not the current API version — consumers always receive the schema they integrated against. 4. Stripe's webhook dashboard shows delivery attempt history, response codes, response bodies, and the ability to replay any delivery — dramatically reducing consumer debugging time. Outcome: Stripe's webhook retry architecture became the reference design for Twilio, GitHub, and most SaaS platforms that launched webhook support after 2015. ## Source - [Stripe Webhooks Documentation](https://stripe.com/docs/webhooks) - [GitHub Webhooks Documentation](https://docs.github.com/en/webhooks) - [webhooks.fyi — Webhook Standards Reference](https://webhooks.fyi) - [Google AIP-151 — Long-running Operations](https://aip.dev/151) - [Standard Webhooks Specification](https://www.standardwebhooks.com) ## Process 1. Define the event catalog (event type strings, payload schemas, `api_version`) before implementing delivery infrastructure — consumers build against the schema. 2. Implement delivery as an async queue-based system; commit the event to the queue after the originating transaction commits, never inline. 3. Set per-subscriber independent queues for fan-out; configure retry policy with exponential backoff and a maximum attempt count (5–10 attempts over 24–72 hours). 4. Expose a `POST /hooks/{id}/test` endpoint and a delivery history dashboard so consumers can verify integration without generating real events. 5. Run `harness validate` to confirm skill files are well-formed and related skills are correctly cross-referenced. ## Harness Integration - **Type:** knowledge — this skill is a reference document, not a procedural workflow. - **No tools or state** — consumed as context by other skills and agents. - **related_skills:** api-webhook-security, events-webhooks-pattern, api-idempotency-keys, api-long-running-operations ## Success Criteria - Every webhook payload includes a unique delivery `id`, an `event` type string, a `created` timestamp, and an `api_version` field. - Deliveries are dispatched asynchronously after the originating transaction commits; no synchronous delivery in the request path. - The retry policy uses exponential backoff with a defined maximum attempt count and total retry window. - Fan-out to multiple subscribers uses per-subscriber independent queues so one failing subscriber cannot block others. - A test delivery mechanism (`POST /hooks/{id}/test`) is available for consumers to verify handler integration without generating real events.