Webhook Provider
Emit events to customer endpoints reliably, securely, and observably. Consuming webhooks is covered by idempotent-financial-workflows. Signing formulas and event schema template are in REFERENCE.md.
Payload Signing
- Sign:
HMAC-SHA256(secret, "v1:" + timestamp + ":" + raw_body) - Header:
X-Webhook-Signature: t=<unix_ts>,v1=<hex_digest> - Timestamp in the signed string prevents replay attacks.
- Dual-secret verification during rotation (old + new accepted for 24 h).
- Ed25519 option: publish a public key — customer verifies without a shared secret.
Delivery & Retry
- Success = any 2xx within timeout (30 s recommended).
- Retry on 4xx (except 410), 5xx, timeout, network error.
- Backoff + jitter: 30 s → 5 min → 30 min → 2 h → 8 h → 24 h.
- After max retries: dead-letter queue + dashboard + optional alert.
- On 410: auto-disable the endpoint. Never retry 2xx.
Endpoint Health & Circuit Breaker
Track success/failure ratio over a sliding window (last 100 deliveries). Above threshold (e.g. >50% for 1 h): disable, notify customer, require explicit re-enable. On re-enable, replay dead-lettered events (bounded, customer-configurable window).
Ordering & Idempotency
- At-least-once delivery; include
idempotency_keyin every payload — customers must deduplicate. - Per-resource delivery queue keyed by resource ID; cross-resource order not guaranteed (document this).
- Monotonic
sequenceper resource so consumers detect gaps.
Event Schema
Fields: id, type (e.g. invoice.paid), created_at (ISO 8601 UTC), api_version, idempotency_key, data. Emit a new event type for breaking changes — never mutate an existing schema. Publish an AsyncAPI/OpenAPI catalog (template in REFERENCE.md).
Fan-Out
Event → internal bus (Kafka, SQS, Postgres LISTEN/NOTIFY) → delivery worker per subscription. Never deliver synchronously from the originating handler. Outbox: write the event in the same DB transaction as the state change; a relay publishes to the bus.
Customer Controls
- Register endpoint URL; select subscribed event types.
- Delivery log with request + response bodies (retain 72 h); manual replay of any event.
- Test-mode
pingon registration; pause/resume without losing queued events.
Testing
- Unit: HMAC sign/verify; retry backoff math.
- Integration: tunnel (ngrok, Hookdeck); emit a test event; assert signature verifies.
- Failure: 500 → retry fires; 410 → endpoint disables.
- Replay: disable, emit events, re-enable, assert ordered delivery of all missed events.
Guardrails
- Never log raw bodies beyond retention (PII risk); never deliver synchronously (latency cascades).
- Never reuse idempotency keys across event types; document retry and ordering guarantees explicitly.