Change Data Capture Integration

Change Data Capture Integration

Use this skill when an external system needs to receive near-real-time Salesforce record changes without polling. CDC publishes structured change events to a durable event bus that external subscribers consume via Pub/Sub API (gRPC) or the CometD Streaming API. This skill covers the full lifecycle: entity selection, channel topology, event structure, replay, gap handling, and subscriber operations.

Before Starting

Gather this context before working on anything in this domain:

• Which objects need CDC? Confirm whether they are standard objects (count toward the 5-entity default limit), custom objects, or both.
• What subscriber technology is in use? Pub/Sub API (gRPC) is the modern preferred path; CometD is the legacy path still supported for existing middleware.
• What Edition is the org on? Delivery allocations differ: 50,000/day (Performance/Unlimited), 25,000/day (Enterprise), 10,000/day (Developer). The CDC add-on removes entity limits and moves delivery to a monthly model.
• Replay requirement: Does the subscriber need durable replay (full 72-hour window) or tip-only? Is storing and recovering the replay ID in external state required?
• Is Apex the subscriber? If yes, stop — use apex/platform-events-apex instead. This skill covers external (non-Apex) subscribers only.

Core Concepts

Change Event Structure

Every CDC message wraps a ChangeEventHeader object alongside the record fields. Key header fields:

changedFields, nulledFields, and diffFields are not available in CometD clients — they require Pub/Sub API or Apex triggers. This is a critical architectural difference.

Enabling CDC and Entity Selection

CDC is enabled per object in Setup > Integrations > Change Data Capture. This page controls entity selections for the default ChangeEvents standard channel only. Custom channels are managed separately via PlatformEventChannel and PlatformEventChannelMember metadata types.

Default entity selection limit is 5 objects (standard + custom combined) across all channels and all editions. If the same entity is selected in multiple channels, it counts once. Exceeding 5 requires purchasing the Change Data Capture add-on.

Custom channels are created with Metadata API (PlatformEventChannel type) or Tooling API and support up to 100 custom channels. Selections made in PlatformEventChannelMember for custom channels are NOT reflected in the Setup UI — query Tooling API to audit selections.

Change Event Channels

Two channel types exist:

• Default standard channel — /data/ChangeEvents — delivers events for all entities selected via Setup UI. All subscribers on this channel see events for all selected objects.
• Standard per-entity channels — /data/AccountChangeEvent, /data/Contact__ChangeEvent — pre-built channels delivering events for one standard or custom object respectively. Custom objects use <ObjectName>__ChangeEvent.
• Custom channels — /data/<ChannelName>__chn — subscriber-isolated channels; each subscriber receives events only for entities assigned to that channel. Use custom channels when different downstream systems need different entity subsets, or when event enrichment (additional fields beyond changed fields) is required.

Custom channels support event enrichment: you can add fields to the change event payload that were not part of the originating DML change, allowing downstream systems to avoid follow-up queries.

External Subscription Protocols

Pub/Sub API (gRPC) — preferred for new integrations:

• Subscribe by topic name, e.g., /data/AccountChangeEvent
• Supports FetchRequest with replay ID for durable subscription
• Returns changedFields, nulledFields, diffFields in the header — richer than CometD
• Available from API version 54.0+

CometD (Streaming API) — legacy, still supported:

• Subscribe via long-polling HTTP handshake on /cometd/<version>
• Use the EMP Connector (open-source Java library from Salesforce) or MuleSoft CDC connector
• Replay controlled via {"replay": <replayId>} in the subscription request
• Does NOT surface changedFields, nulledFields, or diffFields in the message body

Replay IDs and Durable Subscription

Change events are retained for 72 hours (3 days). Subscribers can replay events within this window using replay IDs:

Durable subscription requires the subscriber to persist the last successfully processed replay ID to external state (database, cache, or file). On reconnect, pass the stored replay ID to resume without gaps.

Replay ID storage responsibility lies entirely with the subscriber. Salesforce does not maintain per-subscriber cursors — if a subscriber reconnects without a stored replay ID, it can only start at tip (-1) or replay all retained events (-2).

Gap Events And Overflow Events — Two Different Failures

Gap events (GAP_CREATE, GAP_UPDATE, GAP_DELETE, GAP_UNDELETE) are sent when Salesforce cannot generate a full change event: "The change event size exceeds the maximum 1 MB message size", certain field-type conversions, internal errors, or "Changes that occur outside the application server transaction and are applied directly in the database." They carry real recordIds but no field data — detect the GAP_ prefix, re-query current state by recordIds through REST, and reconcile against commitTimestamp so a newer non-gap event is not overwritten.

GAP_OVERFLOW is not one of these and must not share the code path. It is emitted when a single transaction exceeds 100,000 changes: "The first 100,000 changes generate change events. The set of changes beyond that amount generates one overflow event for each entity type." The weighting is asymmetric — "A record creation, deletion, or undeletion counts as one change toward the threshold", but "In a record update, each field change counts toward the overflow threshold", so a 3-field update across 34,000 records crosses the line. Typical causes: a cascade delete of accounts with many child opportunities and activities, or a recurring event with hundreds of occurrences and attendees.

The overflow payload carries nothing usable: all data fields are null, changedFields/nulledFields/diffFields are empty, and "The record ID for the change is always set to 000000000000000AAA, which is the empty record ID." So there is no per-record recovery path — unlike the other GAP_ types you cannot re-query what you missed. The fix is upstream (batch the originating operation below 100,000 changes) or a scheduled reconciliation job, never consumer scaling or replay tuning.

Common Patterns

Pattern 1: Single External Subscriber — Default Channel with Durable Replay

When to use: One middleware system (e.g., MuleSoft, Boomi) needs all CDC changes for a small set of objects.

How it works:

1. Enable the target objects in Setup > Integrations > Change Data Capture (max 5 entities without add-on).
2. Subscribe to /data/ChangeEvents or the per-entity channel /data/AccountChangeEvent.
3. Use replay ID -2 on first connection to catch up, then switch to persisting the last processed replay ID.
4. Store the replay ID after each successful batch commit to external state.
5. On gap event, fetch current record via REST API using the record ID in the header.

Why not just polling: Polling via REST API misses field-level delta information and cannot detect deletes or undeletes. CDC provides explicit changeType and the exact set of changed fields.

Pattern 2: Multiple Subscribers with Isolated Custom Channels

When to use: Two or more downstream systems need CDC but for different entity subsets, or with different event enrichment needs.

How it works:

1. Create a custom PlatformEventChannel per subscriber group via Metadata API.
2. Add PlatformEventChannelMember records for the relevant entities per channel.
3. Configure event enrichment on channel members where subscribers need additional fields beyond the changed ones.
4. Each subscriber connects to its specific channel path, e.g., /data/ERP_Sync__chn.
5. Each subscriber manages its own replay ID independently.

Why not the default channel: it broadcasts all selected entities to all subscribers; custom channels filter server-side before delivery.

Pattern 3: CDC → Kafka Bridge for Fan-Out

When to use: Multiple downstream consumers exist and Kafka is the enterprise event backbone.

How it works:

1. One dedicated integration layer (MuleSoft CDC connector, custom Pub/Sub API gRPC client, or Kafka Connect Salesforce connector) subscribes to Salesforce CDC via Pub/Sub API.
2. That layer publishes to Kafka topics mapped per entity (account-changes, opportunity-changes).
3. Kafka consumers subscribe independently; the CDC-to-Kafka bridge manages Salesforce replay ID in its own persistent store.
4. Kafka provides its own offset management and fan-out — Salesforce event delivery allocation is consumed only by the single bridge subscriber.

Why not multiple Salesforce subscribers: each additional external subscriber counts independently against the delivery allocation; one bridge protects it.

Decision Guidance

Recommended Workflow

1. Confirm scope and limits — Count the entities needed. Verify org Edition and whether the CDC add-on is licensed. Confirm subscriber technology (Pub/Sub API or CometD).
2. Enable entities and configure channels — For up to 5 entities, enable via Setup > Change Data Capture. For custom channels or more entities (with add-on), create PlatformEventChannel and PlatformEventChannelMember via Metadata API deployment.
3. Design replay ID strategy — Decide where the subscriber persists the last replay ID (external DB, Redis, config file). Decide initial replay behavior: -2 for full catch-up or -1 for tip-only.
4. Implement subscriber — For Pub/Sub API: implement a gRPC Subscribe call with FetchRequest carrying the stored replay ID. For CometD: configure the EMP Connector or middleware connector with the channel path and replay extension.
5. Implement gap event handling — Add conditional logic that detects GAP_ prefixed changeType values, marks records dirty, and fetches current state from REST API using recordIds.
6. Test replay and reconnection — Validate reconnect behavior by stopping the subscriber, waiting, and replaying using the stored replay ID. Confirm the subscriber resumes from the correct position.
7. Monitor delivery allocation usage — Check Setup > Event Monitoring or REST API to track daily delivery usage against the org's allocation. Alert before the limit is reached.

Review Checklist

• All target entities are enabled and the 5-entity limit (or add-on) is accounted for.
• Channel topology is confirmed: default channel vs per-entity channel vs custom channel.
• Subscriber uses Pub/Sub API (preferred) or has a documented reason for CometD.
• Replay ID is persisted to durable external state after each processed batch.
• Gap event handling is implemented and tested with a GAP_CREATE or GAP_UPDATE scenario.
• Initial replay strategy is documented (-1 tip or -2 full catch-up).
• Event delivery allocation usage is monitored with alerting before the daily cap.
• Subscriber reconnect behavior under the 72-hour window is tested.
• If multiple subscribers exist, evaluate custom channels to prevent shared delivery allocation exhaustion.

Salesforce-Specific Gotchas

1. changedFields is absent in CometD — changedFields, nulledFields and diffFields come only from Pub/Sub API or Apex triggers. CometD delivers the full field set, so diff logic there must compare against stored state.

2. Gap events contain no changed field data — the subscriber gets only recordIds and the gap changeType. Check the GAP_ prefix before touching field values, and branch GAP_OVERFLOW separately (see Core Concepts) — it carries no usable record IDs either.

3. Entity selections in custom channels are invisible in Setup UI — the CDC Setup page reflects only the default ChangeEvents channel, so a Setup-based audit undercounts event volume. Use SELECT QualifiedApiName FROM PlatformEventChannelMember (Tooling API).

4. Event delivery allocation is per-subscriber, cumulative — two subscribers on the same 20,000-event stream consume 40,000 of the allocation. See references/gotchas.md Gotcha 4 for which subscriber types consume it and the exact disconnect strings.

5. 72-hour retention is hard — a subscriber offline longer than that has a gap no replay can close. Recovery is a full Bulk API re-sync followed by re-subscribing at tip.

6. Formula fields and roll-ups do not generate CDC events — CDC captures explicit DML on stored values. Formula recalculations and roll-up updates driven by child records produce no parent change event, so formula state tracked via CDC silently drifts.

Output Artifacts

Related Skills

• apex/platform-events-apex — Use when Apex triggers are the CDC subscriber, or when the integration publishes/consumes custom Platform Events.
• integration/platform-events-integration — Use for custom business events (not row-change CDC). Covers Pub/Sub API and CometD for Platform Events.
• architect/platform-selection-guidance — Use when deciding between CDC, Platform Events, and outbound messaging for an integration pattern.
• data/external-data-and-big-objects — Use when data volume considerations affect CDC strategy or when Bulk API is needed for re-sync after gap events.