Sf Data Cloud

Salesforce Data Cloud

Data Cloud is a lakehouse bolted to the Salesforce Platform, not another sObject. It ingests, harmonises and unifies data, then exposes it read-only through SQL, a restricted SOQL subset, and Connect API resources. Everything a developer does here is schema, mapping, ingest, query - never DML. Salesforce rebranded Data Cloud to Data 360 on 14 October 2025; both names appear in the product and the documentation, and the API names still say cdp and ssot.

Agent grounding built on top of this data is skill sf-agentforce-development. Record loading into CRM objects is skill sf-data-management. Callout mechanics and named credentials are skill sf-integration-patterns.

When to use

Data Cloud is the wrong answer when the data is small, must be writable, must be shared by role hierarchy, or must participate in a CRM transaction. Its object-level access is coarse: DMOs in every data space are reachable from Apex in system mode, and there is no field-level security and no record-level access control for them. If a story needs those controls, model it as a custom object and read skill sf-security-model.

Vocabulary and suffixes

In SQL and SOQL statements, data lake objects appear with a __dll suffix (sfmc_email_engagement_click_{EID}__dll) while the object reference documents the DLO object suffix as __dlo. Standard DMOs carry the ssot__ namespace on both object and fields (ssot__Individual__dlm.ssot__Id__c); some REST samples show the unprefixed alias. Resolve the exact name from the org before hardcoding it - see the metadata call in references/query-reference.md.

Core patterns

1. Decide the primary key before anything else

A DLO's primary key is declared per field with primaryIndexOrder on the transport field: an integer starting at 1 that also orders a compound key; a missing value means the field is not part of the key. Ingested rows upsert on that key, and updates are a full replace - patch semantics are not supported. A key that is not stable in the source system produces duplicate rows that identity resolution then has to clean up.

<!-- externalDataTranObjects/Ecom_Order.externalDataTranObject-meta.xml (excerpt) -->
<ExternalDataTranObject xmlns="http://soap.sforce.com/2006/04/metadata">
    <availabilityStatus>Available</availabilityStatus>
    <creationType>Custom</creationType>
    <masterLabel>Ecommerce Order</masterLabel>
    <objectCategory>Engagement</objectCategory>
    <externalDataTranFields>
        <masterLabel>Order Id</masterLabel>
        <creationType>Custom</creationType>
        <datatype>text</datatype>
        <isDataRequired>true</isDataRequired>
        <primaryIndexOrder>1</primaryIndexOrder>
    </externalDataTranFields>
</ExternalDataTranObject>

primaryIndexOrder carries the same meaning on MktDataTranField under MktDataTranObject. Pick whichever transport type the connector produced and set it there.

2. Map DLO to DMO in source-tracked metadata

Object-level mapping is ObjectSourceTargetMap (objectSourceTargetMaps/, API 51.0+); field-level mapping is either the nested FieldSourceTargetMap or the standalone DataSrcDataModelFieldMap (dataSrcDataModelFieldMaps/, API 53.0+). Both are retrievable and deployable.

<?xml version="1.0" encoding="UTF-8"?>
<DataSrcDataModelFieldMap xmlns="http://soap.sforce.com/2006/04/metadata">
    <filterApplied>true</filterApplied>
    <filterOperationType>equals</filterOperationType>
    <filterValue>Active</filterValue>
    <masterLabel>DataSrcDataModel26</masterLabel>
    <sourceField>Account1.LastModifiedDate__c</sourceField>
    <targetField>ssot__Account__dlm.ssot__LastModifiedDate__c</targetField>
    <versionNumber>1.0</versionNumber>
</DataSrcDataModelFieldMap>

Map a record-modified field into the DMO. Without it, the model cannot tell which copy of a duplicated record is newest, and reconciliation rules have nothing to rank on.

3. Ingest: streaming for events, bulk for files

One data stream accepts both interaction patterns. Pick per payload, not per project.

# Bulk: create, upload, close. {instance_url} is the Data Cloud token-exchange instance URL.
curl -X POST "$DC_INSTANCE/api/v1/ingest/jobs" \
  -H "Authorization: Bearer $DC_TOKEN" -H 'Content-Type: application/json' \
  -d '{"object":"Ecom_Order","sourceName":"Ecom_Connector","operation":"upsert"}'

curl -X PUT "$DC_INSTANCE/api/v1/ingest/jobs/$JOB_ID/batches" \
  -H "Authorization: Bearer $DC_TOKEN" -H 'Content-Type: text/csv' --data-binary @orders.csv

curl -X PATCH "$DC_INSTANCE/api/v1/ingest/jobs/$JOB_ID" \
  -H "Authorization: Bearer $DC_TOKEN" -H 'Content-Type: application/json' \
  -d '{"state":"UploadComplete"}'

Ingestion is eventually consistent: allow at least 30 seconds after ingest before the rows are queryable. Request shapes, job states and the CSV contract are in references/ingestion-reference.md.

4. Query from Apex with the sfsqlquery namespace

sfsqlquery is the recommended Apex surface. ConnectApi.CdpQuery remains as a lower-level interface that mirrors the Connect REST endpoints.

public with sharing class LoyaltyLookup {
    public static List<String> topAccountNames() {
        sfsqlquery.SqlRowIterator rows = sfsqlquery.SqlStatement.create(
                'SELECT Id__c, Name__c FROM accounts__dlm LIMIT 200', 'default')
            .withWorkloadName('loyalty-userdashboard')
            .execute();

        List<String> names = new List<String>();
        for (sfsqlquery.Row row : rows) {
            names.add(row.getString('Name__c'));
        }
        return names;
    }
}

The iterator is single-use and submits nothing until iteration begins. withWorkloadName tags the query so Salesforce Support can trace it; without it the framework uses dcsql_row_iterator_workload.

create(sql, dataspace) takes a SQL string and nothing else, so never build a filter by concatenating caller input into it. When the query needs a user-supplied value, a server-side row limit or custom query settings, use the create(ConnectApi.QuerySqlInput, dataspace) overload, which is the documented home for parameterised queries.

5. Page large result sets asynchronously

A synchronous iterator is bounded by the Apex transaction. For anything large, extend sfsqlquery.SqlQueueable; the framework handles submission, status polling, pagination and chaining.

public with sharing class OrderRollupJob extends sfsqlquery.SqlQueueable {
    public OrderRollupJob(sfsqlquery.SqlStatement stmt) { super(stmt); }
    public OrderRollupJob(sfsqlquery.QueryHandle handle) { super(handle); }

    public override void processDataChunk() {
        for (sfsqlquery.Row row : getRows()) {
            OrderRollup.accumulate(row.getString('Id__c'), row.getDecimal('Amount__c'));
        }
    }

    public override void chainNextJob(sfsqlquery.QueryHandle handle) {
        System.enqueueJob(new OrderRollupJob(handle));
    }
}

cancel() inside processDataChunk() stops the chain. getQueryId() returns the server-assigned id; persist it to resume later through sfsqlquery.QueryHandle. Async limits: skill sf-async-apex-patterns and skill sf-governor-limits.

6. Use SOQL only where the subset covers you

SOQL against DMOs, DLOs and the unified profile is supported from API 51.0. It is a subset: no subqueries, no aggregate functions, no date functions, no HAVING. LIMIT defaults to 100 and caps at 2,000 records per call. Relationship traversal is not supported - joins go through SQL.

// A static SOQL query against a DMO is treated as a callout.
List<UnifiedIndividual__dlm> members = [
    SELECT Id, ssot__FirstName__c, ssot__LastName__c, ssot__Email__c
    FROM UnifiedIndividual__dlm
    WHERE ssot__CompanyId__c = :companyId
];

Because it is a callout, pending DML in the same transaction throws UnexpectedException: A callout was unsuccessful because of pending uncommitted work.... Query first, write second, or move the query into a Queueable. Query locators and for loops over DMOs work from API 61.0 (earlier versions returned only the first 201 records); Batch Apex is blocked with QueryLocator but supported with Iterable. Selectivity rules are the same discipline as skill sf-soql-sosl-optimization.

7. Resolve a source record to its unified profile

Identity resolution applies match rules to group records and reconciliation rules to pick the best value per attribute, writing link tables that leave the source rows intact. Three hops get you from a source id to unified contact points.

-- 1. source record id -> unified record id
SELECT UnifiedRecordId__c FROM IndividualIdentityLink__dlm WHERE SourceRecordID__c = '{sourceId}' LIMIT 100
-- 2. unified individual
SELECT FirstName__c, LastName__c FROM UnifiedIndividual__dlm WHERE Id__c = '{unifiedId}' LIMIT 100
-- 3. unified contact point
SELECT EmailAddress__c FROM UnifiedContactPointEmail__dlm WHERE PartyId__c = '{unifiedId}' LIMIT 100

Joining non-unified DMOs needs the fully qualified key KQ_Id__c as well as the business key, compared with IS NOT DISTINCT FROM because it can be null. Unified DMOs have no KQ_Id__c - there are no duplicate record ids to disambiguate.

8. Surface it in the UI through an Apex bridge, and in Flow through cdp actions

There is no Data Cloud-specific LWC wire adapter in the Lightning Web Components reference [unverified]; the documented client path is an @AuraEnabled(cacheable=true) Apex method that runs sfsqlquery or ConnectApi.CdpQuery in the running user's context. Flow gets standard invocable actions instead - cdpGetDataGraph (API 61.0+), cdpGetDataGraphByLookup (63.0+), cdpGetDataGraphMetadata (64.0+), cdpPublishCalculatedInsight, cdpPublishSegment, cdpRefreshDataStream, cdpValidateSegmentMember (all 60.0+), cdpRunIdentityResolution (57.0+) and dataCloudIngestionApi (61.0+). Full wiring: references/apex-lwc-flow.md.

9. Test without an org

Two independent mocking frameworks, one per query surface.

@IsTest
static void returnsMockedRow() {
    sfsqlquery.SqlTester.clearMocks();
    ConnectApi.QuerySqlMetadataItem col = new ConnectApi.QuerySqlMetadataItem();
    col.name = 'Name__c';
    col.type = ConnectApi.TypeEnum.VARCHAR;
    sfsqlquery.SqlTester.setMockMetadata(new List<ConnectApi.QuerySqlMetadataItem>{ col });

    ConnectApi.QuerySqlRow mockRow = new ConnectApi.QuerySqlRow();
    mockRow.rowData = new List<Object>{ 'Acme' };
    sfsqlquery.SqlTester.setMockRows(new List<ConnectApi.QuerySqlRow>{ mockRow });

    sfsqlquery.SqlRowIterator it = sfsqlquery.SqlStatement.create(
        'SELECT Name__c FROM accounts__dlm', 'default').execute();
    Assert.areEqual('Acme', it.next().getString('Name__c'));
}

Data modelling: DLO to DMO mapping and key choice

1. Name the grain. One DLO row equals one source record. If the source emits a change event per field, the grain is the event, not the entity, and the key must include the event id.
2. Pick a key the source owns. primaryIndexOrder starting at 1; compound keys number the attributes in order. Never key on an ingest timestamp or a hash of the whole row.
3. Declare the extract method. DataStreamDefinition.dataExtractMethods takes FULL_REFRESH, DATETIME_CDC or NUMERIC_CDC; DATETIME_CDC and NUMERIC_CDC need dataExtractField set to the transport field that carries the watermark.
4. Map to the standard DMO first. The C360 model ships over 300 objects. Map Individual, ContactPointEmail, ContactPointPhone and ContactPointAddress before inventing a custom DMO - identity resolution, segments and data graphs are written against the standard shapes.
5. Map the record-modified field. Reconciliation cannot rank source copies without it.
6. Use sourceFormula for cheap harmonisation. isSourceFormula plus sourceFormula covers concatenation, date functions and constants at map time, avoiding a second DLO.
7. Filter at the mapping. filterApplied / filterOperationType / filterValue keep rows out of the DMO instead of filtering in every query.

Every metadata type, folder, file suffix and field is tabulated in references/object-model.md.

Anti-patterns

Verification

Retrieve and deploy the Data Cloud metadata the data kit manifest names:

sf project retrieve start --manifest package.xml --target-org vf-dev
sf project deploy start --manifest package.xml --target-org vf-dev

# One type at a time when debugging a mapping
sf project deploy start \
  --metadata DataStreamDefinition:Ecom_Order_Stream \
  --metadata ObjectSourceTargetMap:Ecom_Order_To_SalesOrder \
  --target-org vf-dev

Prove the data is there. sf data query runs the SOQL subset; sf api request rest reaches the Connect REST query resource with the CLI's authenticated session. v67.0 comes from config/vibe-force.defaults.json (apiVersion); the Salesforce sample shows v64.0.

sf data query \
  --query "SELECT Id__c, FirstName__c FROM UnifiedIndividual__dlm LIMIT 10" \
  --target-org vf-int

sf api request rest "/services/data/v67.0/ssot/query-sql" \
  --method POST --header 'Content-Type: application/json' \
  --body '{"sql":"SELECT COUNT(*) FROM ssot__Individual__dlm"}' \
  --include --target-org vf-int

Exercise the Apex surface directly before trusting a controller:

sf apex run --file scripts/apex/dc-probe.apex --target-org vf-int

Project gates, in the usual order:

node "${CLAUDE_PLUGIN_ROOT}/scripts/checks/vf-check.mjs" static --changed
node "${CLAUDE_PLUGIN_ROOT}/scripts/checks/vf-check.mjs" local --changed
node "${CLAUDE_PLUGIN_ROOT}/scripts/checks/vf-check.mjs" apex \
  --target-org vf-dev --tests LoyaltyLookupTest,OrderRollupJobTest
node "${CLAUDE_PLUGIN_ROOT}/scripts/checks/vf-check.mjs" deploy-validate --target-org vf-int
node "${CLAUDE_PLUGIN_ROOT}/scripts/checks/vf-check.mjs" smoke --target-org vf-int

Never point an ingest smoke probe at a production Data Cloud tenant: ingested rows cannot be deleted by rolling back a deploy, and every query consumes credits.

References

• references/object-model.md - DLO, DMO, unified, UDLO/UDMO, CIO and data graph terminology; every Data Cloud metadata type with folder, suffix and API version; mapping and key-selection rules; packaging support.
• references/ingestion-reference.md - Ingestion API bulk job lifecycle and streaming endpoints, CSV and date contracts, connector options, OAuth scopes and the two-step token exchange, identity implications of each path.
• references/query-reference.md - Data 360 SQL surface, Query API v3 and the Connect REST ssot/query-sql resource, the ConnectApi.CdpQuery method catalogue, the SOQL subset, worked queries with sample output.
• references/apex-lwc-flow.md - sfsqlquery class reference, Apex security model for DMOs, the LWC Apex bridge, Flow invocable actions and data actions, and the three test-mocking frameworks.
• references/limits-and-quotas.md - every number with the page it came from.

Official documentation used:

• Query Data in Data 360 - https://developer.salesforce.com/docs/data/data-cloud-query-guide/guide/query-guide-get-started.html
• Query Data 360 Data using Query API - https://developer.salesforce.com/docs/data/data-cloud-query-guide/references/data-cloud-query-api-reference/c360a-api-queryservices-overview.html
• Query Data 360 Data with Apex - https://developer.salesforce.com/docs/data/data-cloud-query-guide/guide/dc-apex-query.html
• Data 360 In Apex - https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/DataCloudInApex.htm
• Mock SOQL Tests for Data 360 DMOs - https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/MockSOQLTestsForDMOs.htm
• Data Cloud Query Profile Parameters (SOQL) - https://developer.salesforce.com/docs/atlas.en-us.api_rest.meta/api_rest/resources_cdp_query.htm
• Salesforce Data Cloud Objects - https://developer.salesforce.com/docs/atlas.en-us.object_reference.meta/object_reference/sforce_api_concepts_data_cloud_objects.htm
• Data 360 Metadata Types - https://developer.salesforce.com/docs/atlas.en-us.api_meta.meta/api_meta/meta_data_cloud_types.htm
• Get Started with Ingestion API - https://developer.salesforce.com/docs/data/data-cloud-int/references/data-cloud-ingestionapi-ref/c360-a-api-get-started.html
• Custom App Development - https://developer.salesforce.com/docs/data/data-cloud-dev/guide/custom-app-dev.html
• Data 360 Architecture - https://developer.salesforce.com/docs/data/data-cloud-dev/guide/dc-architecture.html