Continuous Integration Testing

Continuous Integration Testing

Use this skill when Apex tests must run as part of an automated CI/CD pipeline rather than manually through the Developer Console or Setup UI. The objective is a pipeline configuration that executes the right test level, collects accurate coverage numbers, produces machine-readable results, and fails the build when coverage or test outcomes do not meet deployment thresholds.

Before Starting

Gather this context before working on anything in this domain:

• Which CI platform is in use? The sf CLI commands are the same everywhere, but authentication (JWT, web login, auth URL) and artifact handling differ across GitHub Actions, GitLab CI, Jenkins, and others.
• Does the target org contain managed packages? Managed package tests inflate failure risk under RunAllTestsInOrg and are skipped under RunLocalTests. The wrong choice causes either false failures or silently skipped coverage.
• What is the minimum coverage gate? Salesforce enforces 75% org-wide for production deployments, but many teams enforce higher thresholds (80-85%) or per-class minimums that Salesforce does not natively enforce.

Core Concepts

Test Levels Control What Runs

The sf project deploy start and sf apex run test commands accept a --test-level flag with four values:

The critical nuance: RunSpecifiedTests applies coverage requirements per-component in the package, not org-wide. A single under-covered class in your changeset will fail deployment even if the org is at 90% overall.

Coverage Collection Is Not Coverage Enforcement

The --code-coverage flag on sf apex run test instructs the platform to collect coverage data and include it in the result payload. It does not enforce any threshold. Your pipeline script must parse the coverage result and fail the build if coverage is below the team standard. Salesforce only enforces the 75% threshold during actual deployment, not during standalone test runs.

JUnit Output for CI Report Ingestion

Most CI platforms can parse JUnit XML to display test results in their UI. Use --result-format junit on sf apex run test to produce this format. The output file can be declared as a test artifact in GitHub Actions, GitLab CI, or Jenkins to get per-test pass/fail reporting in the build dashboard.

--result-format accepts four values, not just junit: human (the default terminal-readable table), tap (Test Anything Protocol), junit (the XML most dashboards ingest), and json (the full machine-readable payload you parse in a coverage-gate script). Pick junit for dashboard artifacts and json when the pipeline script needs to read coverage numbers programmatically.

For per-test-method coverage rather than only aggregate per-class numbers, add --detailed-coverage alongside --code-coverage. This surfaces which specific test method exercised each class, which is useful when diagnosing why a class sits just under the gate — but it does not change the enforcement behavior, only the level of detail in the report.

Asynchronous Test Execution and Polling

By default, sf apex run test with --synchronous runs tests in a single synchronous request, which is limited to a smaller set of tests. Without --synchronous, tests run asynchronously and the CLI polls for completion when --wait is specified. A known platform behavior: combining --code-coverage with --wait on asynchronous runs can return 0% coverage. The workaround is to let the run complete, then retrieve results separately with sf apex get test --test-run-id <id> --code-coverage.

The UI Reporting Surfaces the Pipeline Replaces

Before wiring coverage into CI, know the two in-org UI surfaces the same numbers come from — they are where developers cross-check a pipeline result or investigate a coverage drop by hand:

• Setup — Application Test Execution. Enter Application in Quick Find, select Application Test Execution, click Select Tests, choose the test classes, and click Run. Clicking View Test History on this page shows every test run in the org — not just the ones the current user launched — so it is the shared record of test activity across the team. One constraint matters for anyone relying on it: results are retained for only 30 days after they finish (unless cleared sooner), so it is not a long-term audit trail — treat it as a shared cross-check, not a system of record.
• Developer Console — Tests tab. The Tests tab has an Overall Code Coverage panel reporting the coverage percentage for every Apex class in the org that has been included in a test run, and it highlights covered versus uncovered lines directly in the class editor. The gotcha: this line-level highlighting goes stale the instant a class is edited — the highlighting dims to signal the coverage is no longer valid, and you must rerun the tests to get accurate numbers again. Never trust dimmed coverage highlighting as a current signal.

These UI surfaces do not enforce anything beyond the platform's own 75% deploy gate; they are for inspection. The pipeline is still where enforcement lives.

Common Patterns

Pattern: Two-Stage Deploy with Validation Then Quick-Deploy

When to use: Production deployments where you want a CI validation gate before the actual deployment window.

How it works:

1. Run sf project deploy validate --test-level RunLocalTests --target-org prod during CI. This runs all local tests and validates coverage without actually deploying.
2. Capture the validation ID from the output.
3. During the deployment window, run sf project deploy quick --job-id <validation-id> --target-org prod to deploy without re-running tests.

Why not the alternative: Running sf project deploy start directly means tests and deployment happen atomically. If CI validates in the morning and deployment is approved in the afternoon, validation-then-quick-deploy avoids re-running the full test suite.

Pattern: RunSpecifiedTests with Explicit Coverage Gate

When to use: Large orgs where RunLocalTests takes too long (30+ minutes) and the team wants fast CI feedback.

How it works:

1. Determine which test classes cover the changed Apex classes. Many teams maintain a mapping file or naming convention (MyClass -> MyClassTest).
2. Run sf project deploy start --test-level RunSpecifiedTests --tests MyClassTest,MyServiceTest --target-org target.
3. Parse the deployment result for per-class coverage. Fail the build if any component in the package is below 75%.

Why not the alternative: RunLocalTests in a large org can take over an hour. RunSpecifiedTests gives targeted feedback in minutes but requires the team to maintain accurate test-to-class mappings.

Pattern: Standalone Test Run with Separate Coverage Retrieval

When to use: When the pipeline needs test results and coverage data but is not performing a deployment (e.g., a PR validation build against a sandbox).

How it works:

1. Run sf apex run test --test-level RunLocalTests --result-format junit --output-dir ./test-results --wait 30 --target-org sandbox.
2. Run sf apex get test --test-run-id <id> --code-coverage --result-format json to retrieve coverage separately.
3. Parse the JSON coverage output and enforce the team threshold.

Why not the alternative: Combining --code-coverage with --wait on the initial run can return 0% coverage due to a known CLI behavior. Separating the retrieval step avoids this.

Decision Guidance

Recommended Workflow

Step-by-step instructions for configuring CI testing in a Salesforce pipeline:

1. Authenticate the CI runner -- configure JWT-based or auth-URL-based authentication so the pipeline can connect to the target org without interactive login. Store the connected app consumer key and server key as CI secrets.
2. Choose the test level -- use the decision table above. Default to RunLocalTests for production, RunSpecifiedTests for PR builds when test mappings exist, and NoTestRun for metadata-only changesets.
3. Configure the deploy or test command -- use sf project deploy validate for production validation or sf apex run test for sandbox-only test runs. Always include --wait with a timeout (e.g., --wait 60) so the pipeline blocks until tests finish.
4. Capture results in JUnit format -- add --result-format junit --output-dir ./test-results and publish the output directory as a CI artifact. Most CI platforms auto-detect JUnit XML.
5. Retrieve coverage separately if needed -- if using sf apex run test with --code-coverage, verify coverage numbers are non-zero. If they report 0%, fall back to sf apex get test --test-run-id <id> --code-coverage as a second step.
6. Enforce coverage thresholds in script -- parse the coverage JSON and fail the build if org-wide or per-class coverage falls below the team-defined minimum. Do not rely solely on Salesforce's 75% deployment gate.
7. Report results -- publish JUnit artifacts, log coverage percentages, and surface failures clearly in the CI dashboard.

Review Checklist

Run through these before marking work in this area complete:

• CI pipeline authenticates to the target org without interactive prompts
• Test level is explicitly set (not relying on default behavior)
• --wait timeout is long enough for the org's full test suite
• JUnit XML output is collected and published as a CI artifact
• Coverage enforcement script parses actual coverage and gates the build
• The 0% coverage bug workaround is in place if using async test runs with --code-coverage
• Pipeline fails fast on test failures before attempting deployment

Salesforce-Specific Gotchas

Non-obvious platform behaviors that cause real production problems:

1. --code-coverage with --wait returns 0% -- when running asynchronous tests, combining these flags can result in coverage data showing 0% even when tests pass. The platform has not finished aggregating coverage by the time the CLI reads the result. Fix: retrieve coverage separately using sf apex get test --test-run-id <id> --code-coverage.
2. RunSpecifiedTests enforces per-class coverage, not org-wide -- unlike RunLocalTests, which applies the 75% threshold org-wide, RunSpecifiedTests requires 75% coverage on every individual class and trigger in the deployment package. A single under-covered class blocks the entire deployment.
3. Parallel test execution is opt-in per class -- by default, Salesforce runs test classes serially in a test run. The @isTest(isParallel=true) annotation opts a class into parallel execution, which can reduce total run time but introduces risk if tests share mutable state through SeeAllData or poorly isolated setup.
4. Developer Console coverage highlighting goes stale on edit -- the line-level covered/uncovered highlighting in the Tests tab dims the moment you edit the class, signaling the coverage is no longer valid. A developer who reads dimmed highlighting as "still covered" is looking at numbers from before their change. Rerun the tests before trusting it.
5. Setup test history is retained only 30 days -- results on the Application Test Execution / View Test History page disappear 30 days after they finish (or sooner if cleared). Do not treat it as a durable coverage-trend record — capture JUnit/JSON artifacts in the pipeline if you need history.

Output Artifacts

Related Skills

• test-class-standards -- use when the problem is test design quality (assertions, isolation, data factories) rather than CI pipeline configuration
• sf-cli-and-sfdx-essentials -- use for general sf CLI setup, authentication, and project structure questions beyond testing
• batch-job-scheduling-and-monitoring -- relevant when CI tests exercise batch Apex and need async test patterns