Continuous Integration Testing

Continuous Integration Testing

Use this skill when Apex tests must run as part of an automated CI/CD pipeline rather than manually through the Developer Console or Setup UI. The objective is a pipeline configuration that executes the right test level, collects accurate coverage numbers, produces machine-readable results, and fails the build when coverage or test outcomes do not meet deployment thresholds.

Before Starting

Gather this context before working on anything in this domain:

• Which CI platform is in use? The sf CLI commands are the same everywhere, but authentication (JWT, web login, auth URL) and artifact handling differ across GitHub Actions, GitLab CI, Jenkins, and others.
• Does the target org contain managed packages? Managed package tests inflate failure risk under RunAllTestsInOrg and are skipped under RunLocalTests. The wrong choice causes either false failures or silently skipped coverage.
• What is the minimum coverage gate? Salesforce enforces 75% org-wide for production deployments, but many teams enforce higher thresholds (80-85%) or per-class minimums that Salesforce does not natively enforce.

Core Concepts

Test Levels Control What Runs

The sf project deploy start and sf apex run test commands accept a --test-level flag with four values:

• NoTestRun -- skip tests entirely. Only valid for non-production targets or metadata-only changes.
• RunSpecifiedTests -- run only the named test classes. Salesforce enforces 75% coverage per class and per trigger in the deployment package, which is stricter than org-wide.
• RunLocalTests -- run all tests in the org except managed package tests. Coverage is calculated org-wide at 75%.
• RunAllTestsInOrg -- run everything including managed package tests. Rarely appropriate in CI because managed package test failures block your deployment.

The critical nuance: RunSpecifiedTests applies coverage requirements per-component in the package, not org-wide. A single under-covered class in your changeset will fail deployment even if the org is at 90% overall.

Coverage Collection Is Not Coverage Enforcement

The --code-coverage flag on sf apex run test instructs the platform to collect coverage data and include it in the result payload. It does not enforce any threshold. Your pipeline script must parse the coverage result and fail the build if coverage is below the team standard. Salesforce only enforces the 75% threshold during actual deployment, not during standalone test runs.

JUnit Output for CI Report Ingestion

Most CI platforms can parse JUnit XML to display test results in their UI. Use --result-format junit on sf apex run test to produce this format. The output file can be declared as a test artifact in GitHub Actions, GitLab CI, or Jenkins to get per-test pass/fail reporting in the build dashboard.

Asynchronous Test Execution and Polling

By default, sf apex run test with --synchronous runs tests in a single synchronous request, which is limited to a smaller set of tests. Without --synchronous, tests run asynchronously and the CLI polls for completion when --wait is specified. A known platform behavior: combining --code-coverage with --wait on asynchronous runs can return 0% coverage. The workaround is to let the run complete, then retrieve results separately with sf apex get test --test-run-id <id> --code-coverage.

Common Patterns

Pattern: Two-Stage Deploy with Validation Then Quick-Deploy

When to use: Production deployments where you want a CI validation gate before the actual deployment window.

How it works:

1. Run sf project deploy validate --test-level RunLocalTests --target-org prod during CI. This runs all local tests and validates coverage without actually deploying.
2. Capture the validation ID from the output.
3. During the deployment window, run sf project deploy quick --job-id <validation-id> --target-org prod to deploy without re-running tests.

Why not the alternative: Running sf project deploy start directly means tests and deployment happen atomically. If CI validates in the morning and deployment is approved in the afternoon, validation-then-quick-deploy avoids re-running the full test suite.

Pattern: RunSpecifiedTests with Explicit Coverage Gate

When to use: Large orgs where RunLocalTests takes too long (30+ minutes) and the team wants fast CI feedback.

How it works:

1. Determine which test classes cover the changed Apex classes. Many teams maintain a mapping file or naming convention (MyClass -> MyClassTest).
2. Run sf project deploy start --test-level RunSpecifiedTests --tests MyClassTest,MyServiceTest --target-org target.
3. Parse the deployment result for per-class coverage. Fail the build if any component in the package is below 75%.

Why not the alternative: RunLocalTests in a large org can take over an hour. RunSpecifiedTests gives targeted feedback in minutes but requires the team to maintain accurate test-to-class mappings.

Pattern: Standalone Test Run with Separate Coverage Retrieval

When to use: When the pipeline needs test results and coverage data but is not performing a deployment (e.g., a PR validation build against a sandbox).

How it works:

1. Run sf apex run test --test-level RunLocalTests --result-format junit --output-dir ./test-results --wait 30 --target-org sandbox.
2. Run sf apex get test --test-run-id <id> --code-coverage --result-format json to retrieve coverage separately.
3. Parse the JSON coverage output and enforce the team threshold.

Why not the alternative: Combining --code-coverage with --wait on the initial run can return 0% coverage due to a known CLI behavior. Separating the retrieval step avoids this.

Decision Guidance

Recommended Workflow

Step-by-step instructions for configuring CI testing in a Salesforce pipeline:

1. Authenticate the CI runner -- configure JWT-based or auth-URL-based authentication so the pipeline can connect to the target org without interactive login. Store the connected app consumer key and server key as CI secrets.
2. Choose the test level -- use the decision table above. Default to RunLocalTests for production, RunSpecifiedTests for PR builds when test mappings exist, and NoTestRun for metadata-only changesets.
3. Configure the deploy or test command -- use sf project deploy validate for production validation or sf apex run test for sandbox-only test runs. Always include --wait with a timeout (e.g., --wait 60) so the pipeline blocks until tests finish.
4. Capture results in JUnit format -- add --result-format junit --output-dir ./test-results and publish the output directory as a CI artifact. Most CI platforms auto-detect JUnit XML.
5. Retrieve coverage separately if needed -- if using sf apex run test with --code-coverage, verify coverage numbers are non-zero. If they report 0%, fall back to sf apex get test --test-run-id <id> --code-coverage as a second step.
6. Enforce coverage thresholds in script -- parse the coverage JSON and fail the build if org-wide or per-class coverage falls below the team-defined minimum. Do not rely solely on Salesforce's 75% deployment gate.
7. Report results -- publish JUnit artifacts, log coverage percentages, and surface failures clearly in the CI dashboard.

Review Checklist

Run through these before marking work in this area complete:

• CI pipeline authenticates to the target org without interactive prompts
• Test level is explicitly set (not relying on default behavior)
• --wait timeout is long enough for the org's full test suite
• JUnit XML output is collected and published as a CI artifact
• Coverage enforcement script parses actual coverage and gates the build
• The 0% coverage bug workaround is in place if using async test runs with --code-coverage
• Pipeline fails fast on test failures before attempting deployment

Salesforce-Specific Gotchas

Non-obvious platform behaviors that cause real production problems:

1. --code-coverage with --wait returns 0% -- when running asynchronous tests, combining these flags can result in coverage data showing 0% even when tests pass. The platform has not finished aggregating coverage by the time the CLI reads the result. Fix: retrieve coverage separately using sf apex get test --test-run-id <id> --code-coverage.
2. RunSpecifiedTests enforces per-class coverage, not org-wide -- unlike RunLocalTests, which applies the 75% threshold org-wide, RunSpecifiedTests requires 75% coverage on every individual class and trigger in the deployment package. A single under-covered class blocks the entire deployment.
3. Parallel test execution is opt-in per class -- by default, Salesforce runs test classes serially in a test run. The @isTest(isParallel=true) annotation opts a class into parallel execution, which can reduce total run time but introduces risk if tests share mutable state through SeeAllData or poorly isolated setup.

Output Artifacts

Related Skills

• test-class-standards -- use when the problem is test design quality (assertions, isolation, data factories) rather than CI pipeline configuration
• sf-cli-and-sfdx-essentials -- use for general sf CLI setup, authentication, and project structure questions beyond testing
• batch-job-scheduling-and-monitoring -- relevant when CI tests exercise batch Apex and need async test patterns