CLI Commands Reference

CLI Commands Reference

Complete reference for SF CLI commands related to Agentforce testing.

⚠️ CRITICAL: Agent Testing Center Required

All sf agent test commands require Agent Testing Center feature enabled in your org.

# Check if Agent Testing Center is enabled
sf agent test list --target-org [alias]

# If you get these errors, Agent Testing Center is NOT enabled:
# ❌ "Not available for deploy for this organization"
# ❌ "INVALID_TYPE: Cannot use: AiEvaluationDefinition in this organization"

See SKILL.md for enabling this feature.

Command Overview

sf agent test
├── create          Create agent test in org from spec (requires Agent Testing Center)
├── list            List available test definitions (requires Agent Testing Center)
├── run             Start agent test execution (requires Agent Testing Center)
├── results         Get completed test results
└── resume          Resume incomplete test run

sf agent
├── preview         Interactive agent testing (works without Agent Testing Center)
├── generate
│   └── test-spec   Generate test specification YAML (interactive only - no --api-name flag)
└── (other agent commands in sf-ai-agentscript)

Note: sf agent preview works WITHOUT Agent Testing Center - useful for manual testing when automated tests are unavailable.

Test Specification Generation

sf agent generate test-spec

Generate a YAML test specification interactively (no batch/scripted mode available).

sf agent generate test-spec [--output-file <path>]

⚠️ Important: This command is interactive only. There is no --api-name flag to auto-generate from an existing agent. You must manually input test cases through the prompts.

Flags:

⛔ Non-existent flags (DO NOT USE):

• --api-name - Does NOT exist (common misconception)
• --agent-name - Does NOT exist
• --from-agent - Does NOT exist

Interactive Prompts:

The command interactively prompts for:

1. Utterance - Test input (user message)
2. Expected topic - Which topic should be selected
3. Expected actions - Which actions should be invoked
4. Expected outcome - Response validation rules
5. Custom evaluations - JSONPath expressions for complex validation
6. Add another? - Continue adding test cases

Example:

sf agent generate test-spec --output-file ./tests/support-agent-tests.yaml

# Interactive session:
# > Enter utterance: Where is my order?
# > Expected topic: order_lookup
# > Expected actions (comma-separated): get_order_status
# > Expected outcome: action_invoked
# > Add another test case? (y/n): y

Test Creation

sf agent test create

Create an agent test in the org from a YAML specification.

sf agent test create --spec <file> --target-org <alias> [--api-name <name>] [--force-overwrite]

Required Flags:

Optional Flags:

Example:

# Create test from spec
sf agent test create --spec ./tests/support-agent-tests.yaml --target-org dev

# Force overwrite existing test
sf agent test create --spec ./tests/updated-spec.yaml --api-name MyAgentTest --force-overwrite --target-org dev

# Preview without deploying
sf agent test create --spec ./tests/spec.yaml --preview --target-org dev

Output:

Creates AiEvaluationDefinition metadata in the org at:

force-app/main/default/aiEvaluationDefinitions/[TestName].aiEvaluationDefinition-meta.xml

Test Execution

sf agent test run

Execute agent tests asynchronously.

sf agent test run --api-name <name> --target-org <alias> [--wait <minutes>]

Required Flags:

Optional Flags:

Example:

# Run test and wait up to 10 minutes
sf agent test run --api-name CustomerSupportTests --wait 10 --target-org dev

# Run async (returns job ID immediately)
sf agent test run --api-name MyAgentTest --target-org dev

# Run with JSON output for CI/CD
sf agent test run --api-name MyAgentTest --wait 15 --result-format json --output-dir ./results --target-org dev

# Run with verbose output
sf agent test run --api-name MyAgentTest --wait 10 --verbose --target-org dev

Async Behavior:

Without --wait, the command:

1. Starts the test
2. Returns a job ID
3. Exits immediately

Use sf agent test results --job-id <id> to retrieve results later.

Test Results

sf agent test results

Retrieve results from a completed test run.

sf agent test results --job-id <id> --target-org <alias> [--result-format <format>]

⚠️ CRITICAL BUG: The --use-most-recent flag is documented in --help but NOT IMPLEMENTED. You will get "Nonexistent flag" error. ALWAYS use --job-id explicitly.

Flags:

⛔ Non-working flags (DO NOT USE):

• --use-most-recent - Documented but NOT implemented as of Jan 2026

Example:

# Get results from specific job (REQUIRED - must use job-id)
sf agent test results --job-id 4KBak0000001btZGAQ --result-format json --target-org dev

# Save results to file
sf agent test results --job-id 4KBak0000001btZGAQ --output-dir ./results --target-org dev

# With verbose output to see action details
sf agent test results --job-id 4KBak0000001btZGAQ --verbose --target-org dev

Getting the Job ID: The sf agent test run command outputs the job ID when it starts:

Job ID: 4KBak0000001btZGAQ

Save this ID to retrieve results later.

Test Resume

sf agent test resume

Resume or retrieve results from an incomplete test.

sf agent test resume --job-id <id> --target-org <alias> [--wait <minutes>]
sf agent test resume --use-most-recent --target-org <alias>

Flags:

Example:

# Resume specific job
sf agent test resume --job-id 0Ah7X0000000001 --wait 5 --target-org dev

# Resume most recent test
sf agent test resume --use-most-recent --wait 10 --target-org dev

Test Listing

sf agent test list

List all agent test runs in the org.

sf agent test list --target-org <alias>

Example:

sf agent test list --target-org dev

Output:

Test Name                  Status      Created
─────────────────────────────────────────────────
CustomerSupportTests       Completed   2025-01-01
OrderAgentTests           Running     2025-01-01
FAQAgentTests             Failed      2024-12-30

Interactive Preview

sf agent preview

Test agent interactively via conversation.

sf agent preview --api-name <name> --target-org <alias> [options]

Required Flags:

Optional Flags:

Modes:

Example:

# Simulated preview (default - safe for testing)
sf agent preview --api-name Customer_Support_Agent --target-org dev

# Save transcripts
sf agent preview --api-name Customer_Support_Agent --output-dir ./logs --target-org dev

# Live preview with real actions
sf agent preview --api-name Customer_Support_Agent --use-live-actions --client-app MyConnectedApp --target-org dev

# Live preview with debug logs
sf agent preview --api-name Customer_Support_Agent --use-live-actions --client-app MyApp --apex-debug --output-dir ./logs --target-org dev

Interactive Session:

> Hello, how can I help you today?

You: Where is my order?

Agent: I'd be happy to help you check your order status. Let me look that up...
[Action: get_order_status invoked]
Your order #12345 is currently in transit and expected to arrive tomorrow.

You: [ESC to exit]

Save transcript? (y/n): y
Saved to: ./logs/transcript.json

Output Files:

When using --output-dir:

• transcript.json - Conversation record
• responses.json - Full API messages with internal details
• apex-debug.log - Debug logs (if --apex-debug)

Result Formats

Human (Default)

Formatted for terminal display with colors and tables.

sf agent test run --api-name Test --result-format human --target-org dev

JSON

Machine-parseable for CI/CD pipelines.

sf agent test run --api-name Test --result-format json --target-org dev

JSON Structure:

{
  "status": "Completed",
  "summary": {
    "passed": 18,
    "failed": 2,
    "total": 20
  },
  "testResults": [
    {
      "name": "route_to_order_lookup",
      "status": "Passed",
      "expectedTopic": "order_lookup",
      "actualTopic": "order_lookup",
      "expectedActions": ["get_order_status"],
      "actualActions": ["get_order_status"]
    }
  ],
  "metrics": {
    "topicSelectionAccuracy": 95.0,
    "actionInvocationRate": 90.0,
    "duration": 45200
  }
}

JUnit

XML format for test reporting tools.

sf agent test run --api-name Test --result-format junit --output-dir ./results --target-org dev

JUnit Structure:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite name="CustomerSupportTests" tests="20" failures="2" time="45.2">
  <testcase name="route_to_order_lookup" classname="topic_routing" time="2.1"/>
  <testcase name="action_invocation_test" classname="action_invocation" time="3.2">
    <failure type="ACTION_NOT_INVOKED">Expected action get_order_status was not invoked</failure>
  </testcase>
</testsuite>

TAP (Test Anything Protocol)

Simple text format for basic parsing.

sf agent test run --api-name Test --result-format tap --target-org dev

TAP Output:

TAP version 13
1..20
ok 1 route_to_order_lookup
ok 2 action_output_validation
not ok 3 complex_order_inquiry
  ---
  message: Expected get_order_status invoked 2 times, actual 1
  category: ACTION_INVOCATION_COUNT_MISMATCH
  ...

Common Workflows

Workflow 1: First-Time Test Setup

# 1. Generate test spec
sf agent generate test-spec --output-file ./tests/my-agent-tests.yaml

# 2. Edit YAML to add test cases (manual step)

# 3. Create test in org
sf agent test create --spec ./tests/my-agent-tests.yaml --api-name MyAgentTests --target-org dev

# 4. Run tests
sf agent test run --api-name MyAgentTests --wait 10 --target-org dev

Workflow 2: CI/CD Pipeline

# Run tests with JSON output
sf agent test run --api-name MyAgentTests --wait 15 --result-format junit --output-dir ./results --target-org dev

# Check exit code
if [ $? -ne 0 ]; then
  echo "Agent tests failed"
  exit 1
fi

Workflow 3: Debug Failing Agent

# 1. Run preview with debug logs
sf agent preview --api-name MyAgent --use-live-actions --client-app App --apex-debug --output-dir ./debug --target-org dev

# 2. Analyze transcripts
cat ./debug/responses.json | jq '.messages'

# 3. Check debug logs
cat ./debug/apex-debug.log | grep ERROR

Error Troubleshooting

⚠️ Common Pitfalls (Lessons Learned)

1. Action Matching Uses Superset Logic

Action assertions use flexible superset matching:

• Expected: [IdentifyRecordByName]
• Actual: [IdentifyRecordByName, SummarizeRecord]
• Result: ✅ PASS (actual contains expected)

This means tests pass if the agent invokes at least the expected actions, even if it invokes additional ones.

2. Topic Names Vary by Agent Type

Best Practice: Run one test first, check actual topic names in results, then update expectations.

3. Test Data Must Exist

Tests referencing specific records will fail if:

• The record doesn't exist (e.g., "Acme" account)
• The record name doesn't match exactly (case-sensitive)

Best Practice: Query org for actual data before writing tests:

sf data query --query "SELECT Name FROM Account LIMIT 5" --target-org dev

4. Two Fix Strategies Exist

Related Commands