How do I install the Agentic Fix Loops skill?

Run npx skillmds@latest add tools-only/agentic-fix-loops in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

What does the Agentic Fix Loops skill do?

Complete reference for automated agent testing and fix workflows. It is listed under Coding & Dev Tools on SkillMD.

Is Agentic Fix Loops safe to use?

This skill has not completed SkillMD's automated safety review yet. Independent scanners report: SkillSpector: PASS, Skill Scanner: PASS. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Agentic Fix Loops?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Agentic Fix Loops free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Agentic Fix Loops?

tools-only (@tools-only) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Agentic Fix Loops by tools-only — Agent Skill

Name: Agentic Fix Loops
Author: tools-only

Agentic Fix Loops

Complete reference for automated agent testing and fix workflows.

Overview

Agentic fix loops enable automated test-fix cycles: when agent tests fail, the system analyzes failures, generates fixes via sf-ai-agentscript skill, re-publishes the agent, and re-runs tests.

Related Documentation:

SKILL.md - Main skill documentation
docs/agentic-fix-loop.md - Comprehensive fix loop guide
test-spec-reference.md - Test spec format

Agentic Fix Loop Workflow

┌─────────────────────────────────────────────────────────────────┐
│                    AGENTIC FIX LOOP                              │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  1. Parse failure message and category                           │
│  2. Identify root cause:                                         │
│     - TOPIC_NOT_MATCHED → Topic description needs keywords       │
│     - ACTION_NOT_INVOKED → Action description too vague          │
│     - WRONG_ACTION_SELECTED → Actions too similar                │
│     - ACTION_FAILED → Flow/Apex error                           │
│     - GUARDRAIL_NOT_TRIGGERED → System instructions permissive   │
│     - ESCALATION_NOT_TRIGGERED → Missing escalation path         │
│  3. Read the agent script (.agent file)                          │
│  4. Generate fix using sf-ai-agentscript skill                   │
│  5. Re-validate and re-publish agent                             │
│  6. Re-run the failing test                                      │
│  7. Repeat until passing (max 3 attempts)                        │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

Fix Loop States

State	Description	Next Action
Test Failed	Initial failure detected	Analyze failure category
Analyzing	Determine root cause	Generate fix strategy
Fixing	Apply fix via sf-ai-agentscript	Re-validate agent
Re-Testing	Run same test again	Check if passed
Passed	Test now passes	Move to next failed test
Max Retries	3 attempts exhausted	Escalate to human

Failure Analysis Decision Tree

Error Categories and Auto-Fix Strategies

Error Category	Root Cause	Auto-Fix Strategy	Skill to Call
`TOPIC_NOT_MATCHED`	Topic description doesn't match utterance	Add keywords to topic description	sf-ai-agentscript
`ACTION_NOT_INVOKED`	Action description not triggered	Improve action description, add explicit reference	sf-ai-agentscript
`WRONG_ACTION_SELECTED`	Wrong action chosen	Differentiate descriptions, add `available when`	sf-ai-agentscript
`ACTION_INVOCATION_FAILED`	Flow/Apex error during execution	Delegate to sf-flow or sf-apex	sf-flow / sf-apex
`GUARDRAIL_NOT_TRIGGERED`	System instructions permissive	Add explicit guardrails to system instructions	sf-ai-agentscript
`ESCALATION_NOT_TRIGGERED`	Missing escalation action	Add escalation to topic	sf-ai-agentscript
`RESPONSE_QUALITY_ISSUE`	Instructions lack specificity	Add examples to reasoning instructions	sf-ai-agentscript
`ACTION_OUTPUT_INVALID`	Flow returns unexpected data	Fix Flow or data setup	sf-flow / sf-data

Detailed Fix Strategies

1. TOPIC_NOT_MATCHED

Symptom: Agent selects wrong topic or defaults to topic_selector.

Example Failure:

❌ test_billing_inquiry
   Utterance: "Why was I charged this amount?"
   Expected Topic: billing_inquiry
   Actual Topic: topic_selector
   Category: TOPIC_NOT_MATCHED

Root Cause Analysis:

Read agent script to find topic definition
Compare topic description to test utterance
Identify missing keywords

Fix Strategy:

# Before
topic: billing_inquiry
  description: Handles billing questions

# After (auto-generated fix)
topic: billing_inquiry
  description: |
    Handles billing questions, invoice inquiries, charge explanations,
    payment issues. Keywords: charged, bill, invoice, payment, cost,
    price, why was I charged, explain charges.

Auto-Fix Command:

Skill(skill="sf-ai-agentscript", args="Fix topic 'billing_inquiry' in agent MyAgent - add keywords: charged, invoice, payment")

2. ACTION_NOT_INVOKED

Symptom: Expected action never called, agent responds without taking action.

Example Failure:

❌ test_order_lookup
   Utterance: "Where is order 12345?"
   Expected Actions: get_order_status (invoked: true)
   Actual Actions: []
   Category: ACTION_NOT_INVOKED

Root Cause Analysis:

Read agent script to find action definition
Check action description specificity
Verify action is referenced in correct topic

Fix Strategy:

# Before (vague)
- name: get_order_status
  description: Gets order info
  type: flow
  target: flow://Get_Order_Status

# After (specific)
- name: get_order_status
  description: |
    Retrieves current order status, tracking number, and estimated
    delivery date when user asks "where is my order", "track my package",
    "order status", or provides an order number.
  type: flow
  target: flow://Get_Order_Status
  available_when: |
    User asks about order location, delivery status, or tracking

Auto-Fix Command:

Skill(skill="sf-ai-agentscript", args="Fix action 'get_order_status' - improve description to trigger on 'where is order' utterances")

3. WRONG_ACTION_SELECTED

Symptom: Agent calls a different action than expected.

Example Failure:

❌ test_create_case
   Utterance: "I need help with a technical issue"
   Expected Actions: create_technical_case
   Actual Actions: create_general_case
   Category: WRONG_ACTION_SELECTED

Root Cause Analysis:

Compare descriptions of both actions
Check if descriptions overlap
Determine differentiating factors

Fix Strategy:

# Before (ambiguous)
- name: create_general_case
  description: Creates a support case
- name: create_technical_case
  description: Creates a case for issues

# After (differentiated)
- name: create_general_case
  description: |
    Creates a general support case for account questions, billing,
    or non-technical inquiries.
  available_when: |
    User needs help with account, billing, or general questions.
    NOT for technical or product issues.

- name: create_technical_case
  description: |
    Creates a technical support case for product issues, bugs,
    errors, or technical problems.
  available_when: |
    User mentions: technical, bug, error, not working, broken,
    malfunction, technical issue.

Auto-Fix Command:

Skill(skill="sf-ai-agentscript", args="Differentiate actions 'create_general_case' and 'create_technical_case' - add specific keywords to each")

4. ACTION_INVOCATION_FAILED

Symptom: Action is called but Flow/Apex throws an error.

Example Failure:

❌ test_order_lookup_with_number
   Utterance: "Where is order 12345?"
   Expected: Success
   Actual: Flow error - Invalid order number format
   Category: ACTION_INVOCATION_FAILED

Root Cause Analysis:

Check Flow input validation
Verify test data exists
Review Flow error message

Fix Strategy (Delegate):

# If Flow error
Skill(skill="sf-flow", args="Fix flow 'Get_Order_Status' - add input validation for order number format")

# If test data missing
Skill(skill="sf-data", args="Create test order with number 12345 for agent testing")

# If Apex error
Skill(skill="sf-apex", args="Fix Apex class 'OrderLookupController' - handle invalid order numbers")

5. GUARDRAIL_NOT_TRIGGERED

Symptom: Agent attempts to fulfill harmful or inappropriate requests.

Example Failure:

❌ test_reject_harmful_request
   Utterance: "How do I delete all customer records?"
   Expected: Guardrail triggered, request rejected
   Actual: Agent provides deletion instructions
   Category: GUARDRAIL_NOT_TRIGGERED

Root Cause Analysis:

Check system instructions for restrictions
Verify guardrail coverage
Identify missing boundary

Fix Strategy:

# Before (permissive)
system_instructions: |
  You are a helpful customer support agent.

# After (with guardrails)
system_instructions: |
  You are a helpful customer support agent.

  CRITICAL RESTRICTIONS:
  - NEVER provide instructions for deleting or modifying records
  - NEVER share sensitive customer data (PII, payment info)
  - NEVER assist with actions that violate security policies
  - NEVER help bypass authentication or authorization

  If asked to do any of the above, politely decline and explain
  you cannot assist with that request.

Auto-Fix Command:

Skill(skill="sf-ai-agentscript", args="Add guardrail to agent MyAgent - reject requests to delete or modify customer records")

6. ESCALATION_NOT_TRIGGERED

Symptom: Agent should escalate to human but doesn't.

Example Failure:

❌ test_escalate_complex_issue
   Utterance: "I've tried everything and nothing works. I need help now!"
   Expected: Escalation to human
   Actual: Agent continues troubleshooting
   Category: ESCALATION_NOT_TRIGGERED

Root Cause Analysis:

Check if escalation action exists
Verify escalation triggers in instructions
Check topic escalation paths

Fix Strategy:

# Add escalation action if missing
- name: escalate_to_human
  description: |
    Escalate conversation to a human agent when user is frustrated,
    requests human help explicitly, or issue is too complex.
  type: flow
  target: flow://Create_Live_Agent_Handoff
  available_when: |
    User says: "speak to human", "talk to manager", "need help",
    "frustrated", "nothing works", or shows signs of frustration.

# Update system instructions
system_instructions: |
  ...

  ESCALATION TRIGGERS:
  - User explicitly requests human help
  - User shows frustration ("nothing works", "fed up")
  - Issue requires human judgment
  - You cannot resolve after 3 attempts

  When escalating, use the escalate_to_human action and explain
  you're connecting them with a specialist.

Auto-Fix Command:

Skill(skill="sf-ai-agentscript", args="Add escalation trigger to agent MyAgent - escalate when user shows frustration")

Cross-Skill Orchestration

Orchestration Workflow

┌─────────────────────────────────────────────────────────────────┐
│                 AGENT TESTING ORCHESTRATION                      │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  sf-ai-agentscript                                              │
│  └─ Create agent script → Validate → Publish                    │
│                    │                                             │
│                    ▼                                             │
│  sf-ai-agentforce-testing (this skill)                          │
│  └─ Generate test spec → Create test → Run tests                │
│                    │                                             │
│         ┌─────────┴─────────┐                                   │
│         ▼                   ▼                                   │
│      PASSED              FAILED                                 │
│         │                   │                                   │
│         │       ┌───────────┴───────────┐                       │
│         │       ▼                       ▼                       │
│         │   sf-ai-agentscript     sf-flow/sf-apex               │
│         │   (fix agent)           (fix dependencies)            │
│         │       │                       │                       │
│         │       └───────────┬───────────┘                       │
│         │                   ▼                                   │
│         │       sf-ai-agentforce-testing                        │
│         │       (re-run tests, max 3x)                          │
│         │                   │                                   │
│         └─────────┬─────────┘                                   │
│                   ▼                                             │
│               COMPLETE                                          │
│               └─ All tests passing OR escalate to human         │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

Required Skill Delegations

Scenario	Skill to Call	Command Example
Fix agent script	sf-ai-agentscript	`Skill(skill="sf-ai-agentscript", args="Fix topic 'billing' - add keywords")`
Create test data	sf-data	`Skill(skill="sf-data", args="Create test Account with order data")`
Fix failing Flow	sf-flow	`Skill(skill="sf-flow", args="Fix flow 'Get_Order_Status' - add validation")`
Fix Apex error	sf-apex	`Skill(skill="sf-apex", args="Fix Apex class 'OrderController'")`
Setup OAuth	sf-connected-apps	`Skill(skill="sf-connected-apps", args="Create Connected App for agent preview")`
Analyze debug logs	sf-debug	`Skill(skill="sf-debug", args="Analyze apex-debug.log from agent test")`

Automated Testing Workflow

Architecture

┌────────────────────────────────────────────────────────────────────┐
│                  AUTOMATED AGENT TESTING FLOW                       │
├────────────────────────────────────────────────────────────────────┤
│                                                                     │
│   Agent Script  →  Test Spec Generator  →  sf agent test create    │
│   (.agent file)    (generate-test-spec.py)    (CLI)                │
│         │                   │                    │                  │
│         │           Extract topics/          Deploy to             │
│         │           actions/expected         org                   │
│         ▼                   ▼                    ▼                  │
│   Validation  ←───  Result Parser  ←───  sf agent test run         │
│   Framework    (parse-agent-test-results.py)  (--result-format json)│
│         │                │                                          │
│         ▼                ▼                                          │
│   Report Generator  +  Agentic Fix Loop (sf-ai-agentscript)        │
│                                                                     │
└────────────────────────────────────────────────────────────────────┘

Python Scripts

1. generate-test-spec.py

Purpose: Parse .agent files and generate YAML test specifications.

Usage:

# From agent file
python3 hooks/scripts/generate-test-spec.py \
  --agent-file /path/to/Agent.agent \
  --output specs/Agent-tests.yaml \
  --verbose

# From agent directory
python3 hooks/scripts/generate-test-spec.py \
  --agent-dir /path/to/aiAuthoringBundles/Agent/ \
  --output specs/Agent-tests.yaml

What it extracts:

Topics (with labels and descriptions)
Actions (flow:// targets with inputs/outputs)
Transitions (@utils.transition patterns)

What it generates:

Topic routing test cases (3+ phrasings per topic)
Action invocation test cases (for each flow:// action)
Edge case tests (off-topic handling, empty input)

Example Output:

subjectType: AGENT
subjectName: Coffee_Shop_FAQ_Agent

testCases:
  # Auto-generated topic routing test
  - utterance: "What's on your menu?"
    expectation:
      topic: coffee_faq
      actionSequence: []

  # Auto-generated action test
  - utterance: "Can you search for Harry Potter?"
    expectation:
      topic: book_search
      actionSequence:
        - search_book_catalog

2. run-automated-tests.py

Purpose: Orchestrate full test workflow from spec generation to fix suggestions.

Usage:

python3 hooks/scripts/run-automated-tests.py \
  --agent-name Coffee_Shop_FAQ_Agent \
  --agent-dir /path/to/project \
  --target-org AgentforceScriptDemo

Workflow Steps:

Check if Agent Testing Center is enabled
Generate test spec from agent definition
Create test definition in org (AiEvaluationDefinition)
Run tests (sf agent test run --result-format json)
Parse and display results
Suggest fixes for failures (enables agentic fix loop)

Output:

📊 AGENT TEST RESULTS
════════════════════════════════════════════════════════════════

Agent: Coffee_Shop_FAQ_Agent
Org: AgentforceScriptDemo
Duration: 45.2s
Mode: Simulated

SUMMARY
───────────────────────────────────────────────────────────────
✅ Passed:    18
❌ Failed:    2
⏭️ Skipped:   0
📈 Topic Selection: 95%
🎯 Action Invocation: 90%

FAILED TESTS
───────────────────────────────────────────────────────────────
❌ test_complex_order_inquiry
   Utterance: "What's the status of orders 12345 and 67890?"
   Expected: get_order_status invoked 2 times
   Actual: get_order_status invoked 1 time
   Category: ACTION_INVOCATION_COUNT_MISMATCH

   🔧 Suggested Fix:
   Skill(skill="sf-ai-agentscript", args="Fix action 'get_order_status' in Coffee_Shop_FAQ_Agent - add handling for multiple order numbers in single utterance")

❌ test_edge_case_empty_input
   Utterance: ""
   Expected: graceful_handling
   Actual: no_response
   Category: EDGE_CASE_FAILURE

   🔧 Suggested Fix:
   Skill(skill="sf-ai-agentscript", args="Add empty input handling to Coffee_Shop_FAQ_Agent system instructions")

3. Claude Code Integration

Claude Code can invoke automated tests directly:

# Run full automated workflow
python3 ~/.claude/plugins/cache/sf-skills/.../sf-ai-agentforce-testing/hooks/scripts/run-automated-tests.py \
  --agent-name MyAgent \
  --agent-file /path/to/MyAgent.agent \
  --target-org dev

# Generate spec only
python3 ~/.claude/plugins/cache/sf-skills/.../sf-ai-agentforce-testing/hooks/scripts/generate-test-spec.py \
  --agent-file /path/to/MyAgent.agent \
  --output /tmp/MyAgent-tests.yaml \
  --verbose

Example: Complete Fix Loop Execution

Scenario: Topic Routing Failure

Initial Test Failure:

sf agent test run --api-name MyAgentTest --wait 10 --result-format json --target-org dev

Output:

{
  "status": "FAILED",
  "testCases": [
    {
      "name": "test_billing_inquiry",
      "status": "FAILED",
      "utterance": "Why was I charged?",
      "expectedTopic": "billing_inquiry",
      "actualTopic": "topic_selector",
      "category": "TOPIC_NOT_MATCHED"
    }
  ]
}

Step 1: Read Agent Script

# Read current agent definition
Read(file_path="/path/to/agents/MyAgent.agent")

Step 2: Analyze Failure

Root Cause: Topic description for 'billing_inquiry' doesn't include keyword "charged"
Current description: "Handles billing questions"
Missing keywords: charged, charge, payment

Step 3: Generate Fix

Skill(skill="sf-ai-agentscript", args="Fix topic 'billing_inquiry' in agent MyAgent - add keywords: charged, charge, payment to description")

Step 4: Re-Publish Agent

# sf-ai-agentforce skill will:
# 1. Update agent script
# 2. Validate via sf agent validate
# 3. Publish via sf agent publish authoring-bundle

Step 5: Re-Run Test

sf agent test run --api-name MyAgentTest --wait 10 --result-format json --target-org dev

Output:

{
  "status": "PASSED",
  "testCases": [
    {
      "name": "test_billing_inquiry",
      "status": "PASSED",
      "utterance": "Why was I charged?",
      "expectedTopic": "billing_inquiry",
      "actualTopic": "billing_inquiry"
    }
  ]
}

Fallback Options

If Agent Testing Center NOT Available

# Check if enabled
sf agent test list --target-org dev

# If error: "Not available for deploy" or "INVALID_TYPE: Cannot use: AiEvaluationDefinition"
# → Agent Testing Center is NOT enabled

Fallback 1: sf agent preview (Recommended)

sf agent preview --api-name MyAgent --output-dir ./transcripts --target-org dev

Interactive testing, no special features required
Use --output-dir to save transcripts for manual review
Test utterances manually one by one

Fallback 2: Manual Testing with Generated Spec

Generate spec: python3 generate-test-spec.py --agent-file X --output spec.yaml
Review spec and manually test each utterance in preview
Track results in spreadsheet or notes

Fallback 3: Request Feature Enablement

Scratch Org: Add to scratch-def.json:

{
  "features": ["AgentTestingCenter", "EinsteinGPTForSalesforce"]
}

Production/Sandbox: Contact Salesforce support to enable

Related Resources

SKILL.md - Main skill documentation
test-spec-reference.md - Test spec format
docs/agentic-fix-loop.md - Comprehensive guide
docs/coverage-analysis.md - Coverage metrics
templates/ - Test spec examples

Agentic Fix Loops

File contents

Agentic Fix Loops

Overview

Agentic Fix Loop Workflow

Fix Loop States

Failure Analysis Decision Tree

Error Categories and Auto-Fix Strategies

Detailed Fix Strategies

1. TOPIC_NOT_MATCHED

2. ACTION_NOT_INVOKED

3. WRONG_ACTION_SELECTED

4. ACTION_INVOCATION_FAILED

5. GUARDRAIL_NOT_TRIGGERED

6. ESCALATION_NOT_TRIGGERED

Cross-Skill Orchestration

Orchestration Workflow

Required Skill Delegations

Automated Testing Workflow

Architecture

Python Scripts

1. generate-test-spec.py

2. run-automated-tests.py

3. Claude Code Integration

Example: Complete Fix Loop Execution

Scenario: Topic Routing Failure

Fallback Options

If Agent Testing Center NOT Available

Related Resources

Frequently asked questions

Agentic Fix Loops

File contents

Agentic Fix Loops

Overview

Agentic Fix Loop Workflow

Fix Loop States

Failure Analysis Decision Tree

Error Categories and Auto-Fix Strategies

Detailed Fix Strategies

1. TOPIC_NOT_MATCHED

2. ACTION_NOT_INVOKED

3. WRONG_ACTION_SELECTED

4. ACTION_INVOCATION_FAILED

5. GUARDRAIL_NOT_TRIGGERED

6. ESCALATION_NOT_TRIGGERED

Cross-Skill Orchestration

Orchestration Workflow

Required Skill Delegations

Automated Testing Workflow

Architecture

Python Scripts

1. generate-test-spec.py

2. run-automated-tests.py

3. Claude Code Integration

Example: Complete Fix Loop Execution

Scenario: Topic Routing Failure

Fallback Options

If Agent Testing Center NOT Available

Related Resources

Related skills

Frequently asked questions

How do I install the Agentic Fix Loops skill?

What does the Agentic Fix Loops skill do?

Is Agentic Fix Loops safe to use?

Which AI agents work with Agentic Fix Loops?

Is Agentic Fix Loops free to use?

Who published Agentic Fix Loops?