Workflow Compose

workflow.compose

Purpose

Allows declarative execution of Betty Framework workflows by reading a YAML definition and chaining skills like skill.create, skill.define, and registry.update.

Enables complex multi-step processes to be defined once and executed reliably with proper error handling and audit logging.

Usage

Basic Usage

python skills/workflow.compose/workflow_compose.py <path_to_workflow.yaml>

Arguments

Workflow YAML Structure

# workflows/create_and_register.yaml
name: "Create and Register Skill"
description: "Complete lifecycle: create, validate, and register a new skill"

steps:
  - skill: skill.create
    args: ["workflow.validate", "Validates workflow definitions"]
    required: true

  - skill: skill.define
    args: ["skills/workflow.validate/skill.yaml"]
    required: true

  - skill: registry.update
    args: ["skills/workflow.validate/skill.yaml"]
    required: false  # Continue even if this fails

Workflow Fields

Step Fields

Behavior

1. Load Workflow: Parses the workflow YAML file
2. Sequential Execution: Runs each step in order
3. Error Handling:
   • If required: true, workflow stops on failure
   • If required: false, workflow continues and logs error
4. Audit Logging: Calls audit.log skill (if available) for each step
5. History Tracking: Records execution history in /registry/workflow_history.json

Outputs

Success Response

{
  "ok": true,
  "status": "success",
  "errors": [],
  "path": "workflows/create_and_register.yaml",
  "details": {
    "workflow_name": "Create and Register Skill",
    "steps_executed": 3,
    "steps_succeeded": 3,
    "steps_failed": 0,
    "duration_ms": 1234,
    "history_file": "/registry/workflow_history.json"
  }
}

Partial Failure Response

{
  "ok": false,
  "status": "failed",
  "errors": [
    "Step 2 (skill.define) failed: Missing required fields: version"
  ],
  "path": "workflows/create_and_register.yaml",
  "details": {
    "workflow_name": "Create and Register Skill",
    "steps_executed": 2,
    "steps_succeeded": 1,
    "steps_failed": 1,
    "failed_step": "skill.define",
    "failed_step_index": 1
  }
}

Example Workflow Files

Example 1: Complete Skill Lifecycle

# workflows/create_and_register.yaml
name: "Create and Register Skill"
description: "Scaffold, validate, and register a new skill"

steps:
  - skill: skill.create
    args: ["workflow.validate", "Validates workflow definitions"]
    required: true

  - skill: skill.define
    args: ["skills/workflow.validate/skill.yaml"]
    required: true

  - skill: registry.update
    args: ["skills/workflow.validate/skill.yaml"]
    required: true

Execution:

$ python skills/workflow.compose/workflow_compose.py workflows/create_and_register.yaml
{
  "ok": true,
  "status": "success",
  "details": {
    "steps_executed": 3,
    "steps_succeeded": 3
  }
}

Example 2: API Design Workflow

# workflows/api_design.yaml
name: "API Design Workflow"
description: "Design, validate, and generate models for new API"

steps:
  - skill: api.define
    args: ["user-service", "openapi", "zalando", "specs", "1.0.0"]
    required: true

  - skill: api.validate
    args: ["specs/user-service.openapi.yaml", "zalando", "true"]
    required: true

  - skill: api.generate-models
    args: ["specs/user-service.openapi.yaml", "typescript", "src/models"]
    required: false  # Continue even if model generation fails

Example 3: Multi-Spec Validation

# workflows/validate_all_specs.yaml
name: "Validate All API Specs"
description: "Validate all OpenAPI specifications in specs directory"

steps:
  - skill: api.validate
    args: ["specs/users.openapi.yaml", "zalando"]
    required: false

  - skill: api.validate
    args: ["specs/orders.openapi.yaml", "zalando"]
    required: false

  - skill: api.validate
    args: ["specs/payments.openapi.yaml", "zalando"]
    required: false

Workflow History

Execution history is logged to /registry/workflow_history.json:

{
  "executions": [
    {
      "workflow_path": "workflows/create_and_register.yaml",
      "workflow_name": "Create and Register Skill",
      "timestamp": "2025-10-23T12:34:56Z",
      "status": "success",
      "steps_executed": 3,
      "steps_succeeded": 3,
      "duration_ms": 1234
    }
  ]
}

Audit Integration

If audit.log skill is available, each step execution is logged:

log_audit_entry(
    skill_name="api.validate",
    status="success",
    duration_ms=456,
    metadata={"workflow": "api_design.yaml", "step": 1}
)

Integration

With workflow.validate

Validate workflow syntax before execution:

# Validate first
python skills/workflow.validate/workflow_validate.py workflows/my-workflow.yaml

# Then execute
python skills/workflow.compose/workflow_compose.py workflows/my-workflow.yaml

With Hooks

Auto-validate workflows when saved:

python skills/hook.define/hook_define.py \
  --event on_file_save \
  --pattern "workflows/*.yaml" \
  --command "python skills/workflow.validate/workflow_validate.py {file_path}" \
  --blocking true

In CI/CD

# .github/workflows/test.yml
- name: Run workflow tests
  run: |
    python skills/workflow.compose/workflow_compose.py workflows/test_suite.yaml

Common Errors

Best Practices

1. Validate First: Run workflow.validate before executing workflows
2. Use Required Judiciously: Only mark critical steps as required: true
3. Small Workflows: Keep workflows focused on single logical task
4. Error Handling: Plan for partial failures in non-required steps
5. Test Workflows: Test workflows in development before using in production
6. Version Control: Keep workflow files in git

Files Modified

• History: /registry/workflow_history.json – Execution history
• Logs: Step execution logged to Betty's logging system

Exit Codes

• 0: Success (all required steps succeeded)
• 1: Failure (at least one required step failed)

Timeout

Each skill has a 5-minute (300 second) timeout by default. If a skill exceeds this, the workflow fails.

See Also

• workflow.validate – Validate workflow syntax (workflow.validate SKILL.md)
• Betty Architecture – Five-Layer Model for understanding workflows
• API-Driven Development – Example workflows

Status

Active – Production-ready, core orchestration skill

Version History

• 0.1.0 (Oct 2025) – Initial implementation with sequential execution, error handling, and audit logging