Skill Tester CI

Skill Tester CI

Validates all CI skills (*-ci) in this repo. Runs spec validation, compiles gh-aw workflows, and checks CI-specific conventions.

When to Use

• After adding or modifying a CI skill
• After upgrading gh-aw to a new version (API changes may break workflows)
• Before committing CI skill changes
• Before submitting the plugin for Anthropic review

Prerequisites

• gh-aw CLI installed (gh extension install github/gh-aw)
• python3 with pyyaml installed
• quick_validate.py available at .claude/skills/skill-creator/scripts/

Checks

1. Anthropic Spec Validation

Run quick_validate.py on every CI skill:

for d in skills/*-ci/; do
  python3 .claude/skills/skill-creator/scripts/quick_validate.py "$d"
done

2. Workflow Example Compilation

Extract workflow blocks from each CI skill's references/workflow-example.md and compile with gh aw compile:

# Extract markdown code blocks with frontmatter
# Copy to temp .github/workflows/
# Run: gh aw compile

Pass criteria: Zero compilation errors. Warnings are reported but don't fail.

3. Permission Checks

CI workflows in strict mode must NOT use write permissions directly. Verify:

• No issues: write (use safe-outputs: add-comment instead)
• No pull-requests: write (use safe-outputs: create-pull-request-review-comment instead)
• No contents: write unless the workflow creates files (eval-creator create mode)
• Required read permissions present for declared toolsets

4. Structural Checks

5. Cross-Workflow Validation

For workflows that use call-workflow:

• Verify the target workflow exists and has workflow_call in its on: section
• Verify the target's inputs match what the caller provides

Output Format

## CI Skill Test Results

**Date:** YYYY-MM-DD
**gh-aw version:** vX.Y.Z
**Skills tested:** N
**Passed:** N
**Compile errors:** N
**Spec failures:** N

### Compilation Results
- [skill-name]: ✓ compiled (N KB) | ✗ error: [message]

### Spec Results
- [skill-name]: ✓ valid | ✗ [error]

### Permission Issues
- [skill-name]: [issue]

Running

Invoke manually:

/skill-tester-ci

Or run the script directly:

bash skills/skill-tester-ci/scripts/run-tests.sh

What This Skill Does NOT Do

• Does not test interactive skills (use skill-tester for those)
• Does not execute workflows against real repos — only compiles them
• Does not modify skills — reports findings only
• Does not test workflow runtime behavior — compilation validates structure and permissions only