Skills Engineering

Skills Engineering

This skill teaches an agent how to author, evaluate, and iteratively improve Agent Skills. It follows a rigorous TDD lifecycle with nine phases. Each phase has one job. Each phase's output feeds the next phase's red state.

Do not reference this file for phase instructions. Call scripts/generate.sh — the script output is the guidance.

Phase routing

How to invoke

bash skills/skills-engineering/scripts/generate.sh \
  --phase <spec|behavioral|script-test|script|skill|adversary|eval|improve|refactor> \
  --skill-path .agents/skills/<skill-name>

The script emits a targeted prompt and the next phase name. Follow its output — it handles tier detection, test set selection, and phase sequencing. The LLM never sees cross-phase content.

Skill directory layout

<skill-name>/
├── SKILL.md                     # Required: YAML frontmatter + instructions
├── spec.md                      # Lightweight intent + script contracts
├── references/                  # Documentation loaded on demand
├── scripts/                     # Executable code (Bash, Python)
├── assets/                      # Templates, images, data files
└── tests/
    ├── behavioral-tests.json    # BDD: Given/When/Then behavioral contracts
    ├── adversarial-tests.json   # Boundary attacks
    ├── smoke-tests.json         # Fast subset for trivial changes
    └── test-<script>.sh         # Acceptance tests for each script

Progressive disclosure in skill design

1. Metadata (name + description) — loaded at startup. Teaches the agent when to use the skill.
2. SKILL.md body — loaded on activation. Route to references for detail, don't embed everything.
3. References — loaded on demand. Encyclopedic knowledge, schemas, detailed procedures.

Complexity tiers

The phase router detects change scope and selects the appropriate test level:

Key design principles

• Every script gets TDD. Write acceptance tests first (script-test phase), then implement AC-by-AC (script phase). Scripts are code — same discipline as .py or .sh.
• Code-based pre-grading catches deterministic failures. Test JSON supports assert_sh — shell assertions that run before LLM grader dispatch. If any assertion fails, the test fails immediately without a subagent call.
• Adversarial tests come after the skill exists. You can't write effective boundary attacks against a skill you haven't read. The adversary phase studies the concrete skill and crafts targeted attacks.
• The context window is the API. Phase isolation protects context. Authoring never sees eval criteria. Eval never sees authoring instructions.
• Skills are code. Skill files are markdown syntax. Non-trivial edits require worktree isolation.
• Description is the trigger. The description field is the primary discovery mechanism. Write it to describe both what the skill does and when to use it.
• Behavioral expectations live in the skill, not in agent memory. When a user gives feedback about how a skill should behave (output format, required sections, mandatory steps, prohibitions), that requirement MUST be encoded in SKILL.md, a referenced file, or a script/template inside the skill directory. NEVER store it as agent memory — memory is invisible to other consumers of the skill (other users, other sessions, CI runs). If you find yourself reaching for the memory tool to record a skill's required behavior, stop and update the skill instead. Memory is for cross-skill, cross-project preferences about how the operator wants to collaborate; skills carry their own behavioral contracts.
• Shaped outputs get a Jinja2 template. When a skill produces structured/formatted output (summaries, reports, documents, serialized data), place a Jinja2 template in assets/ that declaratively defines the output shape — variable slots ({{ var }}), conditionals ({% if %}), and loops ({% for %}). Agents default to manually writing output that follows the template, without requiring a full data-collection-and-render pipeline.