Skill Craft Tdd

Skill Craft — TDD-as-Documentation

A SKILL.md is documentation that the agent treats as code. Apply TDD to writing it.

When to Use

• Drafting a new SKILL.md from scratch (skill_manage action=create or write_file to a new path)
• Rewriting an existing SKILL.md after multiple behavior failures
• Closing a loophole that already caused one regression
• Onboarding a new process that another agent will repeat

When NOT to Use

• Editing a single one-line typo — skip the loop
• Updating an in-repo bundled skill to a new upstream version — let the upstream guide you
• Pasting in someone's "ready" SKILL.md — TDD is for SKILL.md you are about to ship

Core Principle

If you didn't watch an agent fail without the skill, you don't know if the skill teaches the right thing.

Source: obra/superpowers HEAD 44c9b2d6, skills/writing-skills/SKILL.md:10 and :16.

The Iron Law

NO SKILL.MD WITHOUT A BASELINE FAILURE OBSERVED FIRST

If the agent already passes the task without your skill, the skill is decoration. Delete it or change the trigger.

RED-GREEN-REFACTOR for Documentation

RED — Observe a baseline failure

Pick one of:

• Existing failure case — find a recent session log or ~/.hermes/sessions/*.jsonl where an agent got the task wrong, then read enough to reproduce the failure pattern.
• Synthetic pressure test — dispatch a fresh subagent with the task and a TIGHT context budget; do NOT pass the skill. Capture what they miss.

Output: a one-line failure statement ("agent skipped auth middleware on 3/3 attempts because description didn't say 'always'") that will become the skill's main claim.

GREEN — Draft the skill

Write the smallest SKILL.md that fixes the baseline failure.

Rules:

• Frontmatter: name kebab-case ≤64 chars, description ≤1024 chars, first 57 chars self-contained trigger phrase (the system-prompt skill index truncates there).
• Start with Use when ... and complete the trigger class within the first 57 chars.
• Body sections in this order: # Title → ## Overview (what + why, 1-2 paras) → ## When to Use (bullets) → body → ## Common Pitfalls (numbered) → ## Verification Checklist (checkboxes).
• Each step ends with a checkable completion criterion ("all changed files accounted for", "pytest -q exits 0", not "summarize changes").
• Cite a concrete verification command the agent can run with terminal. Vague prose is no-op.

The body must reference at least one Hermes native tool (terminal, skill_view, skill_manage, read_file, patch, delegate_task, execute_code, web_search, web_extract, browser_navigate, todo, memory).

Verify GREEN — Run the same subagent with the skill loaded

Dispatch a fresh subagent with the task AND a path-style load of the new SKILL.md. Compare to the RED baseline. If the subagent now follows the discipline, GREEN. If it still fails, edit the skill and re-run. Loop until compliant.

Common edits at this stage:

• "Always" / "Never" missing → ambiguity lets the agent rationalize
• Completion criterion vague → add the exact command or file path
• Pitfalls section missing → the agent relapses on the first known trap

REFACTOR — Close loopholes

After GREEN, look for:

• Two sentences saying the same thing → keep one
• Section that doesn't change agent behavior → delete (no-op prose)
• "Be careful" / "be thorough" / "use best practices" → replace with a checkable completion criterion
• Description that re-states the body → trim, the body is the truth
• Trigger buried past char 57 → move to front

Verify REFACTOR — re-run the pressure test

If the subagent still passes, REFACTOR preserved behavior. If it fails, the edit was load-bearing. Restore.

Frontmatter Constraints (Hermes enforcement)

• tools/skill_manager_tool.py::_validate_frontmatter enforces: starts with --- (byte 0, no leading blank line), closes with \n---\n, parses as YAML, has name and description, description ≤1024 chars, total file ≤100,000 chars.
• agent/skill_utils.py::extract_skill_description truncates description to 57 chars + "..." in the system-prompt skill index. Front-load the trigger phrase. learn-loop/scripts/synthesize_skill.py enforces the same constraint and rejects if the first 57 chars end with ....
• metadata.hermes.related_skills resolves both skills/<cat>/<name>/ (in-repo) and ~/.hermes/skills/<cat>/<name>/ (user-local). User-local links don't resolve for other users; prefer in-repo references.

Self-Check Before Skill Lands

Run before skill_manage(action="create") or write_file:

python3 ~/.hermes/skills/learning-loop/scripts/synthesize_skill.py \
  --name <name> --description "<desc>" --category <cat> \
  --body "<body without frontmatter>" --research-dir /tmp/x --output /dev/null

Exit 0 = passes; exit 1 = read stderr, fix, retry.

Common Pitfalls

1. Writing SKILL.md first. This is tests-after. The skill will look authoritative and teach nothing.
2. Description starting with "This skill ..." — wastes the 57-char trigger window.
3. No completion criterion in body steps — agents move on before the work is genuinely done.
4. No-op prose — "be careful", "use best practices", "follow TDD" without telling the agent how to verify TDD.
5. Skill is longer than 20k chars — push branch-specific material into references/*.md and link from body.
6. Frontmatter starts with blank line — validator rejects with no clear error.
7. No Hermes native tool cited — synthesize_skill.py rejects.

Verification Checklist

• Baseline failure observed before drafting (RED)
• Trigger phrase self-contained within first 57 chars
• description ≤ 1024 chars, starts with "Use when ..."
• Body references at least one Hermes native tool
• Each step ends with a checkable completion criterion
• Subagent pressure test passes with skill loaded (GREEN)
• At least one round of REFACTOR done; no-op prose removed
• Subagent re-test still passes after REFACTOR
• synthesize_skill.py exits 0
• If in-repo: git add skills/<cat>/<name>/ && git commit
• If user-local: skill_view(name) resolves in a fresh session