Skill Creator

Skill Creator

A skill for creating new skills and iteratively improving them.

At a high level, the process of creating a skill goes like this:

• Decide what you want the skill to do and roughly how it should do it
• Classify the change into the lowest verification tier that can falsify its likely failure modes
• Write a draft of the skill
• Validate with the smallest evidence that can falsify the changed behavior; treat the full paired eval pipeline as separately authorized work, not an automatic consequence of a tier label
• Help the user evaluate qualitative or quantitative results when the selected tier produces them
• Rewrite the skill based on feedback from the user's evaluation of the results (and also if there are any glaring flaws that become apparent from the quantitative benchmarks)
• Repeat until you're satisfied
• Escalate the verification tier only when the current evidence cannot resolve the changed behavior

Your job when using this skill is to figure out where the user is in this process and then jump in and help them progress through the applicable stages. Full A/B benchmarking is a capability, not a tax on every edit. Words such as "optimize", "improve", or "comprehensive" describe intent, not failure surface or evaluation budget. Do not create eval files, fan out paired agents, grade outputs, or launch a viewer merely because a task sounds broad or a long conversation precedes it.

Six standing disciplines apply throughout, because these failure modes ship convincing-looking skills that are wrong:

1. Verify before you write. Every technical assertion that enters the skill (endpoint, parameter, command, version, behavior) must trace to something you executed and observed — in this session or an explicitly approved mined one. Can't verify it right now? Either go verify it, or mark it explicitly ("unverified — from memory"). A skill multiplies whatever it contains: verified knowledge compounds, and so do confidently-stated errors. For knowledge skills (content is mostly facts about an external system — API endpoints, parameters, fields, platform behavior), read references/knowledge-skill-grounding.md for the operational version: the authority ladder (observed behavior > machine-readable contract > exercised production code > official docs > memory), evidence-scope annotation and live/replay/synthetic evidence tiers, pre-ship doc-example smoke runs, and the audience/Windows portability checklist. A source-grounding audit once found multiple confident contract claims that contradicted evidence already available to the author (methodology Case 9).

2. Treat "impossible / not supported" as a hypothesis, not a conclusion. When a capability seems blocked (an API error wall, a tool that won't connect, a format that won't open), exhaust the observation paths — the UI's own network traffic, an alternative channel, a different documented identifier — before writing "the platform doesn't support this" into a skill. Observed behavior outranks speculative request shapes.

3. Stand on the field's shoulders — retrieve the domain's established best-practices into context BY DEFAULT, before authoring or optimizing a skill's methodology. A skill's methodology is only as good as the knowledge in your context window, not the knowledge latent in your weights: pretraining is lossy, goes stale, and often is not even activated unless the canonical sources are actually pulled in. So the quality ceiling of what you write is your training data + the user's input — unless you deliberately retrieve the subject domain's real prior art. Do it: WebSearch the field's canonical theory / standards / methods, and read any bundled or installed skill in that domain, then fold the load-bearing principles into the skill with attribution. This is a different axis from "Prior Art Research" below — that finds tools/infrastructure to reuse; this grounds the quality of the methodology itself in the discipline's accumulated science. Make it the default action, not something you wait to be asked for: briefly tell the user which field you're pulling from and let them say "skip," but never ship a methodology capped by your memory plus their prompt when 40 years of the field's public work is one search away. Examples: a data-visualization skill must absorb Cleveland & McGill's graphical-perception ranking and Bertin's visual variables (position/length beat color beat text — measured, not aesthetic); a date/time skill must surface the mature libraries and their canonical pitfalls; a persuasion/negotiation skill must retrieve the established frameworks rather than reinvent them from memory. If the canonical knowledge lives only in your weights and never enters context, you are guessing where you could be citing.

4. Preserve before you compress an existing skill. Updating an existing skill is a migration, not a blank-page rewrite. Before the first edit, capture the auditable source bundle with the audit tool's snapshot command, or reconstruct it from an explicit Git ref; an arbitrary copy plus a provenance label is not a baseline. Inventory runtime capabilities, trigger contexts, interfaces, references, and eval coverage. Progressive disclosure and concision authorize moving or deduplicating content; they do not authorize silently deleting behavior. After editing, run scripts/audit_skill_regression.py and classify every unmatched old unit. A runtime contract that survives only in evals/, tests, or an unlinked reference is still lost. Do not call the update complete while any candidate is unclassified or any true gap remains unfixed. The same logic governs reversals, not just deletions, and covers any prior commitment — not only the ones carrying a date and a name: overturning a decision already made is a proposal, never a side effect. Say it out loud and get it accepted. A silent rewrite is worse than a silent deletion, because it destroys the artifact and the evidence that could have caught it in one move — and it blinds every downstream reviewer (see #5).

   Classify each intended delta before editing; “compression” is a claim of behavioral equivalence, not a synonym for “shorter.” For every old user scenario, a fresh agent must still be able to find the trigger, decision inputs, supported action, stop/confirmation point, impact/recovery boundary, and verification path. Use these names as a separate change-type label in the plan, review, and changelog; record the label in each regression candidate's reason/semantic review while keeping the audit tool's disposition field to the exact Step 4 enum:

   Change type	Definition	Required handling
   Lossless compression	Only representation changes: relocate detail into a directly reachable bundled reference, deduplicate against a packaged canonical source, or shorten wording while preserving every runtime contract above	May be called compression; prove scenario reachability, not just string survival
   Capability retirement	A previously executable job or action is no longer available	Not compression; name the retired capability, obtain traceable user approval, and publish the boundary
   Scope narrowing / boundary change	Fewer inputs, targets, environments, or modes remain supported; an execution branch becomes analysis-only	Not compression even when safer; surface and publish the boundary/trade-off, then obtain traceable approval unless the user already requested this exact narrowing
   Workflow or safety redesign	Commands, sequence, confirmation, side effects, recovery, or success checks materially change	Not compression; surface the new contract and trade-off, obtain traceable approval unless the user already requested this exact redesign, then verify it independently of preservation
   Bug fix / factual correction	The implementation or prose is brought back to an explicit existing contract or current authority	Not compression; use the narrow authority or deterministic regression that decides it
   Lossy summarization	Exact conditions, commands, decision semantics, or execution exits disappear without a runtime-reachable equivalent	Regression, not optimization; restore it or reclassify it through an approved row above

   Deduplication counts as lossless only when the canonical source ships with or is reliably available to the skill and SKILL.md tells the runtime reader when to load it. “The idea still exists in tests, a changelog, an installed sibling skill, or the author's memory” is not compression. If one patch contains several types, list them separately; never let the compression label launder a retirement or redesign.

5. Nothing ships on self-review alone — an independent, fresh-context adversarial pass is a standing step, not a special case. Your "it's good now" judgment runs on the same model that produced the artifact, so it shares the exact blind spots that produced the defect. Measured, not folklore: intrinsic self-correction without external feedback does not reliably improve output and sometimes degrades it (Huang et al., ICLR 2024), and when generator and evaluator share error modes, iterating raises confidence without adding information. More self-checks cannot escape that; only an outside view can. Full procedure — prompt templates, anchor selection, the findings table, worked cases — in references/independent-review-protocol.md; read it before your first pass. The load-bearing rules:

   • Independence has two faces, and the second one is the one authors miss. The reviewer's context must not be a fork of yours — a fork inherits your blind spots and hands back a "reviewed" stamp. But the evidence it measures against must also sit outside the change's blast radius, or it inherits your conclusion: it reads an artifact and a spec that already agree, and reports agreement. Note "not edited this session" is too weak a line — in the reference's controlled case the poisoned record was edited the previous day, and three reviewers went blind to a defect they caught in its untouched twin. Rank anchors by how hard they are for you to have touched: the user's own transcript words > a git ref predating the work (git log --oneline <ref>..HEAD -- <path>; any commit of yours disqualifies it) > an append-only log (a convention, not an enforcement). Greenfield has no anchor — say so and ask for one rather than reporting a pass you did not run.
   • Give it exactly two evidence-bearing inputs: the immutable artifact and the reader spec. Also provide the non-evidentiary control metadata that bounds the pass: current change blast radius, named failure axes, and terminal condition. Give no design rationale, project background, or "just confirm X is fine"—those convert an independent reviewer into a rubber stamp. The reader spec and control metadata say who executes what and where to stop; they do not tell the reviewer that the implementation is correct. For a SKILL.md the reader is another agent executing it, and its failure mode is "I don't know which tool to call," not "I don't know this word"—ask which instructions it could not act on. Freeze all five fields before dispatch and never use post-hoc context to explain findings away.
   • It is ground truth for comprehensibility and for completeness-against-a-corpus; it has no authority over taste. Apply those two directly. Treat "this might be a bug / I'd suggest Y" as a hypothesis and reproduce it yourself first. Never delegate AI-slop or aesthetic judgment — same-model blind spot.
   • Enumerate failure axes, not content areas — that also decides how many reviewers you run. The axis is the question you ask; the area is the material you read. Three reviewers covering scenarios, arithmetic, and the diff but all asking "is this coherent?" is one reviewer billed three times. One is the default and frequently sufficient; add one only for an additional axis. The axis self-review is worst at is fidelity — "is this still faithful to commitments already made?" — because the author is the one who moved the commitment, and coherence and fidelity are orthogonal: an artifact can be flawlessly self-consistent while being completely unfaithful to what was decided.
   • Freeze the review boundary before dispatch, and stop when that boundary is satisfied. Record the immutable artifact/ref, reader spec, current change's blast radius, named failure axes, and terminal condition. A finding blocks this release when the current change caused it, widened it, newly advertised the affected behavior, or made an old contradiction reachable through the edited path. A pre-existing defect that is unchanged and was not newly promised is a separate backlog hypothesis, not automatic authorization to redesign the skill. After substantive fixes, a new reviewer rechecks the failed axes plus regression; it does not reopen an unbounded whole-skill audit. Full decision procedure and examples: references/independent-review-protocol.md.
   • Leave an artifact, or this is just a warning. Discipline #6 says a check yielding an opinion loses to one yielding an artifact — so this one produces a file too, or it loses to completion-drive exactly when it matters. Write independent-review.md under skill-reviews/<skill-name>/ in your private, git-tracked knowledge repo: the reviewer prompt verbatim (so a later reader can see whether it was leading), the findings with a disposition and reason each (which is what separates legitimate filtering from discarding what hurts), and what could not be checked. If you don't know which repo is your private knowledge repo (or don't have one), say so and ask the user — do not guess a location that lands in either forbidden zone. Two forbidden locations: NOT in <skill-name>-workspace/ (gitignored scratch dirs that get wiped — this file is cross-session review evidence and must survive them) and NOT in any repo that is or may become public or distributed — which normally rules out the reviewed skill's own repo (review content inherently quotes private paths, real names, and project details). Re-review with a new agent after a substantive edit — a rule, contract, or number changed, not a typo. From Step 5 onward this file is the evidence the pass happened; its absence means it did not. Writing the file is not the same as it existing for the next session — git add + git commit it in that private repo in the same turn. An uncommitted file sitting in a git working directory carries none of the "git-tracked" guarantee this rule exists for: it can be lost, overwritten, or simply never picked up by whatever process later checks "was this reviewed?" (real case: the file was written correctly, on the correct path, with real findings — and still failed a later automated check, because it had never been committed; the fix was one git commit, not a relocation).
   • Discipline #4's regression gate is mechanical and complementary: it proves you did not delete behavior. It says nothing about whether what you wrote can actually be followed.

6. Design the checks you write so they cannot self-certify green. Skills are largely made of checks — gates, checklists, "before you ship" steps — and a check that the executing context can pass while violating the very rule it encodes is worse than no check, because it manufactures confidence. Four rules, borrowed from fields that solved this before software:

   • The verification must cover every clause of its rule. If the rule says "A + B + C," the evidence must demand proof of A, and B, and C separately. One confirm line bolted onto a three-clause rule gets satisfied by whichever clause the author already did; the others are invisible. Real case: a report-authoring skill carried a delivery gate whose rule read "options as side-by-side chips + recommendation highlighted + background written as complete, self-sufficient sentences a stranger could follow" — but the evidence line under it asked only for "N decision items, all rendered as chips." The author ran the gate, wrote that evidence, self-certified green, and shipped a page whose labels were single characters with all the context deleted. The rule sat in the file the entire time; the check simply never measured that clause.

   • A check that misfires on healthy input is worse than no check. The failure above is a check that passes when it should fail; this is its mirror — a check that fails when it should pass. It is the more expensive one, because it teaches the operator to bypass reflexively (--no-verify, SKIP=1, --force), and once that reflex exists the gate is off for every input, including the ones it was built for. So when authoring a fail-closed check, false positives outrank false negatives: missing one real problem costs you that instance, while killing one healthy input costs you the entire gate. Watch for the tell: the frustration of having hit the same trap repeatedly is itself the risk signal — it is exactly the state in which an author ships a defense that was never calibrated against healthy input. Real case: after stepping on one formatting trap three times in a day, the author added a regex check to a linter; it killed 33 healthy inputs on the project's own corpus and was reverted the same hour. Calibrate before you arm it — run any fail-closed check across real, known-good material and confirm zero false positives; prefer loosening it until it occasionally misses over letting it ever misfire.

   • Make each item a falsifiable observation, not a self-assessment. "Background is self-sufficient" cannot be failed by the person who wrote it; "cover the rest of the page, read one card alone, and state what it is deciding" can. Prefer checks that yield an artifact — a command's output, a quoted line, a screenshot — over checks that yield an opinion.

   • The same suspicion applies to the checks you run, not just the ones you write. The four rules above govern checks that ship inside a skill. But the greps, finds and one-off scripts you use to verify your own work are instruments too, and a wrong instrument reports a clean result just as confidently as a right one. In one 2026-07 session five separate verification commands lied in both directions: a find without -L reported an installed skill's files missing (they were behind a symlink); a grep --exclude-dir=<name> hid a second copy of the very thing being audited; an inverted shell condition raised a false alarm that a removal had not happened; a regex spanning newlines invented 55 "lost quotations"; and a search over two of five files reported two rules missing that were present in the third. Every one of them was believed at first, and every one was caught only by re-running a differently-shaped check.

     The fix is the oldest one in experimental practice: run the instrument on a case whose answer you already know before trusting it on the case you don't. Grepping for a string you expect to be absent? First grep for one you know is present, in the same command shape — if that returns 0 too, the command is broken, not the file. This costs one line and converts "I checked" into "I checked with an instrument I calibrated."

     "Differently-shaped" means a different predicate, not a different command. Two programs that decide the same question the same way are one instrument billed twice, and running both feels exactly like corroboration — it produces two numbers that agree, which is the shape of evidence. Real case (2026-09): a new check reported which bundled references were unreachable; a separate script written to confirm it returned the same count, and that agreement was written up as "calibrated, no false positives." Both asked does the full path references/<name> appear in SKILL.md — so both were blind to the same healthy form (a ### references/ section listing each file by bare name), and a correct skill was flagged. The known-answer sample above would have caught it in one line; a second implementation of the same rule never could. Before calling a cross-check independent, say out loud what question each side asks — if it is the same sentence, you have one instrument.

     Two specific shapes worth memorizing, because both appeared above and both fail silently: find does not follow symlinks without -L (and skill installs are frequently symlinks into a source repo), and --exclude-dir matches by basename everywhere in the tree, not just at the path you had in mind.

     And there is a second half to this rule that only bites when the check SHIPS: calibrate against the standard implementation, not the one on your machine. The instrument rule above keeps your conclusion honest; this keeps the reader's working. A tool-behavior claim written into a skill — a flag, a recursion mode, an option that "follows symlinks" — is executed on machines whose binaries you have never seen, and the divergence is silent on both ends: it works when you test it, and it quietly does nothing for them. Two mechanisms produce this, and both are invisible from inside a session: the same command name resolves to a different program (a shell alias or function shadowing the binary — note \tool only escapes an alias, so command tool or an absolute path is the only deterministic form), and the same program behaves differently across implementations (BSD vs GNU vs a drop-in replacement). Real case (2026-07): an author verified that grep -R follows symlinks, wrote it into a skill as the fix for a symlink trap, and shipped it to a 1200-star public repo — their grep was ugrep via a shell function; on macOS's own /usr/bin/grep the same -R matches nothing (it needs -RS), so the prescribed fix failed silently for most readers, inside the very section warning that validators fail silently.

     So: before a tool-behavior assertion enters a shipped artifact, re-run it against the standard binary (/usr/bin/<tool>), not the one your shell hands you. If it does not survive that, do not write the flag — prefer the implementation-independent formulation: resolve the path yourself (readlink -f) instead of betting on a recursion flag, do a substring test in a script instead of a line-oriented match, name the behavior you need instead of the option you happen to know. A prescription that only works in your environment is worse than no prescription, because the reader has no way to discover that it silently did nothing.

   • Use what mature checklist practice already settled. Decide whether a list is READ-DO (execute while reading — for low-frequency or unfamiliar procedures) or DO-CONFIRM (work from expertise, then stop at a defined pause point and confirm — for experienced operators under time pressure), and anchor it at a real pause point rather than "somewhere in the workflow." Keep it to the killer items — critical and commonly missed under pressure, roughly five to nine; everything beyond that dilutes compliance (Gawande, The Checklist Manifesto). And prefer Shingo's control over warning (poka-yoke): a prose reminder depends on vigilance and loses to completion-drive, while a step that blocks progress or forces an artifact needs no vigilance at all. Where a skill can only warn, at least put the warning where the decision gets made — a rule filed in a reference the executing context never opens is not, in practice, a rule.

On the other hand, maybe they already have a draft of the skill. In this case you can go straight to the eval/iterate part of the loop.

Of course, you should always be flexible and if the user is like "I don't need to run a bunch of evaluations, just vibe with me", you can do that instead.

Then after the skill is done (but again, the order is flexible), you can also run the skill description improver, which we have a whole separate script for, to optimize the triggering of the skill.

Cool? Cool.

First: coexistence check (official skill-creator plugin)

Before anything else, run one quick check (a single grep, no output needed on the common path): does ${CLAUDE_CONFIG_DIR:-~/.claude}/plugins/installed_plugins.json contain "skill-creator@claude-plugins-official"?

• Not present (the common case): do nothing — do not install anything, do not mention this section to the user. Proceed with the engagement.
• Present: the official plugin's skill-creator and this edition now sit in the skill list with near-identical descriptions, so future sessions will route between them at random. Tell the user this in one or two sentences, then offer (never act without their consent):
  1. Recommended — run scripts/setup_supersede_hook.sh install. It copies a small self-checking SessionStart hook into their Claude config and registers it in settings.json (with a backup), so every future session deterministically routes skill work to this edition. Reversible with scripts/setup_supersede_hook.sh uninstall; the official plugin stays fully usable when asked for by name. On machines without the official plugin the installer refuses to install anything, so it can never leave a useless hook behind.
  2. Alternative — claude plugin disable skill-creator@claude-plugins-official (reversible with enable), which removes the ambiguity by taking the official entry out of the skill list entirely.

If the hook is already installed (scripts/setup_supersede_hook.sh status shows the SessionStart entry as present), skip all of this silently.

The same machinery is available for skills the user creates: when their skill deliberately overlaps an installed one, generate them a kit with scripts/generate_supersede_kit.py — see "Coexistence & Precedence" under Prior Art Research and references/skill-precedence-and-coexistence.md.

Communicating with the user

The skill creator is liable to be used by people across a wide range of familiarity with coding jargon. If you haven't heard (and how could you, it's only very recently that it started), there's a trend now where the power of Claude is inspiring plumbers to open up their terminals, parents and grandparents to google "how to install npm". On the other hand, the bulk of users are probably fairly computer-literate.

So please pay attention to context cues to understand how to phrase your communication! In the default case, just to give you some idea:

• "evaluation" and "benchmark" are borderline, but OK
• for "JSON" and "assertion" you want to see serious cues from the user that they know what those things are before using them without explaining them

It's OK to briefly explain terms if you're in doubt, and feel free to clarify terms with a short definition if you're unsure if the user will get it.

Using AskUserQuestion (Critical — Read This)

Use the AskUserQuestion tool aggressively at every decision point. Do not ask open-ended text questions in conversation when structured choices exist. This is the single biggest UX improvement you can make — users juggle multiple windows and may not have looked at this conversation in 20 minutes.

Every AskUserQuestion MUST follow this structure:

1. Re-ground: State the skill name, current phase, and what just happened (1-2 sentences). The user may have context-switched away.
2. Simplify: Explain the decision in plain language. No function names or internal jargon. Say what it DOES, not what it's called.
3. Recommend: Lead with your recommendation and a one-line reason why. If options involve effort, show both scales: (human: ~X min / Claude: ~Y min).
4. Options: Provide 2-4 concrete, lettered choices. Each option should be a clear action, not an abstract concept.

Rules:

• One decision per question — never batch unrelated choices
• Provide an escape hatch ("Other" is always implicit in AskUserQuestion)
• Accept the user's choice — nudge on tradeoffs but never refuse to proceed
• Skip the question if there's an obvious answer with no tradeoffs (just state what you'll do)
• If a question times out with no answer (user away from keyboard), neither stall nor barrel through the taste/scope decisions. Do the side-effect-free groundwork first — pre-edit snapshot, inventory, eval-case collection, read-only audits/health checks — and hold the judgment calls (restructure direction, what to delete, go/no-go) for when they're back. Then say plainly which you did and what is waiting on them.

Creating a skill

Capture Intent

Start by understanding the user's intent. The current conversation might already contain a workflow the user wants to capture (e.g., they say "turn this into a skill"). If so, extract answers from the live conversation first — the tools used, the sequence of steps, corrections the user made, input/output formats observed. The user may need to fill the gaps, and should confirm before proceeding to the next step.

Source inventory — always before drafting, with consent boundaries. Inventory the live conversation and existing docs/skills that overlap (see Prior Art Research below). Earlier local session JSONL files are a separate private source: do not open or parse them unless the user explicitly asks to mine history or affirmatively approves that source after you explain what will be read. If approved, fold only relevant prior sessions in through the conversation-mining workflow's redacted extraction; never load raw transcripts into your own context. If not approved, continue from the live conversation and existing project sources without treating the missing history as a blocker.

When mining a conversation (or session transcripts), inventory both asset classes — they land in different places. Knowledge — endpoints, parameters, pitfalls, decision rules — becomes SKILL.md guidance or references/. Code the session had to write — helper scripts, injected snippets, renderers, one-off templates — is a scripts/ candidate: first check whether an existing/upstream tool already owns that execution. Parameterize and bundle only the still-needed local helper or adapter; do not preserve a temporary reimplementation merely because this session wrote it. A prior distillation captured polished prose but omitted the reusable helpers; the general lesson is to keep both knowledge→references and code→scripts channels in frame.

When the source material is past session transcripts (the JSONL files under the Claude Code projects directory) rather than the live conversation, enter the conversation-mining workflow. Do not open raw transcripts in the main context or hand raw paths/content to subagents. Its deterministic manifest → discover → redact → chunk sequence must finish before any agent sees content; only redacted chunks may enter the minimum role/shard plan, with the exact unit count and concurrency cap declared under the budget gate.

First, resolve which DIRECTION this is — before the questions below. The request may be one of several opposite things: build a NEW skill / edit an EXISTING skill / optimize skill-creator itself / or it's not-a-skill-at-all (a one-off task). Guessing wrong wastes the whole session — the research you'd do for "new skill" is the wrong research for "optimize the meta-tool." When the phrasing is ambiguous (e.g. "make me a skill" while pointing at skill-creator's own path), one AskUserQuestion here costs 30 seconds. The wrapper-skill fork below is one special case of this; the direction check is general.

Use skill-creator to optimize <existing-skill> means the normal existing-skill update path. It does not authorize conversation-history mining, a broad rewrite, Tier 3 classification, or multi-agent evaluation. Inspect the actual proposed deltas first; escalate only for a failure surface those deltas really introduce.

1. What should this skill enable Claude to do?
2. When should this skill trigger? (what user phrases/contexts)
3. What's the expected output format?
4. Should we set up test cases to verify the skill works? Skills with objectively verifiable outputs (file transforms, data extraction, code generation, fixed workflow steps) benefit from test cases. Skills with subjective outputs (writing style, art, taste-calibrated reports) often can't use assertions — but "no assertions" is not "no verification". Their verification paths, in order of cost:
   • Historical-task replay: re-run one real prompt the skill has served before, old vs new skill, and compare outputs against the specific rules that changed ("does the new output actually follow the tokens / title grammar this update introduced?"). Cheap, catches "the rule was written but nothing reads it".
   • Production-as-eval: acknowledge that the real test is the user's next actual use — then make the loop explicit: every user correction afterward is an incident to fold back (the skill's own "迭代/活文档" section), every approval is corpus material. A taste skill that ships without this write-back habit doesn't improve; one that has it converges without ever running a formal eval. And when the skill's output is something that keeps running — a guard, a monitor, a scheduled job, a hook — its own telemetry is eval data, and the highest-signal record in it is the first false alarm. A user correction requires a user to notice and bother; a deployed mechanism reports on itself unprompted, often within a day, and a false positive is the sharpest form of that report because it proves a rule you wrote is wrong in a way no amount of re-reading would have shown. Treat the first one as a scheduled eval result rather than an annoyance: check it before assuming the mechanism misbehaved, because the more likely finding is that the instruction was too absolute. (Real instance: a skill prescribed a fail-loud check, the deployed check fired once overnight on a perfectly healthy condition, and the fix was to correct the over-absolute sentence in the skill — nobody complained; the telemetry did.)
   • Render + human review for visual outputs (the skill's own visual-QA gates), never a grep assertion pretending to measure aesthetics. And the renderer you verify with must be the same engine the deliverable will be consumed in — whatever previewer is conveniently installed is not a substitute. A thumbnailer whose layout engine differs from the target application will silently hide the exact defects you are looking for, and a green verification on the wrong engine is worse than no verification, because it buys false confidence. Real case (2026-07): a .docx was "visually verified" through macOS Quick Look thumbnails, which do not reproduce justified-text stretching; Word showed the document's info blocks blown apart the moment the user opened it. The fix was to install the Word-compatible engine (LibreOffice), convert to PDF, rasterize per page, and read every page. Match the engine, or the verification is theater. This generalizes past renderers to every verification tool — parser, linter, validator: it must share an implementation with production, or its green is meaningless. Second case, same shape: an author tried to catch a markup pattern that corrupts the final document by checking at the source stage with a different markdown implementation than the production toolchain used — it parsed all three known-bad inputs as perfectly fine, so any pre-check built on it would have silently passed everything. The honest conclusion was that this particular defect is only detectable after the production tool has run, and the check belongs there. When no available tool shares the production implementation, say the check cannot be done at that stage — do not build the one that can only produce false green. Suggest the appropriate default based on the skill type, but let the user decide.

After extracting answers from conversation history (or asking questions 1-3), use AskUserQuestion to confirm the skill type and testing strategy:

Creating skill "[name]" — here's what I understand so far:
- Purpose: [1-sentence summary]
- Triggers on: [key phrases]
- Output: [format]

RECOMMENDATION: [Objective/Subjective/Hybrid] skill → [suggested testing approach]

Options:
A) Objective output (files, code, data) — set up automated test cases (Recommended if output is verifiable)
B) Subjective output (writing, design) — qualitative human review only
C) Hybrid — automated checks for structure, human review for quality
D) Skip testing for now — just build the skill and iterate by feel

This upfront classification drives the entire evaluation strategy downstream. Get it right here to avoid wasted effort later.

The extend-vs-create check — runs BEFORE any specialized branch

Each of the three specialized workflows below ends with "do not continue reading the sections below", and Prior Art Research happens to sit after them. That ordering is layout, not execution order. The extend-vs-create judgment applies to every branch, and skipping it is exactly how a session ships a skill that duplicates one already installed.

So before routing into wrapper-skill / conversation-mining / artifact-corpus, answer one question: does a skill already exist that this capability belongs to?

Discover the roots, don't recall them. A hand-maintained list of install locations is exactly the artifact that goes stale, and the root you forget is the one that bites.

Search for the file, not for a directory named skills. Skill directories are named after the skill (skill-creator/, <suite>/<skill>/), so a source repo, a marketplace clone and a plugin cache contain no directory called skills at all — searching for that name silently skips them while appearing to work. Every skill has a SKILL.md; that is the layout-agnostic handle.

# 1) discover
find ~ -type f -name SKILL.md -not -path '*/node_modules/*' -not -path '*/.git/*' > /tmp/all-skills.txt

# 2) VERIFY COVERAGE BEFORE TRUSTING IT — `2>/dev/null` and permission denials hide gaps
#    silently, which is exactly how a sweep reports "nothing found" from a root it never
#    entered. A 0 on any line you expect means the search did not go there:
for r in '/.claude/skills/' '/plugins/marketplaces/' '/plugins/cache/' '/.claude-profiles/'; do
  printf '%6s  %s\n' "$(grep -c "$r" /tmp/all-skills.txt)" "$r"
done
# ...and grep for your own skill source repos by path; they must appear too.

# 3) filter by capability VOCABULARY, not by skill name — in every language the target
#    skill might be written in (a skill whose body is Chinese will not match English terms):
xargs grep -li -e '<domain-term>' -e '<域内术语>' < /tmp/all-skills.txt

Expect step 3 to take a few seconds and to still return more than you want; narrow with terms specific to the capability rather than generic ones (chart matches everything, stacked bar does not).

The roots this reaches — and that a from-memory list usually misses: the skill source repos (a claude-code-skills checkout and any -pro sibling), ~/.claude/plugins/marketplaces/ and ~/.claude/plugins/cache/ (marketplace-installed suites — nothing in the source repos hints they are there), ~/.claude/skills/, ~/.codex/skills, ~/.agents/skills, per-profile config homes (~/.claude-profiles/<name>/), and — the one with no signposts at all — every project's own .claude/skills/. Step 2 is what makes that a claim you verified rather than one you inherited.

**Per-project skills a

…(truncated)