Chat Skill Creator

Chat Skill Creator (Claude chat edition)

A skill for building Agent Skills that will run in Claude chat with Code Execution / File Creation enabled. The Agent Skills format is an open standard (agentskills.io) originally developed by Anthropic; this skill produces conformant skills tuned to the chat sandbox's constraints: single-session, single-agent, ephemeral, no subagents, restricted network.

The loop:

1. Figure out what the skill should do — and source the expertise it encodes
2. Draft the SKILL.md (and any bundled scripts/resources)
3. Try test prompts against it inside this session
4. Show the user the outputs, get feedback, iterate
5. Validate, package as a .skill file, hand it over

Order is a guide, not a rule. If the user comes in with a draft, jump straight to step 3. If they say "just vibe with me," skip the formal test cases. Read the situation.

Talking to the user

Skill-creator users span a wide range of technical fluency. Read context cues and calibrate. "Evaluation" and "benchmark" are borderline-OK without explanation; "JSON" and "assertion" need cues that the user knows the terms before using them naked. When in doubt, define briefly. Don't condescend, don't drown them in jargon.

Step 1 — Source the expertise, then capture intent

Skills are only useful when they encode something the model wouldn't already do well. The single most common failure in skill-creation is asking the model to spin a skill from generic priors — it produces vague procedures ("handle errors appropriately," "follow best practices") that don't change behaviour. Before drafting, locate the source of the actual expertise. Three productive routes — they're roughly equal in value, and which one applies depends on what the user is bringing in:

• Extract from a hands-on task already in this conversation. The user may have just walked through a workflow they want to capture ("turn this into a skill"). Mine it. What tools were used, in what order? What corrections did the user make? What input/output formats appeared? What context did the user feed in that the model didn't already know? That's the skill.
• Synthesise from existing artefacts. Runbooks, style guides, schema files, code review comments, post-mortems, fix-commits, a sample of the output the user wants to produce. Project-specific material outperforms generic references because it captures your failure modes and conventions, not the LLM's training-data average. Asking for "one example of the output you'd actually use" is often the single highest-leverage question you can pose.
• Run the task end-to-end together first, then crystallise. When the user has a vague intent but no prior workflow or artefacts to mine, don't draft from priors. Offer to do one real instance of the task with them in this session; the corrections, decisions, and choices that emerge from that pass are what you crystallise into the skill. This is often the right answer rather than a fallback — running the task surfaces edge cases and project-specific facts that wouldn't have come up in abstract discussion.

If you can't honestly point at a source for the expertise, say so plainly. Don't bluff a skill out of generic priors — it'll trigger fine and produce nothing.

Then fill the gaps:

1. What should this skill enable Claude to do?
2. When should it trigger? (what phrasings, what contexts, what near-misses to avoid)
3. What's the expected output? (a file? inline analysis? a specific format?)
4. Should we set up explicit test cases? Skills with verifiable outputs (file transforms, data extraction, deterministic workflows) benefit. Skills with subjective outputs (writing style, creative work) usually don't — qualitative review is fine. Default-suggest based on the skill type but let the user decide.

Confirm what you've understood before drafting. A 30-second confirmation saves a 5-minute rewrite.

Step 2 — Write the SKILL.md

Anatomy

skill-name/
├── SKILL.md             (required)
│   ├── YAML frontmatter (name, description required)
│   └── Markdown instructions
└── Bundled resources    (optional)
    ├── scripts/         executable code for deterministic/repetitive tasks
    ├── references/      docs loaded into context only as needed
    └── assets/          files used in output (templates, icons, fonts)

Progressive disclosure (this is the load-bearing concept)

Skills load in three layers:

1. Metadata (name + description) — always in the model's context. Keep small (~100 tokens). This is what makes the skill trigger.
2. SKILL.md body — loaded when the skill triggers. Aim for under 500 lines (~5,000 tokens). Anything longer should be split into references.
3. Bundled resources — never auto-loaded; read only when SKILL.md tells the model to, and only on demand.

Every line in SKILL.md is paid for in every invocation; every line in references/ is free until needed. Tell the model when to load each reference ("read references/api-errors.md if the API returns a non-200 status code") — without the trigger, progressive disclosure isn't actually happening, it's just buried content.

If a skill supports multiple variants (e.g. multiple frameworks), organise by variant:

cloud-deploy/
├── SKILL.md            (workflow + which reference to load when)
└── references/
    ├── aws.md
    ├── gcp.md
    └── azure.md

Keep file references one level deep from SKILL.md. Avoid nested reference chains.

Frontmatter

---
name: kebab-case-name
description: What the skill does AND when to use it.
---

Allowed fields: name, description, license, allowed-tools, metadata, compatibility.

• name — required, kebab-case, max 64 chars, no leading/trailing or consecutive hyphens. Must match the parent directory name — agents key off the directory name. Cannot contain the reserved word claude anywhere in the name — skills run inside Claude, so claude-* names collide with the product and muddy triggering. Name the skill after what it does, not the model running it.
• description — required, max 1024 chars, no angle brackets. The primary triggering mechanism. See the description section below.
• compatibility — optional, max 500 chars. Use when the skill assumes something non-default about the environment. For skills built here, Designed for the Claude chat sandbox (Code Execution / File Creation) is a reasonable string when the workflow depends on the sandbox tools.
• license, metadata, allowed-tools — usually omit unless you have a specific reason.

Draft the description after the body settles, not before. The body is where you discover what the skill can actually claim to do — what gotchas it covers, where it's prescriptive vs flexible, what it deliberately doesn't handle. A description written first tends to overpromise; one written last reflects what's really in there. A short placeholder is fine while you draft.

Body — add what the model lacks, omit what it knows

Imperative tone. The body's job is to teach the model what it wouldn't get right unaided. For each section, ask: would the model produce the wrong output without this instruction? If no, cut it. Don't explain what a PDF is, how HTTP works, or what a database migration does — the model knows.

Explain the why behind instructions. Modern models reason well from rationale and badly from rigid lists of MUSTs. If you find yourself writing ALWAYS or NEVER in all caps, treat it as a yellow flag — usually you can reframe with an explanation and get better behaviour.

Calibrating control

Not every part of a skill needs the same level of prescriptiveness. Match specificity to fragility.

• Give the model freedom where multiple approaches are valid and variation is fine. Describe what to look for, not exact steps. A code-review skill should list the categories of issue to check, not script every cursor movement.
• Be prescriptive where the operation is fragile or the sequence matters. "Run exactly this command. Do not modify the flags." is appropriate for a migration step; it's overkill for "summarise this report."
• Provide defaults, not menus. When several tools could work, pick one and mention alternatives briefly. "Use pdfplumber for text extraction; for scanned PDFs requiring OCR, use pdf2image + pytesseract instead" beats listing four equal options.
• Teach procedures, not specific answers. A skill should explain how to approach a class of problems, not produce the answer to one instance. The approach generalises; the specific answer becomes dead weight on the next task.

Reusable patterns

Techniques worth reaching for. Not every skill needs all of them.

Gotchas section. Often the highest-value content in a skill. Concrete corrections to mistakes the model will otherwise make — environment-specific facts that defy reasonable assumptions:

## Gotchas
- The `users` table uses soft deletes. Queries must include
  `WHERE deleted_at IS NULL` or results will include deactivated accounts.
- The `/health` endpoint returns 200 as long as the web server is up,
  even if the database connection is down. Use `/ready` for full health.

When the user corrects a mistake during testing, that correction probably belongs in Gotchas. It's the most direct iterative-improvement mechanism a skill has.

Output templates. When a specific output format matters, show the template inline. Models pattern-match against concrete structures far more reliably than they follow prose descriptions of structure:

## Report structure
# [Title]
## Executive summary
## Key findings
## Recommendations

Checklists for multi-step workflows. Help the model track progress and avoid skipping steps with dependencies or validation gates.

Validation loops. Make the model verify its own work: do, validate, fix, repeat until the validator passes. Particularly powerful when paired with a bundled script that runs the check.

Plan-validate-execute. For batch or destructive operations: have the model write an intermediate plan to a structured file, validate the plan against a source of truth, then execute. The validation step catches errors before they hit the real target.

Bundled scripts for repeated logic. If across multiple test runs the model keeps reinventing the same helper — a chart builder, a CSV cleaner, a docx writer — write it once, drop it in scripts/, and reference it from SKILL.md. Saves every future invocation from reinventing it. See Designing bundled scripts below.

Principle of lack of surprise

A skill's contents should not surprise the user given its description. Don't go along with requests to create misleading skills, skills that facilitate unauthorised access, exfiltrate data, or otherwise behave maliciously. Roleplay-as-X skills are fine.

Step 3 — Know the sandbox the skill will run in

Skills produced here run in claude.ai chat with Code Execution / File Creation enabled — the same sandbox you're in now. Bake the constraints into the skill so it doesn't ask the model to do impossible things.

What the skill can rely on:

• Bash and Python in a Linux container (Ubuntu 24, working dir /home/claude)
• File creation, viewing, editing tools
• Standard Claude tools: web_search, web_fetch, image_search, etc.
• uv is installed (useful for self-contained Python scripts — see below)
• The model itself is capable of multi-step reasoning

What the skill cannot do:

• No subagents. Single agent, single session. Workflows that "spawn a worker" don't translate. If the original workflow had parallel work, restructure as sequential.
• No claude CLI. That's Claude Code only. Skills that rely on claude -p ... won't run here.
• No persistent state between sessions. /home/claude is wiped at end of conversation. If the skill needs to remember something across runs, surface it as a file the user keeps.
• Restricted bash network. Outbound HTTP from bash is whitelisted (pypi, npm, github, anthropic.com, etc.). For arbitrary web content the skill should use web_fetch / web_search, not curl.
• Some mounts are read-only: /mnt/skills/public, /mnt/skills/private, /mnt/skills/examples, /mnt/user-data/uploads, /mnt/transcripts. Copy first if you need to modify.

Output / file conventions:

• User-uploaded files land in /mnt/user-data/uploads/. Skills that read user files should look there.
• For files the user needs to see or download, write to /mnt/user-data/outputs/ and call the present_files tool. Nothing else makes a file visible to the user.
• For pip installs in skill instructions, use pip install <pkg> --break-system-packages. Skills should include the flag, not omit it.

When uncertain about a capability, have the skill check rather than assume.

Designing bundled scripts

When you put a script in scripts/, design it for an agent caller, not a human at a terminal. The model reads stdout/stderr to decide what to do next; design choices directly shape behaviour.

• Non-interactive only. No input() calls, no TTY prompts, no confirmation menus. The model can't respond — the script will hang. Take everything via CLI flags, environment variables, or stdin.
• Useful --help. This is how the model learns the interface. One-line description, the flags, a usage example. Keep it concise — the output lands in the model's context.
• Helpful errors. Say what went wrong, what was expected, what to try. Error: --format must be one of: json, csv, table. Received: "xml" is worth ten of Error: invalid input.
• Structured stdout, diagnostics on stderr. JSON or TSV on stdout, progress messages and warnings on stderr. Keeps output parseable while preserving diagnostic info.
• Idempotent where possible. "Create if not exists" is safer than "create and fail on duplicate." The model may retry.
• Predictable output size. Tool output is often truncated past ~10–30K chars. For large outputs, default to a summary and support --output <file> for full results.

For Python scripts, two viable patterns:

• Inline dependencies via PEP 723 + uv run — cleanest for self-contained scripts. Declare deps in a # /// script block at the top, run with uv run scripts/foo.py. uv is available in the chat sandbox; isolated env, no system pollution. Pin versions.
• System Python + pip install <pkg> --break-system-packages — the documented fallback when you want the script to just work without uv. The --break-system-packages flag is required in this sandbox; skills should include it.

Don't add a scripts/ directory speculatively. Add one when test runs reveal the model rewriting the same helper repeatedly.

Step 4 — Test in this session

You don't have subagents, so you can't spawn an independent Claude-with-skill the way Claude Code does. The pragmatic workflow:

1. Write 2–3 realistic test prompts. The kind of thing a real user would actually type — rough phrasing, casual language, missing details, plausible noise. Not abstract test fixtures.

2. Run them. For each prompt: re-read the SKILL.md fresh from disk (view it) before executing, then run the prompt as if you were just invoked. This isn't blind — you wrote the skill — but it's a sanity check, and human review compensates.

3. Save artefacts the user should see to /mnt/user-data/outputs/ and present_files them. Show outputs inline and ask: "Here's what I produced for prompt 1 — anything off?"

For verifiable outputs (file exists, contains expected column, parses cleanly), write a small assertion script and run it. More reliable than eyeballing, reusable across iterations. For subjective outputs (style, taste), skip assertions and rely on judgment.

Write assertions after the first run, not before. You usually don't know what "good" looks like until you've seen what the model produces unaided. Strong assertions are specific and observable ("chart has labelled axes," "report includes at least 3 recommendations"). Avoid vague ("output is good") or brittle ("uses exactly the phrase 'Total Revenue: $X'").

Optional baseline check for higher-stakes skills. Before declaring the skill is adding value, run one of the test prompts in a way that doesn't load the skill — e.g. in a fresh conversation, with the skill not installed. If the model produces an acceptable output anyway, the skill may not be earning its context-window cost. The agentskills.io workflow formalises this as with_skill vs without_skill comparison; in this session, treat it as a sanity check, not a benchmark.

Optionally save test prompts as evals/evals.json inside the skill folder so they're easy to rerun later:

{
  "skill_name": "example-skill",
  "evals": [
    {
      "id": 1,
      "prompt": "User's task prompt",
      "expected_output": "Description of expected result",
      "assertions": ["Specific checkable claim", "Another one"],
      "files": []
    }
  ]
}

The evals/ directory is excluded from the packaged .skill by default — test fixtures don't ship to users.

Step 5 — Improve based on feedback

The heart of the loop. The user has seen the outputs; now they tell you what's wrong. How you respond matters more than how you wrote the first draft.

Three sources of signal:

• Failed assertions point to specific gaps — a missing step, an unclear instruction, a case the skill doesn't handle.
• Qualitative feedback from the user points to broader quality issues — wrong approach, poorly structured output, technically correct but unhelpful.
• Execution traces (your own work during testing) reveal why things went wrong. If you ignored an instruction, it may be ambiguous. If you spent time on unproductive steps, the corresponding section of the skill is the culprit.

Generalise from feedback. The skill will run on thousands of future prompts, not just the 2–3 you tested. If the user says "the chart is missing axis labels," don't add "remember to add axis labels to charts" — that's overfitting. Ask what general principle the feedback points at. Maybe "before showing a chart, sanity-check it would make sense to a stranger." The general version transfers; the specific patch is dead weight.

Keep the prompt lean. Read transcripts of test runs, not just final outputs. Wasted steps usually trace back to specific sections of the skill — cut, don't pile on. If pass rates plateau while you keep adding rules, the skill may be over-constrained.

Explain the why. Today's models reason well from rationale. Heavy-handed MUSTs often perform worse than a paragraph explaining what we're trying to achieve and why. If feedback is terse or frustrated, do the work to understand what the user actually meant, then transmit that understanding to the skill.

Bundle repeated work. If every test run independently writes the same helper script, that's a strong signal to bundle the script. Write it once, drop in scripts/, point the skill at it.

After improving: rerun the test prompts, show the new outputs, gather feedback. Stop when the user's happy, when feedback dries up, or when you're not making meaningful progress.

Step 6 — Validate and package

Validate the skill structure:

python -m scripts.quick_validate /home/claude/<skill-name>

This checks frontmatter required fields, kebab-case name, that name matches the parent directory, length bounds, and no unexpected keys. Fix anything it flags before packaging.

Package into an installable .skill file. Run from the chat-skill-creator directory so the relative imports resolve:

cd <chat-skill-creator-directory>
python -m scripts.package_skill /home/claude/<skill-name> /mnt/user-data/outputs

Runs validation again, then zips the skill folder into <skill-name>.skill. The evals/ directory and build artefacts (__pycache__, .pyc, .DS_Store) are excluded automatically.

Hand it over by calling present_files on the .skill file. Briefly tell the user how to install it — the .skill file is a zip; users install through claude.ai's skill management UI.

Updating an existing skill

If the user wants to update a skill that's already installed:

• Preserve the original name. Use the same directory name and name: frontmatter — if the installed skill is research-helper, output research-helper.skill, not research-helper-v2. Otherwise the user ends up with two skills competing on triggering.
• Copy to a writeable location before editing. Installed skills under /mnt/skills/ are read-only. Copy to /home/claude/<skill-name>/, edit there, package from that copy.
• If the user supplied the old version as an upload, unzip it from /mnt/user-data/uploads/ into /home/claude/ first.

Writing a strong description

The description decides whether a skill triggers. After the body is in good shape, do a focused pass on the description.

What helps triggering accuracy:

• Imperative phrasing. "Use this skill when..." not "This skill does...". The model is deciding whether to act; tell it when to act.
• User intent, not implementation. Describe what the user is trying to achieve, not internal mechanics. The model matches against what the user asked for.
• Lead with verb + object. "Create X" / "Convert Y to Z" / "Analyse W." Don't bury the action behind preamble.
• Name the contexts where the skill should fire, especially indirect ones: "Use whenever the user mentions X, Y, Z — even if they don't say 'skill name.'"
• Disambiguate scope to avoid bleeding into adjacent skills. "For .pptx files specifically, not general slide content."
• Be pushy. Models tend to undertrigger skills. Counteract by spelling out casual / indirect phrasings.
• Stay under 1024 characters. Descriptions tend to grow during optimisation; trim back if needed.

Caveat on triggering: simple one-step requests ("read this PDF") may not trigger a skill even when the description matches perfectly — the model can handle them with basic tools. Skills earn their activation on tasks that need specialised knowledge or workflow. If your skill is for a one-step transform that the model already does well, it may not need to be a skill.

Sanity-check the description with 10–15 realistic queries — a mix of should-trigger and should-not-trigger. The interesting negatives are near-misses: queries that share keywords or concepts with your skill but actually need something different. "Write a fibonacci function" tells you nothing about a PDF skill's triggering. "Convert this Word doc to a PDF" — when the skill is only for extracting from PDFs — is the kind of negative that exposes scope problems.

Bad: "Format this data"

Good: "ok so my boss just sent me this xlsx file (its in my downloads, called something like 'Q4 sales final FINAL v2.xlsx') and she wants me to add a column for profit margin"

If you see patterns where the description fails (consistently misses certain phrasings, or triggers on adjacent things), rewrite and re-check. Two or three iterations usually gets somewhere good.

For more rigorous description optimisation, the agentskills.io guide describes an automated train/validation-split workflow with scripted trigger-rate measurement. That requires running many trials non-interactively and isn't practical in a single chat session, but users with Claude Code can run it post-export — worth flagging if they have a high-stakes skill where triggering accuracy really matters.

Wrap-up checklist

Before declaring done:

• SKILL.md frontmatter has name and description, validates cleanly
• name matches the parent directory name
• Body is lean, imperative, explains why not just what
• Description is imperative, scoped, under 1024 chars
• Bundled resources (if any) are referenced from SKILL.md with clear guidance on when to load them
• Any bundled scripts are non-interactive and document their interface
• At least 2 test prompts have been run; user has seen outputs and signed off
• quick_validate.py passes
• .skill file is in /mnt/user-data/outputs/ and presented via present_files

Good luck.