Skill Builder

Overview

Produces a tested, reusable agent skill for the user's own use case, from a description, an agent prompt, or an existing workflow.

Workflow

<Definition - Quick Skills Standard> The Amazon Quick agent skills standard uses a prescriptive standard for creating a skill directory, structured prompting with XML tags, and Amazon Quick's requirements. </Definition - Quick Skills Standard>

<Definition - Skill Directory Structure> A skill is a directory containing:

• SKILL.md: The only required file. Contains YAML frontmatter (metadata for discovery) and a markdown body (instructions for execution).
• scripts/: Executable code the agent runs during a workflow step. Self-contained, callable, returns a result. Example: scripts/analyze_logs.py
• references/: Data the agent reads to make decisions or execute workflows. Example: references/ticket-routing.md
• assets/: Static resources that shape or appear in the deliverable. Templates the agent fills in, schemas to validate against, images to include. The user receives these. Example: assets/my-data.json
• evals/: Test cases that verify the skill works correctly. Contains evals.json, eval output, and optionally evals/files/ for test input data.
• README.md: Optional human-facing overview, pre-requisites, installation, and getting-started. Mandatory when the skill has any dependency (Rule 25). Its presence is signaled by the readme frontmatter field. </Definition - Skill Directory Structure>

<Definition - Frontmatter> The YAML block between --- delimiters at the top of SKILL.md. All possible fields:

• name (required): Kebab-case identifier. Lowercase + hyphens only, max 64 chars. Must match directory name.
• display_name (optional): Human-friendly name shown in UI.
• icon (optional): Emoji identifier.
• description (required): Max 1024 chars. What the skill does and when to use it. Imperative phrasing, include trigger keywords. When a user asks what version of the skill they are on, refer them to the last_updated field.
• created_date (required): ISO date the skill was first created. Format: YYYY-MM-DD.
• last_updated (required): ISO date the skill was last modified. Format: YYYY-MM-DD.
• tools (optional): Built-in Amazon Quick tools this skill calls directly (e.g., file_read, web_search). Anything that needs an external integration (a connector, local MCP, or coding agent) is documented in the README ## Pre-requisites, not here.
• readme (optional): Instruction for the agent about the skill's README.md, present only when a README exists. It is the machine-readable signal that the agent must read the README before running, which is mandatory when the skill has any dependency (Rule 25). The README ## Pre-requisites declares each dependency (type, which connector, cloud or local runtime, required or optional) and ## Installation points to the reference docs in <Resources>. Example: readme: "Read README.md before running. Its ## Pre-requisites lists required connectors; verify each is available and stop if a required one is missing."
• preferred_model (optional): Recommended model tier: fast, balanced, or smart. If omitted, the skill runs on the user's session tier.
• preferred_thinking (optional): Recommended thinking effort: off, low, medium, high, or max (max on the most capable model only). Advisory, like preferred_model.
• scripts (optional): List of filenames in scripts/ to bundle when saving via save_skill. Example: scripts: [analyze_logs.py, format_output.py]
• checksum (required): Integrity digest of the skill's core files (SKILL.md, README.md, scripts/, references/, assets/; evals/, tests/, and transient files excluded), in the form sha256:<hex>. Generated by scripts/create_checksum.py (its single owner), which excludes this line from its own hash so writing it back is idempotent. Regenerated on save; check_skill.py requires it and verifies it is present and well-formed on audit.
• inputs (optional): Parameters that vary between invocations. Each input has:
  • name (required): identifier used as {{name}} placeholder
  • description (required): what this input is for
  • type (optional): string, url, path, number, choice, boolean
  • options (required if type is choice): list of valid values
  • required (optional): true/false
  • default (optional): value if not provided </Definition - Frontmatter>

<Definition - Progressive Disclosure> Skill description loaded at startup for all skills. Full SKILL.md loaded on activation. Other files loaded on demand during execution. </Definition - Progressive Disclosure>

<Definition - Evals> Test cases used to verify a skill behavior. Stored in evals/evals.json.

The file is a JSON object with:

• skill_name (required): name of the skill being tested.
• evals (required): list of test cases. Each case is an object with:
  • id (required): numeric identifier.
  • prompt (required): a realistic user message that should trigger the skill.
  • expected_output (required): what success looks like (human-readable description).
  • assertions (optional): list of verifiable checks. Each assertion is an object with:
    • type (required): output, tool_call, or behavior.
    • check (required): what to verify. Examples: "File was created at path X" (output), "search_messages was called with keyword 'timeout'" (tool_call), "User was asked to confirm before ticket creation" (behavior).
  • files (optional): list of input file paths relative to evals/files/. </Definition - Evals>

<Definition - XML Blocks> Each block type and what belongs in it. Some carry a "Test:" placement check, a question an author asks to decide whether a piece of content belongs in that block. It is an authoring aid, not something the running agent executes.

• <Identity>: Who the agent IS. Posture, expertise, disposition, NOT a restatement of the task. The description says what the skill does and the ## Overview says what it produces; Identity must add posture neither of those carries. Test: "If asked 'who are you?', could you answer with this?"
• <Goal>: Success criteria. Measurable outcomes you could evaluate a run against.
• <Rules>: Your behavioral constraints on the agent. Test: "Would the agent's default behavior violate YOUR standards without this?"
• <Definitions>: Terms the agent would otherwise misinterpret.
• <Agent Annotations>: How to interpret step prefixes and notation.
• <Tools>: Available tools declared inline in the body. Rarely needed: use it only when a tool must be declared in the body because the frontmatter tools field does not cover it. Most skills declare tools in frontmatter and omit this block.
• <Gotchas>: Non-obvious environment facts that contradict likely assumptions. Test: "Would the agent make a FACTUAL ERROR without this?"
• <Instructions>: Wrapper containing all <Workflow> blocks.
• <Workflow - X>: Step-by-step procedures using [Agent]/[Ask user]/[Decide] prefixes. Required inline attributes: description="...", tools=[built_in_tool, ...], triggers=["when X", "when Y"]. Optional advisory execution hints (guidance to the orchestrating agent, not service-enforced): background=true if the workflow's work should run as a spawned task, preferred_model=fast|balanced|smart, preferred_thinking=off|low|medium|high|max. Test: "Does this tell the agent what to DO in sequence?"
• <Templates>: Output format templates referenced by workflows on-demand. Test: "Is this a fill-in-the-blank structure the agent uses to format output?"
• <Resources>: Lookup data the agent references during execution. Tables, URLs, channel maps. Test: "Is this data to LOOK UP, not internalize?" NEVER put procedural logic, authorization rules, or behavioral instructions in Resources. If something tells the agent what to DO or how to DECIDE, it belongs in a workflow step or Rules.

Block ordering (top to bottom): Identity, Goal, Definitions, Rules, Agent Annotations, Gotchas, Instructions (Workflows), Templates (if needed), Resources (if needed). The agent reads sequentially: context and constraints before procedures, lookup data last. </Definition - XML Blocks>

<Definition - Description Writing Formula> Format: [What it does, one sentence] + [Trigger phrases: "Use when asked to..." with 5-8 quoted variations] + [Edge cases: "or any..." catch-all]. This skill's own description is a working example. </Definition - Description Writing Formula>

<Workflow - Router description="Determine what the user needs and dispatch to the correct phase." tools=[] triggers=["User asks to build, create, modify, save, test, or audit a skill", "sb help or any sb command"]

1. [Decide] What is the user asking? Validate: Exactly one path is chosen. If ambiguous, ask user to clarify before proceeding.
   • Create a new skill → <Workflow - Plan>
   • Modify an existing skill → <Workflow - Plan> (load existing skill as context first)
   • Convert a just-completed workflow into a skill → <Workflow - Plan> (use extract_session_data in step 3 to pull tools and context from the live session)
   • Continue building a skill already planned → <Workflow - Build>
   • Save a skill that's ready → <Workflow - Save>
   • Test a skill → <Workflow - Eval>
   • Check a skill against the standard → <Workflow - Audit>
   • sb help, a bare sb, or "what can skill-builder do" → <Workflow - Help>
   • A specific sb command dispatches straight to its workflow: sb plan → <Workflow - Plan>, sb build → <Workflow - Build>, sb save → <Workflow - Save>, sb eval → <Workflow - Eval>, sb audit → <Workflow - Audit> </Workflow - Router>

<Workflow - Help description="Render the skill-builder command menu, or dispatch a specific sb command." tools=[] triggers=["sb help", "sb", "User asks what skill-builder can do or how to use it"]

1. [Agent] Render <Template - Help Menu> populated for skill-builder, then stop and wait for the user's choice. The Router sends only menu requests here (sb help, a bare sb, or "what can skill-builder do"); a specific sb command is dispatched by the Router straight to its workflow. Validate: the menu lists every command with a one-line description. If fails: re-render from <Template - Help Menu>. </Workflow - Help>

<Workflow - Plan description="Research, identify requirements, and determine the approach." tools=[file_read, file_rag_search, web_search, extract_session_data] triggers=["Create a new skill", "Modify an existing skill", "Convert a workflow into a skill"]

1. [Agent] Set the conversation objective with a ready-to-copy /goal prompt. From the user's opening description of what they want, draft a one-line objective and output it as a single fenced code block containing only the /goal ... line, preceded by exactly one lead-in sentence: "Copy this to set the goal of the conversation (edit as needed):". Output nothing after the code block. /goal is Amazon Quick's native command that runs the session in goal-completion mode, so the agent works until the objective is met and self-checks before handing back. If the user opened cold with no description, ask for the objective in one sentence first, then produce the block. Example output:

   /goal Build a skill that <does X> so that <outcome>.
   

   Validate: output is one lead-in line plus one code block containing only a /goal line; no trailing prose. If fails: re-render as the lead-in line plus the code block only.

2. [Ask user] What should this skill do? Accept:

   • A description of the desired behavior
   • An agent prompt to convert
   • A file path to a document or runbook Validate: User provides at least one of the above. If vague, ask a clarifying question. If fails: Re-ask with examples of each input type.

3. [Agent] If a file path was provided, read it via file_read. Validate: File content loaded successfully and is non-empty. If fails: Report "file not found" or "file empty" to user and ask for alternative input.

4. [Agent] Introspection. Check what's already available:

   • Tools in the current session (core registry + connected connectors)
   • For each connector the skill will use: introspect its available actions and required parameters. Understand what data the connector needs (recipients, subject lines, body content, channel names, ticket fields, etc.) so the skill collects the right inputs from the user and workflow steps provide sufficient context for the agent to call the connector correctly at runtime.
   • Skills that already exist (could we extend one instead of creating new?)
   • Knowledge graph and memories for prior work on this topic
   • Data sources and integrations the user has access to Validate: At least one relevant tool or data source identified. For each connector dependency, the agent can describe what parameters it requires. If fails: Note "no existing tools found" and continue. The skill may need new MCPs.

5. [Agent] Public research. Search the web for:

   • Relevant MCPs (local or remote) that might be needed
   • Existing patterns, implementations, or libraries Validate: Search returned results (even if none are relevant). If fails: Note "no relevant patterns found online" and continue with what's available.

6. [Decide] Are all required tools/MCPs/connectors available? Validate: A clear yes/no determination is made for every dependency. For each dependency, record its type, which connector, runtime (cloud or local), and required or optional, so Build can write the README ## Pre-requisites per <Template - README>.

   • All available → continue to step 6.
   • Exists but not connected → tell user to enable in Settings. Pause.
   • Doesn't exist → present options: build with placeholder, build the MCP first, or scope to what's available.

7. [Ask user] Present findings: available tools, dependencies identified, any blockers. Confirm direction. Validate: User confirms or redirects. Do not proceed without explicit approval. If fails: Re-present with simpler summary if user seems confused.

8. [Agent] Suggest 2-3 skill name options. Names must be:

   • Kebab-case per <Definition - Frontmatter>
   • Descriptive of what the skill DOES, not what it's about
   • Short (2-3 words, e.g., quick-ticket-agent, field-signal-harvester) Validate: all names are valid kebab-case per <Definition - Frontmatter>. If fails: Regenerate compliant names.

9. [Ask user] Confirm: name, trigger phrase, inputs, dependencies. User always has final say on naming. Validate: User explicitly confirms or provides alternatives. If fails: Do not assume silence is approval. Re-ask.

10. [Ask user] How do you want to build this?

    • Section by section (review each block together)
    • Draft then review (I write a first pass, we iterate) Validate: User picks one approach. If fails: Default to "section by section" (safer, more checkpoints).

11. [Agent] Write the description per <Definition - Description Writing Formula>. Validate: Description follows <Definition - Description Writing Formula>. If fails: Rewrite until it does.

12. [Ask user] Approve description before proceeding to <Workflow - Build>. Confirm the description and the planned skill serve the recorded /goal objective. Validate: Explicit "yes" or equivalent from user. If fails: Revise description per user feedback and re-present. </Workflow - Plan>

<Workflow - Build description="Construct the skill section by section or as a draft, per the approach chosen in Plan." tools=[file_read, file_write, folder_create, get_current_time] triggers=["Called from <Workflow - Plan> after user approves direction"]

1. [Agent] Identify blocks needed from <Definition - XML Blocks>. Every skill gets: Identity, Goal, Rules, Agent Annotations, Instructions (with at least one Workflow). Optional (include only if needed): Gotchas (only when real non-obvious facts exist, do not invent filler), Definitions, Templates, Resources. Validate: The 5 required blocks are present, each optional block included only when it earns its place. If fails: Cross-reference <Definition - XML Blocks> and add missing blocks.

2. [Agent] Implicit assumptions audit. Scan every workflow step and ask: "Does this reference a value that only works in the skill author's environment?" Catch ALL of:

   • File/folder paths (Obsidian vaults, output directories, config locations)
   • URLs and endpoints (dashboards, wikis, internal tools)
   • Channel/group names (Slack channels, Teams groups, distribution lists)
   • People/team names and aliases
   • Routing tables, category lists, priority tiers
   • Tool configurations or API-specific values
   • Vague integration references (e.g., "send a notification" without specifying which integration or what data to include in the message)
   • Hardcoded output file paths (the skill must ask the user or accept a path as input)
   • Over-prescriptive scope (see Rule 19)
   • Brittle tool/API dependencies (see Rule 21) For each one found: it MUST be placed in <Definitions> (short values), <Resources> (lookup tables), or references/ (large datasets). No value that varies by environment or breaks over time can appear only inside a workflow step. Validate: Produce a list of all findings with proposed placement. Include any extensibility or longevity concerns. If fails: Re-scan. If truly none found, document "no issues detected" and continue.

3. [Ask user] Present the list of environment-specific values discovered. Confirm each value and where it should live (Definitions, Resources, or references/ file). Validate: User confirms every value's placement. If fails: Adjust placement per user feedback and re-confirm.

4. [Agent] Identify supporting files per <Definition - Skill Directory Structure>:

   • Lookup tables, routing maps, URL lists → references/
   • Templates, schemas, static resources → assets/
   • Executable helper scripts → scripts/
   • Everything else stays in SKILL.md body Validate: Every file is assigned to exactly one directory. If fails: Re-categorize using the taxonomy test for each directory type.

5. [Agent] Write the SKILL.md body: ## Overview, then ## Workflow holding the blocks in the canonical order from <Definition - XML Blocks>. Validate: ## Overview, ## Workflow, and the 5 required blocks are present, any blocks in canonical order, body under 1024 lines. If fails: Remove redundancy, push detail to references/ until under limit.

6. [Agent] Write supporting files per <Definition - Skill Directory Structure>. Any code written (scripts, example snippets) follows the sandbox coding standards per Rule 26. Scaffold every reference file and script with the generators per references/reference-file-standard.md so each carries a compliant header. Validate: Each file exists at its declared path and is non-empty, each carries a compliant header per references/reference-file-standard.md, and any generated code follows references/quick-sandbox-code-standards.md and its language companion with every import present in the applicable manifest. If fails: Regenerate missing files or correct code that violates the sandbox standards.

7. [Decide] Does the skill depend on anything beyond built-in tools?

   • No: the README is optional. Offer one, then continue.
   • Yes: write a README per <Template - README> with every dependency in ## Pre-requisites (type, which connector, runtime, required or optional), set the readme frontmatter field, point ## Installation at the <Resources> docs, and generate a prerequisite-introspection step as the skill's first workflow action. Validate: every dependency appears in ## Pre-requisites, readme is set, and a prerequisite step exists that introspects each dependency and links the install reference for missing required ones. If fails: add the missing README section, readme field, or prerequisite step, then re-check.

8. [Agent] Generate the skill's help menu. Derive a short command prefix from the skill name (the words' initials, e.g. api-design-helper becomes adh). Map a command (<prefix> <verb>) to each user-invocable workflow, plus <prefix> help. Write a <Workflow - Help> into the built skill that renders <Template - Help Menu> populated with these commands, and wire the commands into the built skill's <Workflow - Router> and its triggers: menu requests (<prefix> help, a bare <prefix>) route to <Workflow - Help>, and each <prefix> <verb> routes straight to its workflow. Validate: the built skill has a <Workflow - Help>, a <prefix> help command, one command per user-invocable workflow, and its Router dispatches each <prefix> <verb> to the matching workflow. If fails: add the missing help workflow, commands, or router entries, then re-check.

9. [Think] Writing quality review. Read through all written content (Identity, Goal, Rules, Gotchas, Definitions, descriptions) and check:

   • Verify Rule 15 compliance: every factual claim verifiable, prose natural and direct, no invented "best practices" or uncited sources.
   • Does the Identity sound like a real person's job description, not a marketing blurb?
   • Do Rules state clear prohibitions, not vague aspirations?
   • Do Gotchas state surprising facts, not obvious things dressed up as warnings?

10. [Agent] Present full preview (SKILL.md + file tree) to user. Validate: Preview shows the complete SKILL.md content plus all supporting file paths. If fails: Ensure nothing was omitted. Re-present.

11. [Decide] User approves → proceed to <Workflow - Save>. User wants changes → adjust and re-preview. Validate: Clear approval signal from user. If fails: Ask "What would you like me to change?" and loop back to relevant step. </Workflow - Build>

<Workflow - Save description="Persist the skill directory and set metadata." tools=[get_current_time, save_skill, file_read] triggers=["Called from <Workflow - Build> after user approves preview", "User asks to save an existing skill draft directly"]

1. [Agent] If the skill was not built in this session (the user pointed at an existing draft), read that draft via file_read and show its content as the preview. An explicit save request is the approval per Rule 7, so continue without pausing. Validate: The SKILL.md content to save is loaded, from this session's Build or the provided draft. If fails: Ask the user for the draft path.

2. [Agent] Get current date via get_current_time. Validate: Date returned in ISO format. If fails: Use today's date from context as fallback.

3. [Agent] Set created_date (if new skill) and last_updated (always) in frontmatter. Validate: created_date and last_updated are set. If fails: Re-set manually.

4. [Agent] Call save_skill with the SKILL.md content and every companion file bundled in its scripts parameter as {"filename": "<skill-relative path>", "code": "<contents>"} (for example scripts/analyze.py, references/routing.md, assets/template.json). Bundling is the only way to write a skill's companion files: generic file_write into the skills directory is blocked (see Gotchas). A bare filename lands under scripts/; a relative path is preserved at that path. Validate: save_skill returns success and reports the bundled files as written. If fails: Check for validator errors (missing ## Overview or ## Workflow, a hallucinated tool, or an absolute or .. filename that rejects the whole save). Fix and retry once.

5. [Agent] Confirm save_skill persisted every companion file from the step 4 bundle. Do not fall back to file_write or folder_create for the skills directory (they are blocked there, see Gotchas). If the skill has a README (per Rule 25, mandatory when it has dependencies), it must be in the bundle and the readme frontmatter field set. Validate: every file declared in Build step 4 was saved at its intended path; if the skill has dependencies, the README is present with a ## Pre-requisites section and the readme field is set. If fails: re-call save_skill including the missing files in its scripts bundle.

6. [Agent] Regenerate the skill's checksum: run scripts/create_checksum.py so the stored checksum reflects the final files. Do this after all files are written and before the audit, so the audit sees a current checksum. Validate: create_checksum.py exits 0 and the checksum field is present in SKILL.md. If fails: re-run once; if it still fails, report the error to the user.

7. [Agent] Run <Workflow - Audit> automatically. Validate: Audit returns zero issues. If fails: Fix reported issues and re-audit until clean.

8. [Ask user] Generate eval test cases via <Workflow - Eval>? Validate: User responds yes or no. If fails: Default to generating evals (better to have them). </Workflow - Save>

<Workflow - Eval description="Run the eval-driven loop: test a skill against a baseline, grade with evidence, benchmark, and iterate." tools=[file_read, file_write, run_python, start_task, get_current_time] triggers=["User asks to test a skill", "Called from <Workflow - Save> step 6"] preferred_model=smart preferred_thinking=high

The full method lives in references/eval-loop.md. Read it before starting. Grading detail is in references/eval-grading.md, cross-run analysis in references/eval-analysis.md. This workflow only orchestrates the loop. The references carry the depth so this block stays lean.

1. [Agent] Read the target SKILL.md and references/eval-loop.md. Identify the key workflows to cover. Validate: SKILL.md loaded, at least one found, and eval-loop.md read. If fails: Check the path. If the skill is not saved yet, run <Workflow - Save> first.

2. [Agent] Phase 1 in eval-loop.md: write 2-3 varied prompts with expected_output into evals/evals.json. No assertions yet. Validate: evals.json is valid JSON per <Definition - Evals>, 2-3 cases, no assertions field yet. If fails: Fix the JSON and re-write.

3. [Ask user] Review the prompts. Add, remove, or adjust before any runs? Validate: User confirms or provides edits. If fails: Apply edits and re-present.

4. [Agent] Analyze before running (Rule 23). Read the full SKILL.md and reason directly, no code, about structure, step order, claim verifiability, and voice. Fix what you find before spawning any runs. Validate: Analysis is written and any blocking issues resolved. If fails: Resolve the issues. If the skill needs a rewrite, return to <Workflow - Build> first.

5. [Agent] Phase 2 in eval-loop.md: run each prompt twice as isolated background tasks (with_skill and baseline), sequentially (one start_task at a time, awaiting each), at the skill's own preferred_model/preferred_thinking, per the instructions there. Run the full set before any change, and reuse the cached baseline on reruns. Validate: Every prompt has a with_skill and a baseline run (or a cached baseline reused) with outputs and metrics.json. If fails: Re-spawn the missing runs. If a task errored, read its result and report before continuing.

6. [Age

…(truncated)