# Improve Ic Skill > Improve an existing skill in the IC Skills repo (skills.internetcomputer.org). Load this skill whenever asked to improve, fix, update, enhance, review, or add content to an existing skill at skills//SKILL.md — including adding pitfalls, updating canister IDs, fixing code examples, strengthening the description, or updating evals. Do NOT use for creating a brand-new skill (use skill-creator for that). Trigger examples: "the motoko skill is missing a pitfall", "update icp-cli for the new recipe format", "the ckbtc description is weak", "add an eval for the canister-security skill". - Skill: `dfinity/improve-ic-skill` (Agent Skill) - Install (CLI): `npx skillmds@latest add dfinity/improve-ic-skill` - Raw SKILL.md: https://api.skillmd.com/api/skills/dfinity/improve-ic-skill/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Security - Author: dfinity (https://skillmd.com/u/dfinity) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/dfinity/improve-ic-skill --- # Improve IC Skill A token-efficient workflow for improving an existing skill in the IC Skills repo. The guiding principle: **understand the problem first, change precisely, verify only what changed.** Evals exist as regression safety nets — run them selectively, not automatically on every pass. ## Step 0 — Understand the problem before touching anything **This is a hard stop. Do not read any file, run any command, or make any change until you have a specific problem statement from the user.** You need to know: 1. **What is the specific issue?** (e.g. "canister ID X is stale", "missing pitfall about Y", "description doesn't trigger on Z", "code example uses deprecated API W") 2. **Why does it matter?** (e.g. "agents generate broken code", "skill never triggers for this use case", "upstream sync added a new command we don't cover") If the request is vague ("improve the motoko skill", "the ckbtc skill feels weak") — ask: > "What specific issue should I focus on? For example: a missing pitfall, a stale canister ID, a broken code example, or a description that isn't triggering correctly." **If the user cannot or will not provide a specific issue — stop. Do not proceed.** Do not read the skill. Do not run commands. Do not "explore and find improvements yourself." Explain clearly: > "I need a specific problem to solve before I can start. Blind improvements risk changing things that don't need changing. To find a concrete issue, try: running `npm run validate`, checking `evaluations/.json` for failing cases, or pointing me to a GitHub issue or upstream diff." If the user pushes back, repeats the vague request, or expresses frustration — hold the position. The answer is always the same: no specific issue = no work started. The only exception: if the reason is unambiguous from context already in the conversation (a GitHub issue body, an upstream diff, a specific error message) — proceed directly to Step 1 without asking. ## Step 1 — Understand the skill Read `skills//SKILL.md` fully. Check: - What it covers and what it's missing - Whether it's upstream-tracked: `grep .claude/upstream.md` — if it appears, note which sections are owned by icskills (those cannot be overwritten from upstream) - Recent git history: `git log --oneline -5 -- skills//SKILL.md` ## Step 2 — Validate baseline ```bash npm run validate ``` Fix any errors before making content changes. Warnings are acceptable — don't chase them unless they're relevant to your improvement. ## Step 3 — Identify improvements Read with fresh eyes. Common high-value areas: **Pitfalls** — highest-value content; every pitfall documented prevents an agent hallucination. Look for: incorrect usage patterns you know about, error messages agents typically misread, wrong defaults, API gotchas. **Canister IDs and version numbers** — verify against mainnet and current release tags. Stale IDs silently break agent-generated code. **Code examples** — verify syntax compiles, APIs still exist, no deprecated patterns. **Description** — must state what it does, when to use it, AND when NOT to use it. Include specific keywords agents match on. A weak description means the skill never triggers. **Required frontmatter** — `metadata.title` and `metadata.category` are required by CI. Missing these blocks deployment. For upstream-tracked skills (`writing-motoko`, `migrating-motoko-actors`, `troubleshooting-motoko-migrations`, `reviewing-motoko`, `mops-cli`, `static-site`): read `.claude/upstream.md` carefully. Only modify icskills-owned sections freely; changes to shared content should also be filed upstream. ## Step 4 — Apply improvements Edit `skills//SKILL.md`. Stay focused — don't refactor things you weren't asked to touch. ## Step 5 — Validate again ```bash npm run validate ``` Fix any errors. Warnings from unrelated sections are fine to leave. Optionally run LLM quality scoring if the changes were substantial: ```bash skill-validator score evaluate --provider claude-cli skills/ ``` ## Step 6 — Check and update evals Open `evaluations/.json` at the repo root (not inside the skill directory). **If the file doesn't exist yet**, create it as part of this improvement session. A skill being improved is the right moment to seed its first evals. Start lean — 1-2 output_evals covering the most important pitfalls, and 2-3 trigger_evals for the description. Don't try to be comprehensive; a small, accurate eval file beats an overwhelming one that never gets run. For **upstream sync sessions** where no eval file exists: seed cases directly from the upstream diff — new commands, changed defaults, renamed APIs, and new pitfalls in the diff are exactly where agents will hallucinate without updated guidance. The diff is your best source. ```bash # Verify no eval file exists ls evaluations/.json 2>/dev/null || echo "No evals yet — create them" ``` **If the file exists**, review the existing cases alongside your changes. **Always update evals when:** - You added a new pitfall — pitfalls are exactly where agents hallucinate without guidance; every new pitfall deserves a case - You changed a command, API, or canister ID that an existing case now describes incorrectly — fix the case - You introduced a new behavior that agents would likely get wrong without explicit guidance **Keep the suite lean.** Don't add cases for things that are obvious or already well-covered. Each case is a future regression test that costs tokens to run. ## Step 7 — Run evals Two separate things happen here — don't conflate them: **A. Always run every case you added or changed (with baseline).** A new or edited eval case is code you just wrote. Running it confirms it actually passes with the skill and shows a real with-skill vs baseline delta. An unrun case can be mis-scoped — e.g. a prompt that says "just the command" paired with an expected behavior that requires an explanation will fail the with-skill run — and you'd be committing a silently broken regression test. Include these results in the PR. This is not optional, even when the change is "purely additive": additive means new cases, and new cases must be run. **B. Re-run untouched existing cases only when relevant.** Re-running the rest of the existing suite is a regression check — "did I break something that was already tested?" Do this when: - You changed or removed content that an existing eval explicitly tests (canister ID, command name, error message) - You rewrote a section that several existing evals cover - The description changed significantly — run trigger evals only - You're unsure whether a change is safe Skip re-running an existing case only when nothing you touched is content that case covers. Check what evals exist before deciding: ```bash node scripts/evaluate-skills.js --list ``` Run only the relevant subset: ```bash # Single eval by index (cheapest) node scripts/evaluate-skills.js --eval # Trigger evals only — for description changes node scripts/evaluate-skills.js --triggers-only # Run a case you added/changed WITH baseline (default) — this is what goes in the PR node scripts/evaluate-skills.js --eval # Skip baseline (--no-baseline) only for a quick correctness spot-check, not for PR results node scripts/evaluate-skills.js --eval --no-baseline # Full suite — only when you made broad changes across the whole skill node scripts/evaluate-skills.js ``` Eval format reference: ```json { "output_evals": [ { "name": "descriptive name", "prompt": "scoped prompt asking for one thing — specify what to exclude to keep it fast", "expected_behaviors": [ "Specific, checkable behavior the model should exhibit", "Another specific behavior" ] } ], "trigger_evals": [ { "query": "realistic user prompt that should trigger this skill", "expected_behavior": "should trigger" }, { "query": "realistic prompt that looks similar but should NOT trigger this skill", "expected_behavior": "should NOT trigger" } ] } ``` Keep prompts tightly scoped — open-ended prompts generate long responses and risk timeouts. Ask for one thing; explicitly exclude what you don't want ("just the function, no deploy steps"). ## Step 8 — Final check ```bash npm run validate ``` The skill is ready for a PR. Include a brief summary of what changed and, if you ran evals in Step 7, paste the relevant results collapsed in a `
` block (see [Submit a PR](../../../CONTRIBUTING.md#8-submit-a-pr) for the exact format). ## Upstream-tracked skills Before editing `writing-motoko`, `migrating-motoko-actors`, `troubleshooting-motoko-migrations`, `reviewing-motoko`, `mops-cli`, or `static-site`, read `.claude/upstream.md`. It lists which sections are icskills-owned. You can freely improve owned sections. For shared content, improvements should also be filed as issues upstream so they flow back on the next sync.