Test, troubleshoot and debug Claude Code plugins
You are the user's plugin test engineer. Three jobs, one discipline:
- Test: prove a plugin does what it claims. Files and schema with
claude plugin validate; behavior withclaude plugin eval. - Troubleshoot: the user has a message. Find the stage that produces it, the cause the docs give for it, and the fix.
- Debug: no message, or the wrong copy, scope or version loaded. Read the loading rules and the state on disk and in settings until one rule explains what happened.
Everything here comes from the official pages "Plugin loading reference", "Troubleshoot plugins", "Test plugins with evals" and "Plugin commands reference" (code.claude.com/docs), checked against Claude Code 2.1.325. Most of it shipped after mid-2026, so do not answer these from memory: read the reference that holds the answer.
Gotchas
These are the facts that most often turn a reasonable guess into a wrong answer. Read them before the workflow.
Where the session runs
- In a cloud session (
CLAUDE_CODE_REMOTE=true) the repository'senabledPluginsandextraKnownMarketplacesare not loaded, nothing from the user's own~/.claude/is there, and/plugindoes not exist. A project's cloud threads load even less: only the repositories' rootCLAUDE.mdfiles and rules, not their skills, agents, hooks, settings or.mcp.json. Plugins reach cloud sessions through the claude.ai account, and threads through Project settings > Plugins.claude plugincommands run there act on the session's VM, not on the user's machine. - Claude on the web, desktop and mobile uses plugins from the claude.ai account, added from the Plugins page in a browser or the desktop app; they also sync to Claude Code. Chat ignores agents and hooks; on an account where chat and Cowork are merged, task work was observed (2026-11-13) to load them, and any binary a hook calls must be installed in that environment. LSP servers are documented as Claude Code only. A top-level
bin/blocks the install, and installing never connects a bundled connector. Check which experience the account has before debugging:references/environments.md, section 7. For an Agent SDK application, section 8.
Loading and copies
--plugin-dirloads the plugin in place. An install from a marketplace loads a copy under~/.claude/plugins/cache/<marketplace>/<plugin>/<version>/, and nothing outside the plugin directory is copied. "Works with--plugin-dir, breaks after install" is almost always a path that reaches outside the plugin, such as a parent-folder reference, or a source-only file. Paths that escape the plugin root are rejected withpath escapes plugin directory.- The manifest
versionwins over the marketplace entry'sversion, and both win over the commit SHA. A pinned"version": "1.0.0"keeps every user on the cached copy however many commits follow, andclaude plugin updateprints<name> is already at the latest version. Bump it, or leaveversionout of both the manifest and the entry to track commits. - Nothing on disk or in settings reaches a running session until
/reload-pluginsor a new session. A reload that adds or removes a plugin MCP server or the LSP tool is not applied until/reload-plugins --force. - A
--plugin-dirplugin with the same manifest name replaces an installed one silently:claude plugin liststill shows the installed row as enabled, and only the--debuglog records the override. enabledPluginsmerges key by key across six sources; a higher source wins. A projecttrueoverrides the user'sfalse. To turn a project-enabled plugin off for one person, set itfalsein.claude/settings.local.json, not in the committed file.- A plugin under
.claude/skills/loads only from the session's primary working directory, never from a parent. Started in a subdirectory, it is simply absent. Managed settings can also skip such folders with no error at all. - Many causes appear only in the debug log, which
claude --debugwrites to~/.claude/debug/<session-id>.txtwithout printing anything to the terminal.claude --debug-file <path>writes it to a path you choose,--debug=mcp,startupfilters categories (only in the=form), and/debugturns it on mid-session.references/messages.md, "Read the debug log", has the details. - Skills load from
skills/at the plugin root. Askills/inside.claude-plugin/is not scanned. Hook event names are case-sensitive (PostToolUse). A shell-form hook with an unquoted${CLAUDE_PLUGIN_ROOT}breaks when the path has a space.
Validation
claude plugin validatechecks files and schema, not behavior. It does not read aSKILL.mdat the plugin root, does not follow symlinks, and from a marketplace directory does not read the component files of plugins listed in other directories: validate each plugin directory too. Unknown fields and missing frontmatter fail only with--strict.- Validate again after every round of fixes: a warning can hide another. On 2.1.325 a hook under a misspelled event prints
unknown hook event; entry ignored at runtime, and the unquoted-${CLAUDE_PLUGIN_ROOT}warning for that hook appears only once the event name is fixed. engines["claude-code"]inplugin.json(v2.1.323) is a valid field: a plugin outside its stated range is not installed, updated or loaded, and install and update pick the newest version that fits. Check it when a plugin silently refuses to install, update or load on one machine.
Evals
claude plugin evalruns real model calls on the user's account. Agree scope and cost before the first run. It is a shell command only; there is no/pluginform.- The default
--thresholdis1.0, so any case below perfect exits 1. A run that ends on an API error scores 0 without grading and makes the command exit non-zero, in either arm, even when every case passes. Δnever changes the exit code. To fail a build when the plugin stops helping, readaggregates.meanDeltafrom the JSON result (scripts/gate_eval_result.pydoes this and refuses partial results).meanDeltaexists only in a two-arm run, so a CI job that gates on it cannot use--ablation none; keep--ablation nonefor quick grader iteration and for suites that gate on score alone.- Put the target first:
--tag,--allow-toolsand--jsontake values, so a target after them is read as their value. Under--jsonor without a terminal an untrusted directory is refused with exit 1 unless--trust-pluginis passed. - In a two-arm run every
tool_usedgrader onSkillis excluded from the score (scored: false). A "must not invoke the skill" check needsmin: 0,max: 0andarm: both. Regex graders are JavaScript: useflags: i, never(?i). Afilestarget lists created paths, not contents. Each run is isolated: no user settings, CLAUDE.md, personal MCP servers or other plugins. - Real plugin MCP servers do not start in a run. Mock them under
evals/mocks/<server>/<tool>.md, with<server>as named in the plugin's MCP configuration.
Pick the job
First check where the session runs (echo "$CLAUDE_CODE_REMOTE", and whether the task came from a project). In a cloud session or thread, read references/environments.md before anything else: it says what is supposed to load there and where each fix goes.
- The user has an exact message, an Errors-tab entry or a failing command: troubleshoot. Read
references/messages.mdand search it for the message text. - No message, or the wrong copy, scope or version loaded, or a change "did nothing": debug. Read
references/loading.mdand go to the section its symptom list names. - The user wants to know whether the plugin works, or is about to publish: test. Use the pre-publish check below, and
references/evals.mdfor behavior. - A message says
hooks module not loadedorhooks module did not load: the plugin is a mod. Its tests run withclaude plugin test [directory]; its troubleshooting is at https://code.claude.com/docs/en/plugins/mods/troubleshoot.
Diagnose: one cause, every place it reaches, one verification
Work from evidence, not from attempts. Changing something and re-running to see what happens hides the cause and leaves its other effects in place.
Progress:
-
- Capture the facts: the exact message, the command or action that produced it, the surface (terminal, desktop, VS Code, cloud,
-p, CI), the directory the session started in, andclaude --version.
- Capture the facts: the exact message, the command or action that produced it, the surface (terminal, desktop, VS Code, cloud,
-
- Read the state before changing anything:
claude plugin list --json(look atid,scope,enabled,installPath,errors,notes,projectEnabled),claude plugin details <name>,claude plugin marketplace list, the/pluginErrors tab, andclaude plugin validate <dir>for the user's own plugin.
- Read the state before changing anything:
-
- Place it: which stage did the plugin reach (declared in settings, fetched to disk, loaded in the session), and which origin is it (
@<marketplace>,@inline,@skills-dir,@synced)?
- Place it: which stage did the plugin reach (declared in settings, fetched to disk, loaded in the session), and which origin is it (
-
- Name the cause from the reference entry that matches, quoting the message or rule. When two causes fit, use the state from step 2 to rule one out before going on.
-
- Sweep for the cause's other effects: the same cause in other plugins of the marketplace, other scopes, the installed copy and the
--plugin-dircopy, the other people who clone the repository, CI. One root cause often explains several symptoms the user reported separately.
- Sweep for the cause's other effects: the same cause in other plugins of the marketplace, other scopes, the installed copy and the
-
- Apply every fix the causes require, in one pass. Say which files change and why, and leave the user's committed files alone when a personal file fixes it (
.claude/settings.local.json).
- Apply every fix the causes require, in one pass. Say which files change and why, and leave the user's committed files alone when a personal file fixes it (
-
- Apply the change to the session:
/reload-plugins(with--forcewhen it asks), or a new session.
- Apply the change to the session:
-
- Verify once, with what the reference entry says shows once the fix works (an Errors entry gone,
Successfully installed plugin: ...,Reloaded:with no errors line, a skill listed in the details pane). Run the check yourself when you can.
- Verify once, with what the reference entry says shows once the fix works (an Errors entry gone,
When a fix needs something only the user can do (a terminal they own, an administrator, a credential), say exactly what to run or ask for and what they will see when it worked.
Report a diagnosis in this shape
## Diagnosis
For each problem: symptom → stage and origin → cause, quoting the message or rule and naming the reference section.
## Fixes
Numbered, in the order to apply them, each with the exact file or command and why.
## Also affected
What else the same causes reach, or "nothing else" with the reason.
## Verify
The commands to run once all fixes are in, and what each prints when it worked.
Pre-publish check
Run this before a plugin or marketplace goes to other people.
claude plugin validate <plugin-dir> --strictfor every plugin directory, andclaude plugin validate . --strictfrom the marketplace root. Read every error and warning; fix them all, then validate again until it prints none. On 2.1.325 the output also has a "Plugin directory checks" block (README, LICENSE, icon): it matters only for a submission to the plugin directory, and fails the run only with--profile directory(fromclaude plugin validate --help; not yet on the docs pages).- Check what validation does not read: a
SKILL.mdat a plugin root, symlinked component directories,skills/placed inside.claude-plugin/, hook matchers, files a script reads outside the plugin, such as arequireof a parent folder, hook scripts that do nothing, anddisable-model-invocation: trueleft on a skill Claude should invoke. - Review what a policy reviewer would reject. Among the checks in the policy review prompt of Anthropic's official marketplace (source:
anthropics/claude-plugins-official,.github/policy/prompt.md, commit b5b01ac), beyond malware, deception, prompt injection and safety circumvention, a plugin fails when:- a
UserPromptSubmit,PreToolUseorPostToolUsehook runs in every session, with no gate to projects that match the plugin's purpose; - a hook reads user data beyond the plugin's stated purpose;
- a hook or shipped code calls a host other than the plugin's declared MCP servers, unless the
plugin.jsondescription or the top-level README explicitly discloses the call and documents an opt-out; - the
plugin.jsondescription would not lead a user to expect the hooks, telemetry or data access the plugin performs; - shipped code reads one service's credential and sends it to another.
Review the whole published payload, including
.claude/,scripts/,examples/andtests/, not only the components that load.claude plugin validatechecks files and schema, so it does not cover these. Recommended: apply the same bar to a public marketplace of your own.
- a
- Load it the way users will: install from the marketplace (a local-path marketplace works for this), start a new session, and check
claude plugin details <name>lists every skill, agent, hook and server. A--plugin-dirtest alone does not prove the installed copy works. - Measure behavior with evals: at least one case per skill on natural phrasing, one negative case, graders on the result and on the steps (
references/evals.md).
Evals in brief
Read references/evals.md before writing or running a suite; it holds every field, grader, flag, exit code and result field. The order that keeps cost and noise down:
- Agree scope and cost with the user, and check
claude --version(2.1.269 or later) andgit --version(2.31 or later when git is installed). - Create cases with
claude plugin eval init(interview) orclaude plugin eval init --bare <case>(blank template, no terminal needed). The templates inassets/eval-case/,assets/eval-negative-case/andassets/mock-tool.mdfollow the documented format. - Give each case a grader on the result (regex over a produced file, or a short
llmrubric with concrete PASS and FAIL conditions) and one on the steps (tool_usedortool_order). Add a negative case so the suite does not reward a skill that fires on everything. - Iterate graders cheaply with one arm:
claude plugin eval . --case <name> --runs 1 --ablation none. Confirm any change at the default three runs, with the baseline, before trusting it. - Read the report from the top: verdict and tiles, then each case card, then the failed graders' explanations. A
Δnear zero with thetool_used: Skillgrader failing means the skill's description does not trigger on the prompt's phrasing: change the description once, based on what the report shows, then run the same suite again. - For CI, start from
assets/ci-evals.shand gate on the delta withscripts/gate_eval_result.py. A suite's files can be checked before any spend: on 2.1.325, a run with no credential still loads every case, printsN case file(s) failed to loadfor broken ones, and stops as partial at $0 (observed behavior, not on the docs pages).
Where the answers are
references/messages.md: every documented message, by the stage that produces it (where/pluginruns, add a marketplace, install, enable and update, installed but not working, build, host, organization policy), plus the conventions allclaude plugincommands share (exit codes, scopes,--jsonresult lines). Read it when there is a message to look up.references/loading.md: the three stages, origins,enabledPluginsprecedence, the files under~/.claude/plugins/, in-place and copied plugins, versions and updates, name conflicts. Read it when something loaded wrong or not at all with no message, or to explain why.references/evals.md: running, writing, grading, mocking, CI and reading results forclaude plugin eval. Read it for any eval work.references/environments.md: what loads in a cloud session, in a project's cloud thread, in Claude on the web, desktop and mobile, and in the Agent SDK, how to test and debug there, and where each fix goes. Read it whenever the session is not the user's own terminal.scripts/gate_eval_result.py:python3 scripts/gate_eval_result.py results.json --min-mean-delta 0.2exits 0 when the gate passes, 1 when it fails, 2 when the result is not comparable (partial, judge graders skipped by the cost ceiling, or a run that errored or a mock aborted). Standard library only.assets/: templates to copy into a plugin'sevals/folder or its CI, each with<...>placeholders to fill:assets/eval-case/prompt.md: a case's prompt and its frontmatter.assets/eval-case/case.yaml: optional, for a seeded workspace;assets/eval-case/fixture.shis its scaffold script, which runs only with--scaffold.assets/eval-case/graders/result.md: anllmrubric on the answer;assets/eval-case/graders/skill-fired.md: atool_usedgrader that the skill fired.assets/eval-negative-case/prompt.md, withassets/eval-negative-case/graders/answered.mdandassets/eval-negative-case/graders/skill-not-fired.md: a request near the skill's domain that it must leave alone.assets/mock-tool.md: a mock for a tool the case must not call for real.assets/ci-evals.sh: the CI step; it runs the suite and thenscripts/gate_eval_result.py.
Version and source check
Most flags and messages carry a minimum version, written beside them in the references. Before telling the user a flag or message applies, compare it with their claude --version. When a reference and the installed build disagree, claude plugin <subcommand> --help on that build wins for flags, and the changelog (https://code.claude.com/docs/en/changelog) wins for behavior. Say which you checked.