Test Ask Experts

Validate ask-experts-mcp end-to-end

This skill exercises the project's own ask-experts MCP server to confirm:

1. The MCP is registered and tools are visible in this Claude Code session
2. Each enabled expert returns a sensible, on-topic answer to a canonical question
3. No expert is silently broken (auth, network, model name typo, etc.)

Execute the following steps

Step 1 — Discover registered ask-experts tools

Look at your currently available tool list and identify any tool whose name starts with ask_. These come from this project's .mcp.json registration of ask-experts-mcp.

• If you see zero ask_* tools: the MCP isn't loaded. Tell the user:

  No ask_* tools found. To wire up the project MCP:

  1. npm run build (rebuild dist/index.js if you changed source)
  2. Set whichever provider env vars you have keys for (e.g. export HUNYUAN_API_KEY=sk-...)
  3. Restart Claude Code so it reloads .mcp.json
  4. Re-run /test-ask-experts Stop here.

• If you see at least one ask_* tool: continue to Step 2. List the tools you found in a one-line summary.

Step 2 — Probe each available expert with a canonical question

For each ask_* tool that is registered, call it once with the matching probe question from the table below. Use response_language: "zh" (default). Run the calls in parallel in a single message.

For tools you don't have a probe for (e.g. user added a custom expert), skip them — note the skip in the report.

Pass each question via the question parameter. Do not add context. Do not retry on first failure.

Step 3 — Score each response

For each call, classify the result as one of:

• OK — Returned a coherent, on-topic answer naming concrete API/SOP/concept names. The answer doesn't have to be perfect, just clearly drawing on domain knowledge.
• WEAK — Returned text but it's generic, hedged, or could've come from any model (no domain-specific signal).
• ERROR — Tool returned isError: true or an exception. Capture the error code (auth/timeout/rate_limit/server/empty/etc.) verbatim.

Be honest in your scoring — a confident-sounding but generic answer is WEAK, not OK. Look for: specific API method names, version-aware notes, mention of fields you wouldn't know without training data on that ecosystem.

Step 4 — Final report

Output a markdown report with this structure:

## ask-experts-mcp validation — <YYYY-MM-DD>

### Registered tools
- ask_hunyuan, ask_qwen, ... (N total)

### Per-expert results
| Tool | Verdict | Notes |
|---|---|---|
| ask_hunyuan | OK | 给出了 wx.login → code2session → unionid 完整流程,提到基础库 ≥1.x.x |
| ask_qwen | WEAK | 只讲了 STS 通用概念,没提到 PostObject 表单字段细节 |
| ask_deepseek | ERROR (auth) | API Key 无效或无权限 (401) |

### Suggestions
- For the ERROR cases: <concrete next step, e.g. "DEEPSEEK_API_KEY 看起来无效,确认 Key 状态后重启 Claude Code">
- For WEAK cases: <suggest tightening trigger_keywords or trying a different model name>
- For OK cases: no action needed.

Keep the report concise — the user wants signal, not a wall of text. Don't paste full responses; one short sentence summarizing each is enough.

Step 5 — If everything is OK

Add a final line: ✅ ask-experts-mcp 在本项目下工作正常,N/N 专家通过验证。

If anything was WEAK or ERROR, end with ⚠️ 见上方 Suggestions。

Notes

• This skill expects the MCP server's tool list to already be live. It does not restart Claude Code or rebuild the project.
• If dist/index.js is stale (you just edited src/), rebuild with npm run build and restart Claude Code before invoking this skill.
• The probe questions are intentionally narrow; broader routing-quality testing is out of scope here (that's the v0.3.0 tests/routing/ test set).