How do I install the MCP Test Suite skill?

Run npx skillmds@latest add zhengqiuyang/mcp-test-suite in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is MCP Test Suite safe to use?

This skill has not completed SkillMD's automated safety review yet. Capability flags: reads secrets. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with MCP Test Suite?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is MCP Test Suite free to use?

Yes. Installing skills from SkillMD is free. This skill is licensed under MIT.

Who published MCP Test Suite?

zhengqiuyang (@zhengqiuyang) published this skill. Their other Agent Skills are listed on their SkillMD profile.

MCP Test Suite by zhengqiuyang — Agent Skill

Name: MCP Test Suite
Rating: 4.5 (4 reviews)
Author: zhengqiuyang

MCP Test Suite

Authors test suites for MCP servers using the mcp-test tool: functional YAML assertions, prompt-injection security probes, false-positive tuning, and CI gating. Use when building or maintaining an MCP server that needs more than manual poking in an inspector — especially before exposing it to agent clients you do not control.

zhengqiuyang Updated Sep 22, 2026

File contents

Authoring mcp-test Suites

mcp-test runs YAML-defined functional tests and injection probes against a running MCP server over stdio, with CI-ready exit codes (0 pass, 1 findings, 2 config error). This skill covers the parts the README assumes.

Start functional, then arm security

Handshake smoke: listTools: true + toolsContain: [expected-tools] — catches startup config errors instantly.
One call per tool: empty assert: {} asserts "completed" — the cheapest possible coverage. Then add contains/jsonSchema/maxLatencyMs for the tools that matter.
Security section last, on the tools that touch untrusted input (files, web, tickets, notes).

Probe philosophy (where FPs come from)

Probes send known injection payloads (instruction-override and credential-file families — the exact payload table ships in the mcp-test README and probes source) through tool arguments, then run leak detectors (private-key headers, token shapes, env-dump shapes, system-prompt markers) on responses. Two design facts matter when reading results:

A tool merely echoing the payload is not a leak — payloads are stripped before detection, including slash-normalized echoes (Windows paths). If you see a leak, look for the detector evidence line: it names what was found.
suspicious never fails CI — it means compliance language + echo, worth a look, not a gate.

FP tuning workflow

A detector firing on your legitimate server:

Read the evidence excerpt — which detector, which shape?
Drop that probe for the tool (probes: [ssh-key-exfil, env-exfiltration]) as an immediate unblock, then
Fix the underlying imprecision (usually: severity should follow data flow, not vocabulary — env | grep VAR presence probes are debugging idioms, not dumps), add a regression test with the exact false-positive input.

CI integration

- run: npx mcp-test -c mcp-test.yaml --format github

--format github emits ::error file=...,line=... annotations pointing at the failing YAML entry; the exit code gates the build with no extra steps. Note: .cmd server commands on Windows need command: cmd, args: [/c, ...] (spawn is shell-less).

Schema-drift upgrade gate (companion)

Pin tool versions with mcp-test snapshot; before any upgrade run mcp-test diff --snapshot old.json --new <candidate> — it spins both versions over stdio and classifies breaking (removed tool, +required param, type change) vs additive. Exit 1 on breaking changes gates the upgrade in CI.

---name: mcp-test-suitedescription: Authors test suites for MCP servers using the mcp-test tool: functional YAML assertions, prompt-injection security probes, false-positive tuning, and CI gating. Use when building or maintaining an MCP server that needs more than manual poking in an inspector — especially before exposing it to agent clients you do not control.license: MIT---# Authoring mcp-test Suites`mcp-test` runs YAML-defined functional tests and injection probes against arunning MCP server over stdio, with CI-ready exit codes (0 pass, 1 findings,2 config error). This skill covers the parts the README assumes.## Start functional, then arm security1. **Handshake smoke**: `listTools: true` + `toolsContain: [expected-tools]` — catches startup config errors instantly.2. **One call per tool**: empty `assert: {}` asserts "completed" — the cheapest possible coverage. Then add `contains`/`jsonSchema`/`maxLatencyMs` for the tools that matter.3. **Security section last**, on the tools that touch untrusted input (files, web, tickets, notes).## Probe philosophy (where FPs come from)Probes send known injection payloads (instruction-override and credential-filefamilies — the exact payload table ships in the mcp-test README and probessource) through tool arguments, then run leak detectors (private-key headers,token shapes, env-dump shapes, system-prompt markers) on responses. Two designfacts matter when reading results:- **A tool merely echoing the payload is not a leak** — payloads are stripped before detection, including slash-normalized echoes (Windows paths). If you see a leak, look for the *detector evidence line*: it names what was found.- **`suspicious` never fails CI** — it means compliance language + echo, worth a look, not a gate.## FP tuning workflowA detector firing on your legitimate server:1. Read the evidence excerpt — which detector, which shape?2. Drop that probe for the tool (`probes: [ssh-key-exfil, env-exfiltration]`) as an immediate unblock, then3. Fix the underlying imprecision (usually: severity should follow data flow, not vocabulary — `env | grep VAR` presence probes are debugging idioms, not dumps), add a regression test with the exact false-positive input.## CI integration```yaml- run: npx mcp-test -c mcp-test.yaml --format github````--format github` emits `::error file=...,line=...` annotations pointing atthe failing YAML entry; the exit code gates the build with no extra steps.Note: `.cmd` server commands on Windows need `command: cmd, args: [/c, ...]`(spawn is shell-less).## Schema-drift upgrade gate (companion)Pin tool versions with `mcp-test snapshot`; before any upgrade run`mcp-test diff --snapshot old.json --new <candidate>` — it spins both versionsover stdio and classifies breaking (removed tool, +required param, typechange) vs additive. Exit 1 on breaking changes gates the upgrade in CI.

Frequently asked questions