MCP E2E Verification
Full-stack verification against a real dbt project of the 8 MCP tools this harness
covers: lineage_diff, schema_diff, row_count_diff, query, query_diff,
profile_diff, list_checks, run_check.
list_tools registers at most 20 tools, and the count is mode-dependent —
7 always, analyze_model only in local mode, and 12 only in server mode, so
preview and read-only mode expose 8. A green run here is a smoke test of the core
diff surface, not full-surface coverage. Do not report it as "all tools
verified". See the recce-mcp-dev skill for the per-mode breakdown and the
uncovered list.
When to Use
- After modifying
recce/mcp_server.py or _tool_* handlers
- After changing single-env logic or error classification
- Before merging any MCP-related PR
- Not for: unit test changes only, frontend-only changes, docs-only changes
Usage
Invoke as /recce-mcp-e2e or /recce-mcp-e2e <project_path>.
- With argument: use the given path as the dbt project directory
- Without argument: ask the user for the dbt project path
The project directory must contain target/manifest.json and target-base/manifest.json.
Process
- Resolve project path from argument or user input
- Validate
target/ and target-base/ exist with manifest.json
- Detect recce source — find the repo root containing
recce/mcp_server.py. If recce-nightly is also installed (pip show recce recce-nightly), set PYTHONPATH=<RECCE_REPO_ROOT>:$PYTHONPATH
- Generate
test_mcp_e2e.py in the project directory from test_mcp_e2e_template.py (in this skill directory). Replace PROJECT_DIR_PLACEHOLDER with the resolved absolute path.
- Execute with appropriate PYTHONPATH prefix
- Report results — all 13 checks must show PASS. Expected output:
=== FULL MODE (8 tools) ===
PASS lineage_diff: PASS
...
=== SINGLE-ENV MODE ===
PASS row_count_diff (_warning): PASS
...
ALL PASS
- Clean up — delete
test_mcp_e2e.py
Quick Reference
| Test Suite |
Checks |
What's Verified |
| Full mode (8 tools) |
lineage_diff, schema_diff, row_count_diff, query, query_diff, profile_diff, list_checks, run_check |
Non-empty results from each tool |
| Single-env _warning (3) |
row_count_diff, query_diff, profile_diff |
_warning field present with SINGLE_ENV_WARNING |
| Single-env no _warning (2) |
lineage_diff, schema_diff |
_warning field NOT present |
Additional manual checks (not in script):
| Check |
Command/Action |
| --help |
recce mcp-server --help shows Prerequisites section |
| Server modes |
Non-server mode: list_tools returns only lineage_diff + schema_diff |
Common Mistakes
| Problem |
Fix |
ImportError: cannot import name 'SINGLE_ENV_WARNING' |
recce-nightly conflict — use PYTHONPATH=<RECCE_REPO_ROOT>:$PYTHONPATH |
| lineage_diff returns empty |
Use view_mode="all" (default changed_models filters out unchanged) |
| list_checks returns empty |
Preset checks from recce.yml must be loaded via load_preset_checks() — script handles this |
portalocker FileNotFoundError on exit |
Cosmetic thread error in event collector — does not affect results |
| Single-env test uses target-base |
By design — load_context needs both, single_env=True flag simulates the mode |
1---2name: recce-mcp-e2e3description: Use when MCP server code is modified and needs smoke verification of the core diff tools against a real dbt project. Triggers after changes to recce/mcp_server.py, MCP tool handlers, single-env logic, or error classification. Also use before merging MCP PRs.4---56# MCP E2E Verification78Full-stack verification against a real dbt project of the 8 MCP tools this harness9covers: `lineage_diff`, `schema_diff`, `row_count_diff`, `query`, `query_diff`,10`profile_diff`, `list_checks`, `run_check`.1112`list_tools` registers **at most 20** tools, and the count is mode-dependent —137 always, `analyze_model` only in local mode, and 12 only in server mode, so14preview and read-only mode expose 8. A green run here is a smoke test of the core15diff surface, **not** full-surface coverage. Do not report it as "all tools16verified". See the `recce-mcp-dev` skill for the per-mode breakdown and the17uncovered list.1819## When to Use2021- After modifying `recce/mcp_server.py` or `_tool_*` handlers22- After changing single-env logic or error classification23- Before merging any MCP-related PR24- **Not for**: unit test changes only, frontend-only changes, docs-only changes2526## Usage2728Invoke as `/recce-mcp-e2e` or `/recce-mcp-e2e <project_path>`.2930- **With argument**: use the given path as the dbt project directory31- **Without argument**: ask the user for the dbt project path3233The project directory must contain `target/manifest.json` and `target-base/manifest.json`.3435## Process36371. **Resolve project path** from argument or user input382. **Validate** `target/` and `target-base/` exist with `manifest.json`393. **Detect recce source** — find the repo root containing `recce/mcp_server.py`. If `recce-nightly` is also installed (`pip show recce recce-nightly`), set `PYTHONPATH=<RECCE_REPO_ROOT>:$PYTHONPATH`404. **Generate** `test_mcp_e2e.py` in the project directory from `test_mcp_e2e_template.py` (in this skill directory). Replace `PROJECT_DIR_PLACEHOLDER` with the resolved absolute path.415. **Execute** with appropriate PYTHONPATH prefix426. **Report** results — all 13 checks must show PASS. Expected output:43 ```44 === FULL MODE (8 tools) ===45 PASS lineage_diff: PASS46 ...47 === SINGLE-ENV MODE ===48 PASS row_count_diff (_warning): PASS49 ...50 ALL PASS51 ```527. **Clean up** — delete `test_mcp_e2e.py`5354## Quick Reference5556| Test Suite | Checks | What's Verified |57|-----------|--------|----------------|58| Full mode (8 tools) | lineage_diff, schema_diff, row_count_diff, query, query_diff, profile_diff, list_checks, run_check | Non-empty results from each tool |59| Single-env _warning (3) | row_count_diff, query_diff, profile_diff | `_warning` field present with `SINGLE_ENV_WARNING` |60| Single-env no _warning (2) | lineage_diff, schema_diff | `_warning` field NOT present |6162**Additional manual checks** (not in script):6364| Check | Command/Action |65|-------|---------------|66| --help | `recce mcp-server --help` shows Prerequisites section |67| Server modes | Non-server mode: `list_tools` returns only lineage_diff + schema_diff |6869## Common Mistakes7071| Problem | Fix |72|---------|-----|73| `ImportError: cannot import name 'SINGLE_ENV_WARNING'` | recce-nightly conflict — use `PYTHONPATH=<RECCE_REPO_ROOT>:$PYTHONPATH` |74| lineage_diff returns empty | Use `view_mode="all"` (default `changed_models` filters out unchanged) |75| list_checks returns empty | Preset checks from `recce.yml` must be loaded via `load_preset_checks()` — script handles this |76| `portalocker` FileNotFoundError on exit | Cosmetic thread error in event collector — does not affect results |77| Single-env test uses target-base | By design — `load_context` needs both, `single_env=True` flag simulates the mode |
datarecce