MCP Headless Test

MCP Headless Test Pipeline

Purpose

Validate the Ariadne MCP server by running it with Claude Code in headless mode (claude -p). This tests the real user experience rather than just programmatic MCP client behavior.

Prerequisites

Before running tests, ensure:

1. Claude Code CLI is installed and configured
2. API key is set (either ANTHROPIC_API_KEY env var or Claude's configured key)
3. MCP package is built:

npm run build -w packages/mcp

Running the Tests

Quick Run

cd packages/mcp
./tests/claude-headless-test.sh

Verbose Mode (for debugging)

./tests/claude-headless-test.sh --verbose

Test Scenarios

The script runs 3 tests:

Interpreting Results

Success Output

========================================
 Ariadne MCP - Claude Headless Tests
========================================

[INFO] Checking prerequisites...
[INFO] All prerequisites met.

[INFO] Test 1: Tool Discovery
[INFO] Checking if Claude can see the list_entrypoints tool...
[INFO] PASS: Tool 'list_entrypoints' discovered

[INFO] Test 2: Tool Invocation
[INFO] Running list_entrypoints on fixture code...
[INFO]   - Found 'entry point' in output
[INFO] PASS: Tool invocation succeeded

[INFO] Test 3: Filtered Analysis
[INFO] Testing file/folder filtering parameters...
[INFO] PASS: Filtered analysis completed

========================================
 Test Summary
========================================
Total:  3
Passed: 3
Failed: 0

[INFO] All tests passed!

Failure Indicators

Verifying Correctness

Test 1: Tool Discovery

Expected behavior: Claude should list list_entrypoints when asked about available tools.

Manual verification:

claude -p "What MCP tools do you have?" \
  --mcp-config ./packages/mcp/tests/mcp-test-config.json \
  --output-format json

Look for output containing:

• Tool name: list_entrypoints
• Description mentioning "entry point" and "call tree"

Test 2: Tool Invocation

Expected behavior: Claude should successfully invoke the tool and return analysis results.

Manual verification:

claude -p "Use list_entrypoints to analyze the codebase" \
  --mcp-config ./packages/mcp/tests/mcp-test-config.json

Valid output should contain:

• "Entry Points" header
• Function signatures (e.g., function_name(...): return_type)
• Tree size indicators (e.g., -- N functions)
• File locations (e.g., Location: ...)
• Total count (e.g., Total: X entry points)

Test 3: Filtered Analysis

Expected behavior: Tool accepts folders parameter and analyzes only specified directories.

Manual verification:

claude -p "Use list_entrypoints with folders=['functions']" \
  --mcp-config ./packages/mcp/tests/mcp-test-config.json

Valid output: Should show entry points only from the functions/ subdirectory of the fixtures.

Test Configuration

The tests use packages/mcp/tests/mcp-test-config.json:

{
  "mcpServers": {
    "ariadne": {
      "type": "stdio",
      "command": "node",
      "args": ["./packages/mcp/dist/server.js"],
      "env": {
        "PROJECT_PATH": "./packages/core/tests/fixtures/typescript/code"
      }
    }
  }
}

This points to TypeScript fixtures in packages/core/tests/fixtures/typescript/code/ for deterministic, reproducible tests.

Server Lifecycle Notes

Important: The MCP server process stays alive across tool calls within a session, BUT each tool call creates a fresh Project instance and re-indexes the codebase. This is intentional to support scoped analysis (file/folder filtering).

Performance implication: Large codebases may take 2+ seconds per tool call due to re-indexing.

Troubleshooting

Claude hangs or times out

• Check ANTHROPIC_API_KEY is valid
• Try with --verbose flag to see debug output
• Ensure network connectivity to Anthropic API

Tool not discovered

1. Verify MCP config path is correct

2. Check server.js exists: ls packages/mcp/dist/server.js

3. Test server directly:

   echo '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | \
     PROJECT_PATH=. node packages/mcp/dist/server.js
   

Unexpected tool output

1. Check fixtures exist: ls packages/core/tests/fixtures/typescript/code/
2. Test with E2E tests first: npm test -w packages/mcp
3. Review server stderr for errors (use --verbose)

Related Files

Source: CRJFisher/ariadne — distributed by TomeVault.