MCP Gateway Manual Acceptance Testing

MCP Gateway Manual Acceptance Testing

Acceptance testing for MCP Gateway. Use these example steps to ensure a deploy MCP Gateway is working correctly in your environment.

graph TB
    subgraph "TESTER WORKSTATION"
        PIP[pip install mcp-contextforge-gateway]
        JWT[mcpgateway.utils.create_jwt_token<br/>Generates Bearer Token]
        WRAPPER[mcpgateway.wrapper<br/>stdio to HTTP bridge]
        VSC_SSE[VS Code with SSE/HTTP<br/>Direct connection]
        INSPECTOR[MCP Inspector]
        AGENTS[AI Agents<br/>LangChain / CrewAI]
        HEY[hey<br/>Performance Testing]

        PIP --> JWT
        JWT --> WRAPPER
        JWT --> VSC_SSE
        JWT --> AGENTS
    end

    subgraph "MCP GATEWAY"
        GATEWAY_ENTRY[ ]
        GW[MCP Gateway<br/>Remote: $GW_URL]
        GW_AUTH[Authentication<br/>Bearer Token]
        GW_FEDERATION[Federation Layer]
        GW_VIRTUAL[Virtual Servers]
        GW_RESOURCES[Resources]
        GW_PROMPTS[Prompts]
        GW_TOOLS[Tool Registry]

        GW --> GW_AUTH
        GW_AUTH --> GW_FEDERATION
        GW_FEDERATION --> GW_VIRTUAL
        GW_FEDERATION --> GW_TOOLS
        GW --> GW_RESOURCES
        GW --> GW_PROMPTS
    end

    subgraph "MCP SERVERS RUNTIME"
        RUNTIME_ENTRY[ ]
        TIME[time server<br/>Port: 8101]
        TIME_AUTH[time server auth<br/>Port: 8102]
        FAST_TIME[fast_time_server]
        GIT[git server<br/>Port: 8103]
        GITHUB[github]
        JIRA[jira]
        SERVICENOW[servicenow]
        PLAYWRIGHT[playwright]
        FIGMA[figma]
        MONDAY[monday.com]
        BOX[box]
    end

    %% Simulate full subgraph connection
    GATEWAY_ENTRY -->|↕ Full Runtime ↔ Gateway| RUNTIME_ENTRY

    %% Connections from Workstation to Gateway
    WRAPPER -.->|HTTP/JSON-RPC| GW
    VSC_SSE -.->|SSE/HTTP| GW_VIRTUAL
    INSPECTOR -.->|HTTP| GW
    AGENTS -.->|HTTP API| GW
    HEY -.->|Load Testing| GW

    %% Styling for dark theme visibility
    classDef workstation fill:#2196F3,stroke:#64B5F6,stroke-width:3px,color:#fff
    classDef gateway fill:#FF6B35,stroke:#FF8C61,stroke-width:4px,color:#fff
    classDef servers fill:#9C27B0,stroke:#BA68C8,stroke-width:3px,color:#fff
    classDef auth fill:#F44336,stroke:#EF5350,stroke-width:3px,color:#fff

    class PIP,JWT,WRAPPER,VSC_SSE,INSPECTOR,AGENTS,HEY workstation
    class GW,GW_FEDERATION,GW_VIRTUAL,GW_RESOURCES,GW_PROMPTS,GW_TOOLS,GATEWAY_ENTRY gateway
    class TIME,TIME_AUTH,FAST_TIME,GIT,GITHUB,JIRA,SERVICENOW,PLAYWRIGHT,FIGMA,MONDAY,BOX,RUNTIME_ENTRY servers
    class GW_AUTH,TIME_AUTH auth

Test Environment Setup

Setting up MCP Servers for Testing

1. Core Protocol Operations

2. Gateway Federation

3. Tool Discovery & Execution

4. Virtual Server Management

5. Resource Management

6. Prompt Management

7. REST Tool Creation

8. MCP Wrapper Testing

9. VS Code Integration Testing

9.1. VS Code with MCP Wrapper (stdio)

"gateway-virtual": {
  "command": "python",
  "args": ["-m", "mcpgateway.wrapper"],
  "env": {
    "MCP_SERVER_URL": "$GW_URL/servers/$TIME_SERVER_UUID",
    "MCP_AUTH": "$MCPGATEWAY_BEARER_TOKEN"
  }
}

} }``` | Server configured | ☐ | Replace $GW_URL and tokens | | Restart VS Code | Reload window | Cmd/Ctrl+Shift+P → "Reload Window" | MCP server connects | ☐ | Check output panel | | Verify Connection | Check MCP panel | View → MCP Servers | Gateway tools visible | ☐ | Should show time tools | | Execute Tool | Use command palette | Cmd/Ctrl+Shift+P → "MCP: Execute Tool" → Select time tool | Tool executes successfully | ☐ | Returns time data |

9.2. VS Code Direct SSE/HTTP Connection

"gateway-direct": {
  "transport": "sse",
  "url": "$GW_URL/servers/$TIME_SERVER_UUID/sse",
  "headers": {
    "Authorization": "Bearer $MCPGATEWAY_BEARER_TOKEN"
  }
}

} }| Server configured | ☐ | Direct SSE connection | | Test HTTP Transport | Alternative config |json { "mcp.servers": { "gateway-http": { "transport": "http", "url": "$GW_URL", "headers": { "Authorization": "Bearer $MCPGATEWAY_BEARER_TOKEN" } } } }``` | HTTP transport configured | ☐ | REST-based connection | | Verify SSE Stream | Check developer tools | F12 → Network tab → Filter "EventSource" | SSE connection established | ☐ | Real-time updates | | Execute via HTTP | Use MCP panel | Click tool → Enter parameters → Execute | Tool runs successfully | ☐ | Check response |

10. AI Agent Integration Testing

10.1. LangChain Integration

langchain_test_1.py

from langchain_mcp import MCPToolkit
from langchain.agents import create_react_agent

toolkit = MCPToolkit(
    gateway_url="$GW_URL",
    auth_token="$MCPGATEWAY_BEARER_TOKEN"
)
tools = toolkit.get_tools()
agent = create_react_agent(tools=tools)

10.2. CrewAI Integration

crewai_test_1.py

from crewai_tools import MCPGatewayTool

time_tool = MCPGatewayTool(
    gateway_url="$GW_URL",
    tool_name="time-server-noauth-get-system-time",
    auth_token="$MCPGATEWAY_BEARER_TOKEN"
)

crewai_test_2.py

from crewai import Agent

time_agent = Agent(
    role="Time Zone Specialist",
    goal="Provide accurate time information",
    tools=[time_tool]
)

crewai_test_3.py

from crewai import Crew, Task

task = Task(description="Get current time in multiple cities")
crew = Crew(agents=[time_agent], tasks=[task])
result = crew.kickoff()

11. Open Source MCP Server Testing

MCP Server Tool Testing

12. Health & Monitoring

13. Admin Interface Testing

Note: Admin API must be enabled in configuration:

MCPGATEWAY_UI_ENABLED=true
MCPGATEWAY_ADMIN_API_ENABLED=true

14. Input Validation Testing

15. Error Handling Verification

16. System Summary Verification

Test Cleanup

Notes

• Gateway Base URL: Set export GW_URL=http://your-gateway:4444 for remote gateways
• Authentication: Use Bearer token in format: Authorization: Bearer $MCPGATEWAY_BEARER_TOKEN
• JWT Token Generation: Can also be done inside Docker container: docker exec mcpgateway python3 -m mcpgateway.utils.create_jwt_token -u admin@example.com -e 10080 --secret my-test-key
• Time Servers: The time server gateways are used throughout testing as reference implementations
• Gateway Tool Separator: Default is __ (double underscore) between gateway name and tool name, but newer versions may use -
• Status Column: Check ☐ when test passes, add ✗ if test fails with failure reason
• Pipe Character: The \| in commands represents the pipe character - remove the backslash when copying
• jq: Install with apt-get install jq or brew install jq for JSON formatting
• MCP Servers: Start each MCP server on its designated port before registering as gateway
• VS Code: Reload window after changing MCP configuration in settings.json
• AI Agents: Ensure you have API keys configured for LLM providers when testing agents

MCP Gateway Test Report Template

Test Environment: MCP Gateway v___ Test Date: ___________ Tester: ___________ Gateway URL: ___________

Status Legend

• ✅ Passed - All tests in section completed successfully
• ⚠️ Partial - Some tests passed, some failed or incomplete
• ❌ Failed - Critical failures preventing section completion
• 🚧 In Progress - Testing currently underway
• 🔒 Blocked - Cannot proceed due to dependencies or external factors

Executive Summary

Overall test completion: ___% | Total Tests: ___ | Passed: ___ | Failed: ___ | Blocked: ___

Test Results by Section

Issue Summary

No critical issues were found during this test cycle.

🔴 Critical Issues

None

🟡 Major Issues

None

🟢 Minor Issues

None

Recommendations

1. Immediate Actions:

   • Verify environment-specific configurations
   • Rotate test credentials after completion

2. Short-term:

   • Automate regression suite execution

3. Long-term:

   • Expand coverage for edge cases

Test Environment Notes

• Performance: Standard baseline met
• Stability: Stable
• Federation: Functional
• Compatibility: Verified with current clients