Virtual MCP Server - How It Works

Virtual MCP Server - How It Works

This document explains how Virtual MCP Servers work using diagrams and examples. For detailed implementation specifics, see virtual-mcp-server.md.

What Problem Are We Solving?

Consider a typical development setup: you have separate MCP servers for GitHub (code search, PRs), Slack (messaging), and Jira (issue tracking). Your AI agent needs tools from all three, which means:

• Managing three separate connections
• Handling three different sessions
• Dealing with tool name conflicts (both GitHub and Jira have a search tool)

A Virtual MCP Server solves this by providing a single endpoint that aggregates tools from multiple backends. Your agent connects once and gets access to all the tools it needs.

WITHOUT Virtual Server:              WITH Virtual Server:

  You                                  You
   |                                    |
   +---> GitHub Server                  |
   |        |-> search                  v
   |        |-> create_pr         +------------+
   |                              |  Virtual   |
   +---> Slack Server             |  Server    |
   |        |-> send_message      +-----+------+
   |        |-> list_channels           |
   |                              +-----+-----+-----+
   +---> Jira Server              |           |     |
            |-> create_issue      v           v     v
            |-> search_issues   GitHub     Slack   Jira
                                Server     Server  Server

Benefits:

• Your app only connects to ONE server instead of many
• You can pick exactly which tools you want from each backend
• You can rename tools to avoid confusion (like "github_search" vs "jira_search")
• You can control who has access to which tools

The Big Picture

Request flow when a client connects to a Virtual MCP Server:

+----------------+                    +------------------+
|   Your App     |                    |   MCP Gateway    |
|                |                    |                  |
|  "I want to    | ---(1) Request --> |  Nginx receives  |
|   search on    |                    |  your request    |
|   GitHub"      |                    +--------+---------+
|                |                             |
|                |                             v
|                |                    +------------------+
|                |                    |  Lua Router      |
|                |                    |  (the brain)     |
|                |                    |                  |
|                |                    |  "Ah, this tool  |
|                |                    |   belongs to     |
|                |                    |   GitHub backend"|
|                |                    +--------+---------+
|                |                             |
|                |                             v
|                |                    +------------------+
|                |                    |  GitHub Backend  |
|                |                    |                  |
|                | <--(4) Response -- |  (does the       |
|                |                    |   actual work)   |
+----------------+                    +------------------+

Each component is described below.

The Three Key Players

1. Nginx (Reverse Proxy)

Nginx receives incoming requests and handles:

• JWT authentication via auth_request subrequest
• Path-based routing to determine which virtual server
• Invoking the Lua content handler for MCP protocol processing

Request arrives at /virtual/dev-tools
                |
                v
        +---------------+
        |    Nginx      |
        |               |
        |  1. Check JWT |  <-- "Is this token valid?"
        |  2. Read path |  <-- "Which virtual server?"
        |  3. Call Lua  |  <-- "Hand off to the router"
        +---------------+

2. Lua Router (Content Handler)

The Lua router (virtual_router.lua) runs as an nginx content handler. It:

• Reads tool-to-backend mappings from JSON config files
• Translates tool aliases back to original names
• Manages session multiplexing across backends
• Issues concurrent subrequests for aggregation methods

3. Backend Servers

The actual MCP servers (GitHub, Slack, Jira, etc.) that execute tool calls. The virtual server coordinates requests but delegates all execution to backends.

How Tool Mapping Works

This is the core mechanism. The process works as follows:

Step 1: Configuration is Created

When someone creates a virtual server, they specify which tools to include:

Virtual Server: "dev-tools"
Path: /virtual/dev-tools

Tool Mappings:
  +------------------+------------------+------------------+
  | Tool Name        | Backend Server   | Alias            |
  +------------------+------------------+------------------+
  | search           | /github          | github_search    |
  | search           | /jira            | jira_search      |
  | send_message     | /slack           | (none - use as-is)|
  +------------------+------------------+------------------+

Both GitHub and Jira have a tool called "search". Aliases resolve this naming conflict.

Step 2: Mapping File is Generated

The system writes a JSON file that the Lua router will read:

File: /etc/nginx/lua/virtual_mappings/dev-tools.json

{
  "tool_backend_map": {
    "github_search": {
      "original_name": "search",
      "backend_location": "/_backend/github"
    },
    "jira_search": {
      "original_name": "search",
      "backend_location": "/_backend/jira"
    },
    "send_message": {
      "original_name": "send_message",
      "backend_location": "/_backend/slack"
    }
  }
}

This file is a lookup table mapping tool names to their backend locations.

Step 3: Request Comes In

When your app calls a tool:

Your app sends:
{
  "method": "tools/call",
  "params": {
    "name": "github_search",      <-- The alias you see
    "arguments": { "query": "bug fixes" }
  }
}

Step 4: Lua Router Translates

The Lua router:

1. Reads the mapping file
2. Looks up "github_search"
3. Finds: backend is "/_backend/github", original name is "search"
4. Rewrites the request:

Forwarded to /_backend/github:
{
  "method": "tools/call",
  "params": {
    "name": "search",             <-- Original name the backend knows
    "arguments": { "query": "bug fixes" }
  }
}

Step 5: Response Goes Back

The GitHub backend responds. The Lua router passes it back to your app unchanged.

The Complete Request Flow (Sequence Diagram)

Sequence diagram for a tools/call request:

Your App          Nginx           Lua Router        Backend
   |                |                  |               |
   |  POST /virtual/dev-tools          |               |
   |  tools/call: github_search        |               |
   |--------------->|                  |               |
   |                |                  |               |
   |                |  auth_request    |               |
   |                |  (check JWT)     |               |
   |                |----------------->|               |
   |                |  OK + scopes     |               |
   |                |<-----------------|               |
   |                |                  |               |
   |                |  content_by_lua  |               |
   |                |----------------->|               |
   |                |                  |               |
   |                |      Read mapping file           |
   |                |      "github_search" ->          |
   |                |        backend: /_backend/github |
   |                |        original: search          |
   |                |                  |               |
   |                |      Check session cache         |
   |                |      (do we have a session       |
   |                |       with this backend?)        |
   |                |                  |               |
   |                |      Rewrite tool name           |
   |                |      github_search -> search     |
   |                |                  |               |
   |                |                  | POST to       |
   |                |                  | /_backend/github
   |                |                  |-------------->|
   |                |                  |               |
   |                |                  |   Response    |
   |                |                  |<--------------|
   |                |                  |               |
   |<----------------------------------|               |
   |      Response                     |               |

Session Management (The Tricky Part)

Each backend server requires its own session.

When your app connects to the virtual server, it gets ONE session ID:

Your app <---> Virtual Server (session: vs-abc123)

But behind the scenes, the virtual server maintains SEPARATE sessions with each backend:

Virtual Server:
  +-- Session with GitHub: sess-gh-001
  +-- Session with Slack:  sess-sl-002
  +-- Session with Jira:   sess-jr-003

The Lua router keeps track of this mapping so you don't have to.

The Two-Tier Cache

Looking up sessions from the database on every request would be slow. So we use two levels of caching:

Request: "What's the GitHub session for client vs-abc123?"

        +-------------------+
        |  Level 1 Cache    |  <-- Super fast (in nginx memory)
        |  (Shared Dict)    |      TTL: 30 seconds
        |                   |
        |  "Do I have it?"  |
        +--------+----------+
                 |
        MISS     |
                 v
        +-------------------+
        |  Level 2 Cache    |  <-- Fast (MongoDB lookup)
        |  (MongoDB)        |      TTL: 1 hour
        |                   |
        |  "Check database" |
        +--------+----------+
                 |
        MISS     |
                 v
        +-------------------+
        |  Create New       |  <-- Send "initialize" to backend
        |  Session          |      Store in both caches
        +-------------------+

Why two levels?

• Level 1 is in memory - no network call, extremely fast
• Level 2 is in MongoDB - survives server restarts
• If the server restarts, we lose Level 1 but Level 2 still has sessions

Listing Tools (Aggregation)

When your app asks "what tools do you have?", the virtual server needs to ask ALL backends:

Your app asks: tools/list

Lua Router:
  +-- Ask GitHub: "What tools do you have?"
  |     Response: [search, create_pr, list_repos]
  |
  +-- Ask Slack: "What tools do you have?"
  |     Response: [send_message, list_channels]
  |
  +-- Ask Jira: "What tools do you have?"
        Response: [create_issue, search_issues]

Lua Router combines them:
  [github_search, create_pr, list_repos,     <-- Applied aliases
   send_message, list_channels,
   jira_search, create_issue]                 <-- Renamed "search" to "jira_search"

Important optimization: These backend calls happen IN PARALLEL, not one after another. This makes the aggregation fast.

Time ------>

Sequential (slow):
  [GitHub call]---[Slack call]---[Jira call]---Done

Parallel (fast):
  [GitHub call]-----
  [Slack call]------+---Done
  [Jira call]-------

What the Nginx Config Looks Like

When a virtual server is enabled, the system generates two things:

1. A Location Block (for routing)

location /virtual/dev-tools {
    # Tell Lua which virtual server this is
    set $virtual_server_id "dev-tools";

    # Check authentication first
    auth_request /validate;

    # Run the Lua router
    content_by_lua_file /etc/nginx/lua/virtual_router.lua;
}

2. Internal Backend Locations

# These are marked "internal" - only Lua can use them
# Regular users can't access them directly

location /_backend/github {
    internal;
    proxy_pass https://github-mcp.example.com/mcp;
}

location /_backend/slack {
    internal;
    proxy_pass https://slack-mcp.example.com/mcp;
}

The Lua router uses these internal locations to talk to backends.

Error Handling

What happens when things go wrong?

Backend is Down

Lua Router tries to call /_backend/github
  |
  v
Connection fails or returns error
  |
  v
Lua Router returns error to your app:
{
  "error": {
    "code": -32000,
    "message": "Backend server unreachable: /github"
  }
}

Session Expired

Lua Router uses cached session sess-gh-001
  |
  v
GitHub returns: "400 Bad Request - Invalid session"
  |
  v
Lua Router:
  1. Delete sess-gh-001 from both caches
  2. Send new "initialize" to GitHub
  3. Get new session: sess-gh-002
  4. Cache it in both levels
  5. Retry the original request with new session

User Lacks Permission

User has scopes: ["mcp-access"]
Tool "create_pr" requires: ["github-write"]
  |
  v
Lua Router checks scopes... DENIED
  |
  v
Response: 403 Forbidden
{
  "error": "Missing required scope: github-write"
}

Access Control in Simple Terms

Access control works at two levels:

Level 1: Server Access

To use the virtual server at all, you need certain scopes:

Virtual Server: /virtual/dev-tools
Required Scopes: ["mcp-access"]

User with scopes ["mcp-access"] -> Allowed in
User with scopes ["other-stuff"] -> Blocked at the door

Level 2: Tool Access

Individual tools can require additional scopes:

Tool: create_pr
Required Scopes: ["github-write"]

User with ["mcp-access", "github-read"] -> Can't use this tool
User with ["mcp-access", "github-write"] -> Can use this tool

When listing tools, the Lua router hides tools the user can't access:

Full tool list:    [search, create_pr, delete_repo]
User scopes:       ["mcp-access", "github-read"]

Filtered list:     [search]  <-- Only shows tools user can actually use

How Changes Are Applied

When you create or update a virtual server:

1. You call the API: POST /api/virtual-servers
          |
          v
2. Service validates the configuration
   - Does each backend server exist?
   - Does each tool exist on its backend?
   - Are all alias names unique?
          |
          v
3. Configuration saved to MongoDB
          |
          v
4. Nginx config regenerated
   - New location block written
   - New mapping JSON file written
          |
          v
5. Nginx reloaded
   - nginx -s reload
   - New config takes effect immediately
          |
          v
6. Virtual server is live!

Quick Reference

Files You Should Know

Key Concepts

Common Operations

Summary

1. Virtual servers aggregate tools from multiple backends into one endpoint
2. Nginx routes requests to the Lua router based on path
3. Lua router reads mapping files to know which tool goes where
4. Aliases solve naming conflicts when two backends have same tool names
5. Sessions are cached in two levels for speed and reliability
6. Access control works at server and tool level using scopes
7. Backend calls happen in parallel when listing tools

The virtual server acts as a coordinator - all tool execution happens on the backend servers. The virtual server's role is to present a unified endpoint to clients.