MCP REST Latency Eval

mcp-rest-latency-eval

AI/ML Model Cards in Edge AI Cyberinfrastructure: towards Agentic AI — Plale et al. (2025) (arXiv:2511.21661, 2025)

What this evaluates

Evaluates the end-to-end and component-level latency of retrieving and searching AI/ML model cards across three server architectures (REST, Native MCP, Layered MCP) under local and wide-area network conditions. It measures how protocol overhead, payload size, and network distance impact system responsiveness in edge computing environments.

Datasets

• Patra Model Cards (Pseudo-Synthetic) — total ?; splits: test (-1)

Metrics

• end-to-end latency (primary) — range: other
  • Total time from client request initiation to full response reception, measured in milliseconds. Broken down into connection setup, SSE handshake, server processing, and database retrieval times.

Input / output format

Input: Client sends HTTP requests or MCP JSON-RPC 2.0 tool calls to retrieve or search model cards. Payloads are either small (few KB) or large (13.63 MB JSON).

Output: Server returns JSON responses containing model card data or search results. Latency is measured per request/response cycle.

Scoring recipe

def compute_latency(request_start, response_end):
    return (response_end - request_start) * 1000  # milliseconds
def compute_component_breakdown(total, setup, handshake, processing, db):
    return {'connection_setup': setup, 'sse_handshake': handshake, 'server_processing': processing, 'database_retrieval': db}

Common pitfalls

• Confusing the layered MCP overhead (double serialization) with native MCP overhead.
• Assuming wide-area latency differences are solely due to network distance; payload size (KB vs 13 MB) dominates end-to-end time.
• Overlooking that REST is stateless while MCP uses persistent SSE sessions, making connection setup a one-time vs per-request cost.

Evidence (verbatim from paper)

In measuring client-server end-to-end latency, timing is broken down into the following components: Connection Setup: captures the time required to establish a connection between the client and the server. It includes the TCP handshake and the creation of a persistent HTTP connection for event streaming. SSE Handshake Server-Sent Events (SSE) is a transport session connection between the MCP client and server. It involves socket initialization and HTTP upgrade negotiation. Local connection overhead includes the initialization of the MCP session. MCP experiment scripts utilize the MCP Python SDK for establishing Server-Sent Events (SSE) transport sessions. SSE is used in MCP but not REST to maintain the session between requests (REST is stateless). Server Processing: the time taken to issue a resource read command over the established session and receive the response.

Citation

@misc{plale2025modelcards,
  title={AI/ML Model Cards in Edge AI Cyberinfrastructure: towards Agentic AI},
  author={Plale et al. (2025)},
  year={2025},
  note={arXiv:2511.21661}
}

• arXiv: 2511.21661