Multi Tool Batch Invocation

Multi-Tool Parallel Batch Invocation Protocol

Overview

When an agent needs to inspect 4 related files (e.g., schema.prisma, auth.ts, routes.ts, and types.ts), naive agents execute 4 sequential back-and-forth turns:

• Turn 1: Call view_file("schema.prisma") $\rightarrow$ Wait for response
• Turn 2: Call view_file("auth.ts") $\rightarrow$ Wait for response
• Turn 3: Call view_file("routes.ts") $\rightarrow$ Wait for response
• Turn 4: Call view_file("types.ts") $\rightarrow$ Wait for response

Sequential tool calling re-sends the entire conversation transcript 4 separate times, incurring 4 API roundtrips and 15 to 20 seconds of latency.

The Multi-Tool Batch Invocation Protocol leverages modern function calling specifications to emit multiple tool calls in a single turn (tool_calls: [...]), executing independent operations concurrently via asyncio.gather or Promise.all.

Sequential Single-Tool Turns vs. Parallel Batch Tooling

┌─────────────────────────────────────────────────────────────┐
│                 Tool Execution Roundtrips                   │
│                                                             │
│  Sequential Tool Calls (4 Turns / 14,800 Tokens):           │
│  • Turn 1: Reads `schema.prisma` ──► 1 API Roundtrip (3.2s) │
│  • Turn 2: Reads `auth.ts`       ──► 1 API Roundtrip (3.4s) │
│  • Turn 3: Reads `routes.ts`     ──► 1 API Roundtrip (3.5s) │
│  • Turn 4: Reads `types.ts`      ──► 1 API Roundtrip (3.1s) │
│  ↳ 4 Roundtrips, 13.2s total latency, 14,800 tokens billed  │
│                                                             │
│  Parallel Batch Invocation (1 Turn / 4,200 Tokens):         │
│  • Turn 1: Model emits array of 4 `view_file` tool calls    │
│    ↳ Client executes all 4 reads concurrently (0.05s CPU)   │
│  • Turn 2: Model receives all 4 results in 1 batch return   │
│  ↳ 1 Roundtrip, 3.4s total latency (3.8x Faster, 71% Cut!)  │
└─────────────────────────────────────────────────────────────┘

The Batching Decision Matrix

┌───────────────────────────────────────────────────────────────────────────┐
│ 🟢 PARALLEL BATCH INVOCATION (Batch in 1 Turn):                           │
│ • Reading multiple independent files (`view_file` on A, B, C)             │
│ • Searching multiple distinct terms (`grep_search` on query 1 & 2)        │
│ • Checking multiple URLs (`read_url_content` across 3 documentation links)│
│                                                                           │
│ 🟡 SEQUENTIAL SERIALIZATION (Execute 1-by-1):                             │
│ • Dependent writes (Editing file A before checking if build passes)       │
│ • Modifying database schema before executing migrations                   │
└───────────────────────────────────────────────────────────────────────────┘

Production Python Parallel Tool Runner (asyncio.gather)

import asyncio
from typing import List, Dict, Any

async def execute_tool_call_async(tool_call: Dict[str, Any]) -> Dict[str, Any]:
    """Executes a single tool call asynchronously."""
    name = tool_call["function"]["name"]
    args = json.loads(tool_call["function"]["arguments"])
    
    if name == "view_file":
        result = await async_read_file(args["AbsolutePath"], args.get("StartLine"), args.get("EndLine"))
    elif name == "grep_search":
        result = await async_grep(args["Query"], args["SearchPath"])
    else:
        result = f"Unsupported async tool: {name}"
        
    return {
        "tool_call_id": tool_call["id"],
        "role": "tool",
        "name": name,
        "content": str(result)
    }

async def handle_parallel_tool_calls(tool_calls: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
    """Executes all tool calls emitted in a single turn concurrently."""
    print(f"⚡ Executing {len(tool_calls)} tool calls in parallel...")
    tasks = [execute_tool_call_async(tc) for tc in tool_calls]
    return await asyncio.gather(*tasks)

TypeScript Multi-Tool Execution (Promise.all)

export async function executeParallelTools(toolCalls: ToolCall[]): Promise<ToolResult[]> {
  return await Promise.all(
    toolCalls.map(async (call) => {
      const args = JSON.parse(call.function.arguments);
      const output = await executeLocalTool(call.function.name, args);
      return {
        tool_call_id: call.id,
        role: "tool",
        name: call.function.name,
        content: output,
      };
    })
  );
}

Benchmark Comparison

Investigating an authentication bug spanning 4 related source files:

Agent Operational Directive

MANDATORY: When an agent requires context from multiple independent files or search queries, it must emit all tool calls in a single turn response (tool_calls array) rather than executing sequential turns.