akillness Agent Tool Routing
Overview
Past ~15 tools, the LLM's tool-selection accuracy collapses. The fix is a router layer: discover capabilities, narrow the tool set per turn, then hand a small focused set to the LLM. This skill builds that router.
When to use
- Agent has 20+ tools and picks the wrong one
- Multiple MCP servers — too many overlapping tools
- Tools come and go at runtime (per-tenant, per-permission)
- Need fallback chains (preferred tool → cheaper tool → cached)
- Cost: every unused tool description wastes input tokens
Architecture
Request → [Capability Discovery] → [Router] → [Top-K Tools] → LLM
↑ ↑
registry index semantic + policy
Tool registry
@dataclass
class ToolEntry:
name: str
description: str # LLM-facing
schema: dict # JSON Schema
embedding: list[float] # for semantic match
tags: set[str] # billing, read, write, hitl
cost_usd_per_call: float
p95_latency_ms: int
requires_scopes: set[str]
fallback: str | None # tool name to try if this fails
Sources: MCP server discovery, local tools.yaml, plugins.
Router policies
| Policy | When |
|---|---|
| Semantic top-K | Embed query, return K nearest tools |
| Tag-filtered | Restrict to tags matching turn intent |
| Scope-filtered | Drop tools the caller lacks scopes for |
| Cost-aware | Prefer cheaper tool if quality equal |
| Sticky | Reuse last turn's tool if still relevant |
Compose: scope filter → tag filter → semantic top-K → cost re-rank.
Quick start
def route_tools(query, caller_scopes, k=8):
candidates = registry.all()
candidates = [t for t in candidates if t.requires_scopes <= caller_scopes]
q_emb = embed(query)
ranked = sorted(candidates, key=lambda t: cosine(q_emb, t.embedding), reverse=True)
return ranked[:k]
Then expose only the top-K to the LLM in the tool list.
Fallback chains
def call_with_fallback(tool_name, args):
cur = registry.get(tool_name)
while cur:
try:
return cur.invoke(args)
except (ToolTimeout, ToolUnavailable):
cur = registry.get(cur.fallback) if cur.fallback else None
raise AllFallbacksExhausted(tool_name)
Tool description rules
LLM picks tools from descriptions. Each must answer:
- Verb — what action it performs
- When — what triggers using it
- Returns — shape of output
- Side effects — what state changes
- Cost/latency hint — "fast", "slow", "expensive"
Bad: "Look up customer."
Good: "Fetch customer by email or ID. Read-only, ~50ms. Returns {id, name, tier, status}. Use when you need account details before any billing action."
Discovery from MCP
from mcp.client import Client
async with Client("http://billing-mcp:8080") as c:
tools = await c.list_tools()
for t in tools:
registry.add(ToolEntry(
name=t.name, description=t.description, schema=t.inputSchema,
embedding=embed(t.description), tags=infer_tags(t),
cost_usd_per_call=0.0, p95_latency_ms=200,
requires_scopes=set(), fallback=None,
))
Further reading
- Anthropic tool-use best practices
- BFCL / Tool-Bench — router benchmarks (2026)
- MCP capability discovery (
mcp-server-design) - Fallback patterns (resilience4j, polly)