lauraflorentin Routing
Routing acts as the traffic controller for an agentic system. Instead of a single generalist agent trying to handle every type of user request, a "Router" (often a fast, lightweight LLM or a classifier) analyzes the intent of the incoming message and delegates it to a specialized handler. This creates a system that is modular, scalable, and easier to maintain.
When to Use
- Specialization: When you have diverse tasks that require different prompts, tools, or context (e.g., Tech Support vs. Sales vs. Refund).
- Cost Optimization: To route simple queries to smaller/cheaper models and complex queries to reasoning models.
- Security: To ensure sensitive requests are handled by agents with stricter guardrails.
- SoC (Separation of Concerns): To keep individual agent prompts focused and clean.
Use Cases
- Customer Service Dispatch: Routing "My internet is down" to Tech Support and "How much is the upgrade?" to Sales.
- Model Selection: Routing logic puzzles to o1 and creative writing to GPT-4.
- Tool Selection: Deciding whether to use a Search tool, a Calculator, or a Database Query tool based on the question.
Implementation Pattern
def routing_workflow(user_query):
# Step 1: Classification
# The router's only job is to pick the right path.
intent = classify_intent(
prompt="Classify this query into: [SALES, SUPPORT, BILLING]",
input=user_query
)
# Step 2: Delegation
# Based on the intent, we call a specific sub-agent.
if intent == "SALES":
return sales_agent.run(user_query)
elif intent == "SUPPORT":
return support_agent.run(user_query)
elif intent == "BILLING":
return billing_agent.run(user_query)
else:
return default_agent.run(user_query)
Troubleshooting
| Problem | Cause | Fix |
|---|---|---|
| Wrong agent selected | Classifier underfitted | Add 50+ labeled examples per class; re-train or few-shot prompt |
| New query types not routed | No catch-all route | Add a "general" fallback route that handles out-of-distribution inputs |
| Routing adds too much latency | Heavy classifier | Use a lightweight keyword router as first stage; only call LLM when uncertain |
| Router and agent disagree on task scope | Misaligned schemas | Define a shared task schema; router populates it, agent consumes it |