lauraflorentin Exception Handling & Recovery
Exception Handling ensures that an agentic system degrades gracefully rather than crashing. In the nondeterministic world of LLMs, failures are common: models hallucinate, APIs time out, and outputs are malformed. This pattern wraps critical operations in "try/catch" blocks that trigger recovery agents or fallback strategies.
When to Use
- Production Systems: Essential for any user-facing application.
- Unreliable Tools: When using 3rd-party APIs that might be down or rate-limited.
- Structured Output: When the model occasionally fails to output valid JSON.
- Safety: When a tool might return dangerous or unexpected data.
Use Cases
- API Fallback: "Primary model API failed? Switch to backup model API." or "Tool A failed? Try Tool B."
- Refusal Handling: If the model refuses to answer (due to safety filters), catch the refusal and rephrase the prompt or explain why it can't answer.
- Validation Repair: If JSON validation fails, pass the error back to the model to fix the syntax.
Implementation Pattern
def resilient_tool_call(tool_name, args):
max_retries = 3
for attempt in range(max_retries):
try:
# Try to execute the tool
return execute_tool(tool_name, args)
except RateLimitError:
# Specific handling for known errors
backoff_sleep(attempt)
except ValidationError as e:
# Self-Correction: Ask the model to fix its input
print(f"Validation failed: {e}. Asking model to fix...")
args = repair_agent.fix_inputs(tool_name, args, error=e)
except Exception as e:
# General fallback
log_error(e)
return fallback_strategy(tool_name)
raise SystemError("Max retries exceeded")
Examples
Input: An agent making API calls that sometimes time out.
@retry(max_attempts=3, backoff=exponential(base=2))
def call_api(endpoint, payload):
try:
return requests.post(endpoint, json=payload, timeout=10)
except Timeout:
log.warning(f"Timeout on {endpoint}, retrying...")
raise # triggers retry decorator
except RateLimitError as e:
wait(e.retry_after)
raise
except FatalError:
alert_human("Non-recoverable error — requires manual intervention")
raise # don't retry
Troubleshooting
| Problem | Cause | Fix |
|---|---|---|
| Retries amplify the problem | Rate limit errors being retried too fast | Use exponential backoff with jitter; respect Retry-After headers |
| Agent silently swallows errors | Bare except clause |
Always log the exception with full stack trace before handling |
| Cascading failures | No circuit breaker | Implement circuit breaker: after 5 failures, open circuit for 60s |
| Human escalation never fires | Alert threshold too high | Test alerting path in staging; set alerting on first fatal-class error |