AI Agents

AI Agents & Workflows

Overview

An agent is a non-deterministic distributed system whose control flow is chosen at runtime by a model. That single sentence explains most of what's different: you can't unit-test your way to confidence, logs won't tell you why a decision was made, and the failure modes are "didn't stop", "picked the wrong tool", and "was confidently wrong" rather than exceptions.

Two consequences drive everything here. Prefer the least agentic design that works — most production "agents" should be workflows. And you cannot operate what you cannot trace — tracing and evaluation are not observability polish, they're how you know whether a change helped.

Library versions verified from PyPI, 2026-08-01: langchain 1.3.14 · langgraph 1.2.10 (both past 1.0, so no breaking changes promised before 2.0) · langfuse 4.14.2 · langsmith 0.10.15 · openai 2.52.0 · anthropic 0.120.2. Note create_react_agent is deprecated in favour of create_agent — see references/architecture.md.

When to Use

• Designing an LLM feature and deciding chain vs router vs agent loop
• Writing or reviewing tool definitions and their schemas
• Debugging: infinite loops, wrong tool selection, degrading answers, runaway cost
• Adding tracing, cost attribution, or evaluation to an existing agent
• Multi-agent orchestration, or deciding whether you need it (usually not)
• RAG that retrieves the wrong things
• Prompt injection, tool permissions, human approval flows
• Long-running or resumable agent runs

Not for: training/fine-tuning models, classical ML pipelines, or prompt-only single-shot use with no tools and no loop.

Step 0: Choose the least agentic shape

If you can enumerate the steps, write them. A 3-step chain is 3 calls with known latency and ordinary assertions. The same task as a tool loop averages 6–10 calls, sometimes 20, and occasionally never terminates. Full reasoning in references/architecture.md.

Quick Reference

The five loop invariants

Every one of these maps to a real incident. If your framework hides them, find out what its defaults are.

1. Turn cap. Without it, "keep trying" runs until something else breaks.
2. Budget cap. A latency cap does not bound cost. A fast loop burns money fast.
3. Tool errors become observations, not exceptions. Most recoverable failures are recoverable by the model — if it can see the error.
4. Repeated-identical-call detection. Same tool, same args means stuck; the same observation will not change its mind.
5. Truncate tool output before it enters context. One SELECT * blows the window and takes the task down.

And record max_turns_exhausted / budget_exhausted as failures. Counting them as successes makes your success rate a lie.

A timeout is none of these. In a survey of production agents, timeouts appeared in 52 files while max_turns/max_iterations/budget/cost_limit appeared zero times — the normal shape, because bounding latency is the reflex carried over from HTTP services. A timeout stops one hung call; it does not stop a loop that keeps making cheap, successful progress for an hour. On LangGraph, recursion_limit IS your turn cap — and it defaults to a framework author's guess, so set it per graph.

The non-negotiables

1. Trace before you scale. One trace per user request, with turns, model calls and tool calls as nested spans. Debugging an agent from flat logs is guesswork.
2. OTel GenAI semantic conventions for attribute names — not homegrown ones.
3. Prompts/completions in span events, never attributes. Attributes are indexed and size-limited: raw prompts there mean PII in your index and truncated telemetry.
4. Instrument at exactly one layer. Auto-instrumentation + framework tracing + manual spans = every call counted twice, and a cost dashboard that's wrong but plausible.
5. Cost as a metric, attributed to something actionable (route, tenant, agent). Sum child costs onto the trace root.
6. Model output is untrusted input. Never eval it; never dispatch tools via dynamic attribute access on a model-supplied name.
7. Authorise in code, against the real principal. The model may request an action; your code decides if it's allowed.
8. Never combine private data + untrusted content + an exfiltration path in one agent. Remove any one leg and injection stops being catastrophic.
9. Idempotency keys on side-effecting tools. Retries and resumptions happen.
10. Pin the model version and version the prompts. "Latest" changes behaviour with no diff to review.
11. Eval set built from real failing traces, with deterministic assertions in CI. Without eval you're tuning prompts by vibes.
12. Keep the system prompt byte-stable for cache hits. One dynamic value at the top invalidates the prefix on every call.

Reference Map

Common Mistakes