Langchain Agents Middleware

Agentic Middleware

Middleware is how you compose cross-cutting agent behavior in LangChain v1+. It plugs into create_agent(...) (and is the underlying implementation of DeepAgents). For any production agent, the question is "which middlewares" — not "do I need middleware".

The model

from langchain.agents import create_agent
from langchain.agents.middleware import (
    SummarizationMiddleware,
    ModelRetryMiddleware,
    ModelFallbackMiddleware,
    ModelCallLimitMiddleware,
    ToolRetryMiddleware,
    PIIMiddleware,
)

agent = create_agent(
    model="claude-sonnet-4-6",
    tools=[...],
    middleware=[
        ModelRetryMiddleware(max_retries=3, backoff_factor=2.0, initial_delay=1.0),
        ModelFallbackMiddleware("openai:gpt-4o-mini"),
        ModelCallLimitMiddleware(run_limit=50),
        ToolRetryMiddleware(max_retries=3, backoff_factor=2.0),
        SummarizationMiddleware(model="claude-haiku-4-5", trigger=("tokens", 4000), keep=("messages", 20)),
        PIIMiddleware("email", strategy="redact", apply_to_input=True),
    ],
)

create_agent returns a compiled LangGraph. All Runnable semantics apply: .invoke, .ainvoke, .stream, .astream, langgraph dev, langgraph build, etc.

Lifecycle hooks (for custom middleware)

Every middleware can override one or more of these:

Node-style hooks (before_* / after_*) run sequentially. Wrap-style hooks compose like Python decorators — first middleware in the list is the outermost wrapper.

Built-in middlewares (provider-agnostic)

Import from langchain.agents.middleware:

DeepAgents-specific (import from deepagents.middleware):

create_deep_agent(...) is a thin wrapper over create_agent(...) that pre-installs FilesystemMiddleware + SubAgentMiddleware + TodoListMiddleware. You can compose additional middlewares on top.

Production middleware stack (start here)

For any production agent, this is the default stack to copy and tune:

middleware=[
    # Cost containment (set BEFORE retries — limits the multiplier)
    ModelCallLimitMiddleware(run_limit=50),
    ToolCallLimitMiddleware(run_limit=200),

    # Resilience to transient failures
    ModelRetryMiddleware(max_retries=3, backoff_factor=2.0, initial_delay=1.0),
    ToolRetryMiddleware(max_retries=3, backoff_factor=2.0, initial_delay=1.0),

    # Provider-level resilience
    ModelFallbackMiddleware("openai:gpt-4o-mini"),

    # Long-conversation hygiene
    SummarizationMiddleware(model="claude-haiku-4-5", trigger=("tokens", 8000), keep=("messages", 20)),

    # Privacy (only if user input may contain PII)
    PIIMiddleware("email", strategy="redact", apply_to_input=True),
    PIIMiddleware("credit_card", strategy="mask", apply_to_input=True),
]

Add HumanInTheLoopMiddleware for any tool that touches money, sends external messages, or makes irreversible changes. Requires a checkpointer (InMemorySaver for dev, PostgresSaver for production — see the deploy skill).

Custom middleware

Inherit from AgentMiddleware:

from typing import Any, Callable
from langchain.agents.middleware import (
    AgentMiddleware, AgentState, ModelRequest, ModelResponse,
)
from langchain.tools.tool_node import ToolCallRequest
from langchain.messages import ToolMessage
from langgraph.types import Command


class TokenBudgetMiddleware(AgentMiddleware):
    """Hard-cap total tokens across the run. Halts the agent when exceeded."""

    def __init__(self, budget: int) -> None:
        self.budget = budget

    def wrap_model_call(
        self,
        request: ModelRequest,
        handler: Callable[[ModelRequest], ModelResponse],
    ) -> ModelResponse:
        used = request.state.get("tokens_used", 0)
        if used >= self.budget:
            # short-circuit: return a synthetic "stop" response without calling the model
            return ModelResponse(
                messages=[{"role": "assistant", "content": "Token budget exceeded."}],
                command=Command(goto="__end__"),
            )
        response = handler(request)
        # ... extract token count from response.usage and add to state
        return response

If you need extra fields in state, declare them on a subclass of AgentState and set state_schema = MyState on the middleware class.

Hard rules

• Order matters. Limits before retries (so retries don't burn through your budget). Privacy redaction before logging. Summarization should run before the model call, not after.
• HumanInTheLoopMiddleware needs a checkpointer. Without one, interrupts have nothing to resume from.
• LLMToolEmulator is a testing-only middleware. Never ship it.
• Retries cost money. A max_retries=3 with backoff_factor=2 means up to 4 calls per failure. Set ModelCallLimitMiddleware BEFORE retries to cap the worst-case cost.
• Don't roll your own retry/fallback/limit. The built-ins handle the edge cases (jitter, retryable error classification, streaming-aware wrapping). Custom middleware is for app-specific concerns.

Skills to load alongside this one

• langchain-agents-deploy — productionisation: durable execution, checkpointers, deployment.
• langchain-agents-observability — tracing what middleware actually does at runtime.
• langchain-agents-langgraph-code — when to drop down to raw StateGraph (rare, but real cases exist).

Source: cwijayasundara/agent_cli_langchain — distributed by TomeVault.