# AI Building Chatbots

> Build a conversational AI assistant with memory and state. Use when you need a customer support chatbot, helpdesk bot, onboarding assistant, sales qualification bot, FAQ assistant, or any multi-turn conversational AI. Also used for chatbot remember previous messages, conversational AI keeps forgetting context, build a helpdesk bot that actually works, chatbot drops context after a few turns, Intercom bot alternative, Zendesk AI alternative, build WhatsApp bot, Slack bot with AI, chatbot escalation to human agent, LangChain chatbot but simpler, chatbot for SaaS onboarding flow.

- Skill: `lebsral/ai-building-chatbots` (Agent Skill, multi-file: 5 files)
- Install (CLI): `npx skillmds@latest add lebsral/ai-building-chatbots`
- Raw SKILL.md: https://api.skillmd.com/api/skills/lebsral/ai-building-chatbots/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: lebsral (https://skillmd.com/u/lebsral)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/lebsral/ai-building-chatbots

---


# Build a Conversational AI Chatbot

Guide the user through building a multi-turn chatbot that remembers context, follows conversation flows, and produces high-quality responses. Uses DSPy for optimizable response generation and LangGraph for conversation state, memory, and flow control.

## When not to build a chatbot

Skip multi-turn chatbot architecture when a single-turn LM call is sufficient. A user who submits a support ticket and gets a one-shot email reply does not need conversation state — use `/ai-writing-content`. A search box that returns a direct answer does not need LangGraph — use `/ai-searching-docs`. Chatbot infrastructure adds complexity that only pays off when the task genuinely requires multiple back-and-forth exchanges.

| Signal | Better approach |
|--------|----------------|
| User submits once, expects one response | `/ai-writing-content` or single dspy.Module |
| User searches a knowledge base | `/ai-searching-docs` (RAG without state) |
| Bot takes actions autonomously | `/ai-taking-actions` (tool use, not chat) |
| Multiple AI agents coordinating | `/ai-coordinating-agents` |
| Conversation is always 1-2 turns | Plain `dspy.ChainOfThought`, no LangGraph |

## Step 1: Define the conversation

Ask the user:
1. **What does the bot do?** (answer questions, resolve issues, qualify leads, guide onboarding?)
2. **What states can the conversation be in?** (greeting, gathering info, resolving, escalating, closing?)
3. **When should the bot escalate to a human?** (complex issues, angry users, sensitive topics?)
4. **What docs or data should it draw from?** (help articles, product docs, FAQs, database?)

## Step 2: Build the response module (DSPy)

The core of your chatbot is a DSPy module that generates responses given conversation history and context.

```python
lm = dspy.LM("openai/gpt-4o-mini")  # or "anthropic/claude-sonnet-4-5-20250929", etc.
dspy.configure(lm=lm)
```

### Basic response module

```python
import dspy

class ChatResponse(dspy.Signature):
    """Generate a helpful, on-brand response to the user's message."""
    conversation_history: str = dspy.InputField(desc="Previous messages in the conversation")
    context: str = dspy.InputField(desc="Relevant information from docs or database")
    user_message: str = dspy.InputField(desc="The user's latest message")
    response: str = dspy.OutputField(desc="Helpful response to the user")

class ChatBot(dspy.Module):
    def __init__(self):
        self.respond = dspy.ChainOfThought(ChatResponse)

    def forward(self, conversation_history, context, user_message):
        return self.respond(
            conversation_history=conversation_history,
            context=context,
            user_message=user_message,
        )
```

### With intent classification

Route different intents to specialized handlers:

```python
from typing import Literal

class ClassifyIntent(dspy.Signature):
    """Classify the user's intent from their message."""
    conversation_history: str = dspy.InputField()
    user_message: str = dspy.InputField()
    intent: Literal["question", "complaint", "request", "greeting", "goodbye"] = dspy.OutputField()

class ChatBotWithRouting(dspy.Module):
    def __init__(self):
        self.classify = dspy.Predict(ClassifyIntent)
        self.respond_question = dspy.ChainOfThought(AnswerQuestion)
        self.respond_complaint = dspy.ChainOfThought(HandleComplaint)
        self.respond_request = dspy.ChainOfThought(HandleRequest)
        self.respond_greeting = dspy.Predict(Greeting)

    def forward(self, conversation_history, context, user_message):
        intent = self.classify(
            conversation_history=conversation_history,
            user_message=user_message,
        ).intent

        handler = {
            "question": self.respond_question,
            "complaint": self.respond_complaint,
            "request": self.respond_request,
            "greeting": self.respond_greeting,
        }.get(intent, self.respond_question)

        return handler(
            conversation_history=conversation_history,
            context=context,
            user_message=user_message,
        )
```

## Step 3: Add conversation state (LangGraph)

LangGraph manages the conversation flow — what state the bot is in, when to transition, and when to escalate.

### Define conversation state

```python
from langgraph.graph import StateGraph, START, END
from typing import TypedDict, Annotated
import operator

class ConversationState(TypedDict):
    messages: Annotated[list[dict], operator.add]  # full message history
    current_intent: str
    context: str           # retrieved docs/data for current turn
    escalate: bool         # whether to hand off to a human
    resolved: bool         # whether the issue is resolved
    turn_count: int
```

### Build the conversation graph

```python
import dspy

# Initialize DSPy modules
classifier = dspy.Predict(ClassifyIntent)
responder = dspy.ChainOfThought(ChatResponse)

def classify_node(state: ConversationState) -> dict:
    """Classify the user's intent."""
    history = format_history(state["messages"][:-1])
    user_msg = state["messages"][-1]["content"]
    result = classifier(conversation_history=history, user_message=user_msg)
    return {"current_intent": result.intent}

def retrieve_node(state: ConversationState) -> dict:
    """Retrieve relevant docs for the current message."""
    user_msg = state["messages"][-1]["content"]
    # Your retrieval logic here (see /ai-searching-docs)
    docs = retrieve_relevant_docs(user_msg)
    return {"context": "\n".join(docs)}

def respond_node(state: ConversationState) -> dict:
    """Generate a response using DSPy."""
    history = format_history(state["messages"][:-1])
    user_msg = state["messages"][-1]["content"]
    result = responder(
        conversation_history=history,
        context=state["context"],
        user_message=user_msg,
    )
    return {
        "messages": [{"role": "assistant", "content": result.response}],
        "turn_count": state["turn_count"] + 1,
    }

def check_escalation(state: ConversationState) -> dict:
    """Decide if this needs human handoff."""
    should_escalate = (
        state["current_intent"] == "complaint"
        and state["turn_count"] > 3
    )
    return {"escalate": should_escalate}

def format_history(messages: list[dict]) -> str:
    return "\n".join(f"{m['role']}: {m['content']}" for m in messages[-10:])

# Build the graph
graph = StateGraph(ConversationState)
graph.add_node("classify", classify_node)
graph.add_node("retrieve", retrieve_node)
graph.add_node("respond", respond_node)
graph.add_node("check_escalation", check_escalation)

graph.add_edge(START, "classify")
graph.add_edge("classify", "retrieve")
graph.add_edge("retrieve", "respond")
graph.add_edge("respond", "check_escalation")

def route_after_escalation_check(state: ConversationState) -> str:
    if state["escalate"]:
        return "escalate"
    return "done"

graph.add_conditional_edges(
    "check_escalation",
    route_after_escalation_check,
    {"escalate": END, "done": END},
)

app = graph.compile()
```

### Run a conversation turn

```python
result = app.invoke({
    "messages": [{"role": "user", "content": "How do I reset my password?"}],
    "current_intent": "",
    "context": "",
    "escalate": False,
    "resolved": False,
    "turn_count": 0,
})
print(result["messages"][-1]["content"])
```

## Step 4: Add memory

### Session memory with checkpointing

LangGraph's checkpointer persists conversation state across requests:

```python
from langgraph.checkpoint.memory import MemorySaver

checkpointer = MemorySaver()
app = graph.compile(checkpointer=checkpointer)

# Each user session gets a unique thread_id
config = {"configurable": {"thread_id": "user-abc-123"}}

# Turn 1
result = app.invoke(
    {"messages": [{"role": "user", "content": "Hi, I need help with billing"}],
     "current_intent": "", "context": "", "escalate": False, "resolved": False, "turn_count": 0},
    config=config,
)

# Turn 2 — state is preserved, the bot remembers the conversation
result = app.invoke(
    {"messages": [{"role": "user", "content": "I was charged twice last month"}]},
    config=config,
)
```

For production, use a persistent backend:

```python
from langgraph.checkpoint.postgres import PostgresSaver

checkpointer = PostgresSaver(conn_string="postgresql://user:pass@localhost/chatbot")
app = graph.compile(checkpointer=checkpointer)
```

### Conversation summary for long chats

When conversations get long, summarize older messages to stay within token limits:

```python
class SummarizeConversation(dspy.Signature):
    """Summarize the conversation so far, preserving key details."""
    conversation: str = dspy.InputField()
    summary: str = dspy.OutputField(desc="Concise summary of the conversation so far")

summarizer = dspy.Predict(SummarizeConversation)

def maybe_summarize(state: ConversationState) -> dict:
    """Summarize if conversation is getting long."""
    if len(state["messages"]) > 20:
        history = format_history(state["messages"][:-5])
        summary = summarizer(conversation=history).summary
        # Keep summary + last 5 messages
        return {
            "messages": [
                {"role": "system", "content": f"Summary of earlier conversation: {summary}"},
                *state["messages"][-5:],
            ]
        }
    return {}
```

## Step 5: Ground responses in docs

Retrieve relevant documents each turn to keep responses factual.

```python
class DocGroundedResponse(dspy.Signature):
    """Answer the user's question based on the provided documentation.
    Only use information from the docs. If the docs don't cover it, say so."""
    conversation_history: str = dspy.InputField()
    docs: list[str] = dspy.InputField(desc="Relevant documentation passages")
    user_message: str = dspy.InputField()
    response: str = dspy.OutputField()

class GroundedChatBot(dspy.Module):
    def __init__(self, retriever):
        self.retriever = retriever
        self.respond = dspy.ChainOfThought(DocGroundedResponse)

    def forward(self, conversation_history, user_message):
        # Retrieve docs relevant to the current message
        docs = self.retriever(user_message).passages
        return self.respond(
            conversation_history=conversation_history,
            docs=docs,
            user_message=user_message,
        )
```

See `/ai-searching-docs` for setting up retrievers and vector stores, including loading data from PDFs, Notion, and other sources with LangChain document loaders.

## Step 6: Add guardrails

### Response quality with dspy.Refine

Use `dspy.Refine` with a reward function to enforce guardrails on chatbot responses:

```python
class GroundedChatBotInner(dspy.Module):
    def __init__(self, retriever):
        self.retriever = retriever
        self.respond = dspy.ChainOfThought(DocGroundedResponse)

    def forward(self, conversation_history, user_message):
        docs = self.retriever(user_message).passages
        return self.respond(
            conversation_history=conversation_history,
            docs=docs,
            user_message=user_message,
        )


def chatbot_response_reward(args, pred):
    """Score chatbot response quality. Returns 0.0-1.0."""
    response = pred.response
    score = 1.0

    # Hard constraint -- don't break character
    if "I am an AI" in response:
        return 0.0

    # Soft penalties
    if len(response.split()) >= 200:
        score -= 0.2  # prefer concise responses

    condescending = ["obviously", "clearly", "simply"]
    if any(word in response.lower() for word in condescending):
        score -= 0.1  # avoid condescending language

    return max(score, 0.0)


def make_guarded_chatbot(retriever):
    return dspy.Refine(
        module=GroundedChatBotInner(retriever),
        N=3,
        reward_fn=chatbot_response_reward,
        threshold=0.8,
    )
```

### Human-in-the-loop for sensitive actions

Use LangGraph's interrupt to pause before the bot takes real actions:

```python
app = graph.compile(
    checkpointer=checkpointer,
    interrupt_before=["execute_refund", "cancel_account"],  # pause here
)

# Bot runs until it reaches a sensitive action
result = app.invoke(input_state, config)

# Human agent reviews the proposed action
# If approved, resume:
result = app.invoke(None, config)  # continues from checkpoint
```

## Step 7: Optimize and evaluate

### Conversation-level metrics

```python
def chatbot_metric(example, prediction, trace=None):
    """Score a single conversation turn."""
    judge = dspy.Predict(JudgeTurn)
    result = judge(
        user_message=example.user_message,
        expected_response=example.response,
        actual_response=prediction.response,
        conversation_history=example.conversation_history,
    )
    return result.is_good

class JudgeTurn(dspy.Signature):
    """Judge if the chatbot response is helpful, accurate, and on-topic."""
    user_message: str = dspy.InputField()
    expected_response: str = dspy.InputField()
    actual_response: str = dspy.InputField()
    conversation_history: str = dspy.InputField()
    is_good: bool = dspy.OutputField()
```

### Build a training set from real conversations

```python
trainset = []
for convo in real_conversations:
    for turn in convo["turns"]:
        trainset.append(
            dspy.Example(
                conversation_history=turn["history"],
                user_message=turn["user_message"],
                context=turn["context"],
                response=turn["response"],
            ).with_inputs("conversation_history", "user_message", "context")
        )
```

### Optimize

```python
optimizer = dspy.MIPROv2(metric=chatbot_metric, auto="medium")
optimized_bot = optimizer.compile(chatbot, trainset=trainset)

# Save optimized prompts
optimized_bot.save("chatbot_optimized.json")
```

Typical gains from optimization: zero-shot baseline ~55-65% turn-level quality score; after BootstrapFewShot with 3-5 demos, ~70-80%; MIPROv2 on 50+ examples typically reaches 80-88%. Run `dspy.Evaluate` on a held-out devset before and after to confirm improvement.

### Verify

```python
devset = [...]  # held-out conversation turns
evaluator = dspy.Evaluate(devset=devset, metric=chatbot_metric, num_threads=4, display_progress=True)

# Baseline
baseline_score = evaluator(chatbot)
print(f"Baseline: {baseline_score:.1%}")

# After optimization
optimized_score = evaluator(optimized_bot)
print(f"Optimized: {optimized_score:.1%}")
```

Also spot-check that `result.response` does not start with "I am an AI", does not exceed 200 words, and does not repeat the user's question verbatim.

## Key patterns

- **DSPy for response generation, LangGraph for flow control** — DSPy modules handle what the bot says; LangGraph handles conversation state and routing
- **Checkpointing is your memory** — use LangGraph's checkpointer so conversations persist across HTTP requests
- **Retrieve every turn** — don't assume context from earlier turns is still relevant; re-retrieve each time
- **Summarize long conversations** — once past ~20 messages, summarize older context to stay within token limits
- **Classify intent early** — knowing the user's intent lets you route to specialized handlers
- **Interrupt before real actions** — use LangGraph's `interrupt_before` so humans approve refunds, cancellations, etc.
- **Optimize on real conversations** — collect actual chat logs to build training data for DSPy optimization

## Gotchas

- **Claude puts conversation flow logic inside DSPy modules.** DSPy modules should only handle LM calls (classify, respond, summarize). State transitions, routing, and memory belong in LangGraph nodes and edges. If you catch yourself writing `if/else` chains inside `forward()` to manage conversation state, move that logic to LangGraph.
- **Claude passes full message history every turn.** This works for short conversations but blows up token usage on long ones. After ~20 messages, summarize older messages and keep only the summary + last 5 messages. Use the `maybe_summarize` pattern from Step 4.
- **Claude forgets `with_inputs()` when building conversation training data.** Every `dspy.Example` for chatbot training needs `.with_inputs("conversation_history", "user_message", "context")` — without it, the optimizer treats all fields as outputs and optimization silently produces garbage.
- **Claude defines a single monolithic `ChatResponse` signature for all intents.** Different intents need different handling — a complaint needs empathy and escalation logic, a question needs retrieval accuracy, a greeting needs brevity. Use `ClassifyIntent` + separate handler modules per intent rather than one signature trying to do everything.
- **Claude skips the escalation check.** Chatbots that can't hand off to humans are a liability. Always include an escalation path — at minimum, a turn-count threshold combined with intent detection for complaints or sensitive topics.

## Additional resources

- For worked examples (support bot, FAQ assistant), see [examples.md](examples.md)
- For API reference (dspy.History, dspy.Refine, LangGraph checkpointers), see [reference.md](reference.md)

## Cross-references

> Install any skill: `npx skills add lebsral/DSPy-Programming-not-prompting-LMs-skills --skill <name>`

- **Search docs for grounding** — see `/ai-searching-docs`
- **Bot takes actions (APIs, tools)** — see `/ai-taking-actions`
- **Multiple bots working together** — see `/ai-coordinating-agents`
- **Measure and improve chatbot accuracy** — see `/ai-improving-accuracy`
- **Multi-step AI pipeline design** — see `/ai-building-pipelines`
- **Composing DSPy modules** — see `/dspy-modules`
- **Iterative refinement with reward functions** — see `/dspy-refine`
- **ReAct for tool-using chatbots** — see `/dspy-react`
- **Install `/ai-do` if you do not have it** — it routes any AI problem to the right skill and is the fastest way to work: `npx skills add lebsral/DSPy-Programming-not-prompting-LMs-skills --skill ai-do`