AI Assistant Chat

Self-Hosted AI Assistant Chat (CopilotKit + LangGraph)

Build production-ready AI chat interfaces with full infrastructure control. This pattern connects a Next.js frontend (CopilotKit) to a Python backend (LangGraph) without relying on any hosted AI orchestration services.

What This Pattern Is NOT

• NOT Hosted CopilotKit: No cloud.copilotkit.ai - the CopilotKit runtime runs in your Next.js API route
• NOT Hosted LangGraph: No LangSmith Cloud or LangGraph Platform - the agent runs on a self-hosted FastAPI server
• NOT SaaS Dependencies: Full control over data flow, no external orchestration services

Architecture Overview

Browser
   ↓ (HTTP/SSE)
Next.js Frontend (port 3000)
   ├── React UI (CopilotKit components)
   └── /api/copilotkit (CopilotRuntime bridge)
       ↓ (HTTP POST)
FastAPI Backend (port 8123)
   ├── LangGraph Agent (state machine)
   └── Tool Execution (backend + frontend actions)

For detailed architecture diagrams and data flow: See references/architecture.md

Quick Start

1. Frontend Setup (Next.js + CopilotKit)

Install CopilotKit packages:

pnpm add @copilotkit/react-core @copilotkit/react-ui @copilotkit/runtime

Wrap app with CopilotKit provider pointing to local runtime:

// layout.tsx
import { CopilotKit } from "@copilotkit/react-core";

export default function Layout({ children }) {
  return (
    <CopilotKit runtimeUrl="/api/copilotkit" agent="my_agent">
      {children}
    </CopilotKit>
  );
}

Create the runtime bridge (this is the self-hosted runtime):

// app/api/copilotkit/route.ts
import { CopilotRuntime, ExperimentalEmptyAdapter } from "@copilotkit/runtime";
import { LangGraphHttpAgent } from "@copilotkit/runtime/agents";

const AGENT_URL = process.env.AGENT_URL || "http://localhost:8123";

export async function POST(req: Request) {
  const agent = new LangGraphHttpAgent({
    name: "my_agent",
    url: AGENT_URL,
    agentId: "my_agent",
  });

  const runtime = new CopilotRuntime({
    agents: [agent],
    modelAdapter: new ExperimentalEmptyAdapter(),
  });

  return runtime.streamHttpServerResponse(req, req.headers);
}

For complete frontend setup: See references/frontend-setup.md

2. Backend Setup (FastAPI + LangGraph)

Install Python dependencies:

pip install langgraph langchain-openai copilotkit ag-ui-langgraph fastapi uvicorn

Create the LangGraph agent:

# agent/src/agent.py
from langgraph.graph import StateGraph, END
from copilotkit import CopilotKitState

class AgentState(CopilotKitState):
    # Add custom state fields synced with frontend
    custom_data: list[str] = []

def chat_node(state: AgentState, config):
    model = ChatOpenAI(model="gpt-4o")
    # Bind tools from CopilotKit state
    model_with_tools = model.bind_tools(state.copilotkit.tools)
    response = model_with_tools.invoke(state.messages, config)
    return {"messages": [response]}

# Build graph
workflow = StateGraph(AgentState)
workflow.add_node("chat", chat_node)
workflow.set_entry_point("chat")
graph = workflow.compile(checkpointer=MemorySaver())

Create FastAPI server:

# agent/main.py
from fastapi import FastAPI
from ag_ui.langgraph import LangGraphAGUIAgent

app = FastAPI()
agent = LangGraphAGUIAgent(graph=graph, config={"configurable": {"thread_id": "1"}})

@app.post("/")
async def run_agent(request: Request):
    return agent.run(request)

For complete backend setup: See references/backend-setup.md

Key Integration Points

Shared State

State defined in AgentState automatically syncs between frontend and backend:

// Frontend: Access shared state
const { state } = useCoAgent<AgentState>();
console.log(state.custom_data);

# Backend: Modify shared state
def my_node(state: AgentState):
    return {"custom_data": state.custom_data + ["new item"]}

Frontend Actions (Generative UI)

Define actions in React that the backend can invoke:

useCopilotAction({
  name: "render_weather",
  description: "Display weather card",
  parameters: [{ name: "location", type: "string" }],
  renderAndWaitForResponse: ({ args }) => <WeatherCard location={args.location} />,
});

The agent can call render_weather and receive user responses.

Backend Tools

Define tools that execute server-side:

@tool
def search_database(query: str) -> str:
    """Search the internal database."""
    return db.search(query)

# Add to model binding
model.bind_tools([search_database] + state.copilotkit.tools)