Langchain Chat Models

langchain-chat-models (JavaScript/TypeScript)

Overview

Chat models in LangChain provide a unified interface for interacting with various LLM providers. They take a sequence of messages as input and return AI-generated messages as output. Chat models support features like tool calling, structured output, and streaming.

Key Concepts

• Chat Models: Accept messages (with roles: system, user, assistant) and return generated responses
• Providers: Different AI companies offering LLM APIs (OpenAI, Anthropic, Google, Azure, Bedrock, etc.)
• Tool Calling: Models can invoke functions/tools based on user queries
• Streaming: Real-time token-by-token response generation
• Structured Output: Models can return responses in specific formats (JSON, TypeScript types)

Provider Selection Decision Table

When to Choose Each Provider

Choose OpenAI if:

• You need strong function/tool calling capabilities
• You want fast response times
• You're building general-purpose applications

Choose Anthropic if:

• You need very long context windows (100k-200k tokens)
• Safety and constitutional AI principles are important
• You want high-quality analysis and reasoning

Choose Azure OpenAI if:

• You need enterprise SLAs and support
• Data residency and compliance are critical
• You're already using Microsoft Azure

Choose AWS Bedrock if:

• You're in the AWS ecosystem
• You want access to multiple model providers
• You need variety (Claude, Llama, Titan, etc.)

Choose Google (GenAI or Vertex) if:

• You need strong multimodal capabilities
• You're in the GCP ecosystem
• You want access to Gemini models

Code Examples

OpenAI Chat Model

import { ChatOpenAI } from "@langchain/openai";

// Basic initialization
const model = new ChatOpenAI({
  modelName: "gpt-4",
  temperature: 0.7,
  openAIApiKey: process.env.OPENAI_API_KEY, // Optional if set in env
});

// Invoke the model
const response = await model.invoke([
  { role: "system", content: "You are a helpful assistant." },
  { role: "user", content: "What is LangChain?" }
]);

console.log(response.content);

// Streaming responses
const stream = await model.stream("Tell me a story");
for await (const chunk of stream) {
  process.stdout.write(chunk.content);
}

Anthropic Chat Model

import { ChatAnthropic } from "@langchain/anthropic";

const model = new ChatAnthropic({
  modelName: "claude-3-opus-20240229",
  temperature: 0.7,
  anthropicApiKey: process.env.ANTHROPIC_API_KEY,
  maxTokens: 1024,
});

// Long context usage
const response = await model.invoke([
  { role: "user", content: "Analyze this long document..." }
]);

// With tool use
const modelWithTools = model.bindTools([
  {
    name: "get_weather",
    description: "Get weather for a location",
    input_schema: {
      type: "object",
      properties: {
        location: { type: "string" }
      },
      required: ["location"]
    }
  }
]);

Azure OpenAI Chat Model

import { AzureChatOpenAI } from "@langchain/openai";

const model = new AzureChatOpenAI({
  azureOpenAIApiKey: process.env.AZURE_OPENAI_API_KEY,
  azureOpenAIApiInstanceName: process.env.AZURE_OPENAI_API_INSTANCE_NAME,
  azureOpenAIApiDeploymentName: process.env.AZURE_OPENAI_API_DEPLOYMENT_NAME,
  azureOpenAIApiVersion: "2024-02-01",
  temperature: 0.7,
});

const response = await model.invoke("Hello, how are you?");

AWS Bedrock Chat Model

import { ChatBedrockConverse } from "@langchain/aws";

const model = new ChatBedrockConverse({
  model: "anthropic.claude-3-sonnet-20240229-v1:0",
  region: "us-east-1",
  credentials: {
    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
  },
});

const response = await model.invoke("What is AWS Bedrock?");

Google Generative AI

import { ChatGoogleGenerativeAI } from "@langchain/google-genai";

const model = new ChatGoogleGenerativeAI({
  modelName: "gemini-pro",
  apiKey: process.env.GOOGLE_API_KEY,
  temperature: 0.7,
});

const response = await model.invoke("Explain quantum computing");

Using initChatModel (Recommended)

import { initChatModel } from "langchain/chat_models/universal";

// Automatically select model based on environment
const model = await initChatModel("gpt-4", {
  modelProvider: "openai",
  temperature: 0.7,
});

// Or with Bedrock
const bedrockModel = await initChatModel(
  "anthropic.claude-3-sonnet-20240229-v1:0",
  { modelProvider: "bedrock" }
);

Tool Calling Example

import { ChatOpenAI } from "@langchain/openai";
import { z } from "zod";
import { tool } from "@langchain/core/tools";

// Define a tool
const weatherTool = tool(
  async ({ location }) => {
    return `The weather in ${location} is sunny and 72°F`;
  },
  {
    name: "get_weather",
    description: "Get the current weather for a location",
    schema: z.object({
      location: z.string().describe("The city name"),
    }),
  }
);

// Bind tools to model
const model = new ChatOpenAI({
  modelName: "gpt-4",
}).bindTools([weatherTool]);

const response = await model.invoke("What's the weather in San Francisco?");
console.log(response.tool_calls); // Model will suggest calling the weather tool

Boundaries

What Agents CAN Do

✅ Initialize any supported chat model provider

• Install required packages (@langchain/openai, @langchain/anthropic, etc.)
• Configure models with API keys and parameters

✅ Configure model parameters

• Set temperature, max tokens, top_p, frequency_penalty
• Configure streaming, timeout, and retry settings

✅ Use models for text generation

• Send messages and receive responses
• Stream responses token-by-token
• Use system prompts and multi-turn conversations

✅ Implement tool/function calling

• Bind tools to models that support it
• Parse tool call responses
• Execute tools and return results

✅ Switch between providers

• Use initChatModel for provider-agnostic code
• Change providers by updating configuration

What Agents CANNOT Do

❌ Create new model providers

• Cannot add support for unlisted LLM providers
• Must use existing LangChain integrations

❌ Bypass provider requirements

• Cannot use Azure OpenAI without deployment names
• Cannot skip required authentication credentials

❌ Modify model capabilities

• Cannot add tool calling to models that don't support it
• Cannot extend context windows beyond provider limits

❌ Access models without proper setup

• Cannot use providers without valid API keys
• Cannot bypass billing/quota limits

Gotchas

1. API Keys and Environment Variables

// ❌ BAD: Hardcoding API keys
const model = new ChatOpenAI({
  openAIApiKey: "sk-..."  // Never commit this!
});

// ✅ GOOD: Use environment variables
const model = new ChatOpenAI({
  openAIApiKey: process.env.OPENAI_API_KEY
});

Fix: Always use environment variables or secure key management systems.

2. Azure OpenAI Configuration Complexity

// ❌ INCOMPLETE: Missing required fields
const model = new AzureChatOpenAI({
  azureOpenAIApiKey: process.env.AZURE_OPENAI_API_KEY,
});

// ✅ COMPLETE: All required fields
const model = new AzureChatOpenAI({
  azureOpenAIApiKey: process.env.AZURE_OPENAI_API_KEY,
  azureOpenAIApiInstanceName: "my-instance",
  azureOpenAIApiDeploymentName: "gpt-4-deployment",
  azureOpenAIApiVersion: "2024-02-01",
});

Fix: Azure requires instance name, deployment name, and API version.

3. Model Name Variations

// Different providers have different model naming conventions
// OpenAI
const openai = new ChatOpenAI({ modelName: "gpt-4" });

// Bedrock (includes provider prefix)
const bedrock = new ChatBedrockConverse({ 
  model: "anthropic.claude-3-sonnet-20240229-v1:0" 
});

// Anthropic (direct model name)
const anthropic = new ChatAnthropic({ 
  modelName: "claude-3-opus-20240229" 
});

Fix: Check provider documentation for correct model identifiers.

4. Tool Calling Support

// ❌ Not all models support tool calling
const model = new ChatOpenAI({ modelName: "gpt-3.5-turbo-instruct" });
// This older model doesn't support tools!

// ✅ Use models with tool support
const model = new ChatOpenAI({ modelName: "gpt-4" });
const withTools = model.bindTools([myTool]);

Fix: Verify model supports function/tool calling before binding tools. GPT-4, GPT-3.5-turbo, Claude 3, and Gemini Pro all support tools.

5. Rate Limits and Quotas

// ❌ No retry logic
const model = new ChatOpenAI();
const response = await model.invoke("Hello"); // May fail on rate limit

// ✅ Configure retries
const model = new ChatOpenAI({
  maxRetries: 3,
  timeout: 30000, // 30 seconds
});

Fix: Configure retry logic and handle rate limit errors gracefully.

6. Context Window Limits

// ❌ Exceeding context limits
const model = new ChatOpenAI({ modelName: "gpt-3.5-turbo" }); // 4k context
const longText = "...".repeat(10000);
await model.invoke(longText); // Will fail!

// ✅ Use appropriate models for long context
const model = new ChatOpenAI({ modelName: "gpt-4-turbo" }); // 128k context
// OR
const model = new ChatAnthropic({ 
  modelName: "claude-3-opus-20240229" // 200k context
});

Fix: Choose models with appropriate context windows for your use case.

7. Streaming vs Non-Streaming

// ❌ Mixing streaming and non-streaming incorrectly
const response = await model.stream("Hello");
console.log(response.content); // Won't work! response is an async iterable

// ✅ Handle streaming properly
const stream = await model.stream("Hello");
for await (const chunk of stream) {
  console.log(chunk.content);
}

// OR use invoke for non-streaming
const response = await model.invoke("Hello");
console.log(response.content);

Fix: Use invoke() for complete responses, stream() for token-by-token.

Links and Resources

Official Documentation

• LangChain JS Chat Models Overview
• OpenAI Integration
• Anthropic Integration
• Azure OpenAI Integration
• AWS Bedrock Integration
• Google GenAI Integration

Provider Documentation

• OpenAI API Docs
• Anthropic API Docs
• Azure OpenAI Service
• AWS Bedrock
• Google AI Studio

Package Installation

# OpenAI
npm install @langchain/openai

# Anthropic
npm install @langchain/anthropic

# AWS (Bedrock)
npm install @langchain/aws

# Google
npm install @langchain/google-genai
npm install @langchain/google-vertexai