Effect AI Provider

Effect AI Provider

Configure AI provider layers for language model integration using Effect's AI ecosystem.

When to Use This Skill

Use this skill when:

• Integrating AI language models (Anthropic, OpenAI, OpenRouter, etc.) into Effect applications
• Setting up multi-provider AI architectures with ExecutionPlan fallback
• Implementing stateful chat conversations with context history
• Managing AI provider configuration and API keys securely
• Composing AI capabilities with other Effect services

Import Patterns

CRITICAL: Always use namespace imports. Use { } destructured imports for the effect package barrel exports.

// From the "effect" barrel — destructured
import {
	Config,
	Effect,
	ExecutionPlan,
	Layer,
	Ref,
	Schema,
	Context,
	Stream
} from 'effect';

// From "effect/unstable/ai" — namespace imports
import {
	AiError,
	Chat,
	LanguageModel,
	Model,
	Prompt,
	Tool,
	Toolkit
} from 'effect/unstable/ai';
// Or individually:
import * as LanguageModel from 'effect/unstable/ai/LanguageModel';
import * as Chat from 'effect/unstable/ai/Chat';
import * as Model from 'effect/unstable/ai/Model';
import * as Prompt from 'effect/unstable/ai/Prompt';
import * as AiError from 'effect/unstable/ai/AiError';

// Anthropic
import { AnthropicClient, AnthropicLanguageModel } from '@effect/ai-anthropic';

// OpenAI
import {
	OpenAiClient,
	OpenAiClientGenerated,
	OpenAiLanguageModel,
	OpenAiSchema,
	OpenAiTool
} from '@effect/ai-openai';

// OpenRouter
import {
	OpenRouterClient,
	OpenRouterLanguageModel
} from '@effect/ai-openrouter';

// HTTP client (required by all providers)
import { FetchHttpClient } from 'effect/unstable/http';

Provider Layer Pattern

Every provider exposes two constructors:

• model(modelId, config?) — returns a Model.Model (preferred for ExecutionPlan and Effect.provide)
• layer({ model, config? }) — returns a raw Layer<LanguageModel.LanguageModel, never, Client>

-- Model constructor (preferred)
ProviderLanguageModel.model :: (modelId, config?) → Model.Model<providerName, LanguageModel, Client>

-- Layer constructor
ProviderLanguageModel.layer :: { model, config? } → Layer LanguageModel Client

-- Client layer
ProviderClient.layerConfig :: { apiKey } → Layer Client HttpClient

Anthropic Provider

import { AnthropicClient, AnthropicLanguageModel } from '@effect/ai-anthropic';
import { Config, Layer } from 'effect';
import { FetchHttpClient } from 'effect/unstable/http';

// Client layer (reusable across models)
const AnthropicClientLayer = AnthropicClient.layerConfig({
	apiKey: Config.redacted('ANTHROPIC_API_KEY')
}).pipe(Layer.provide(FetchHttpClient.layer));

// Option A: model() — returns Model.Model (preferred)
const claudeModel = AnthropicLanguageModel.model('claude-opus-4-6');
// Use with: Effect.provide(claudeModel) or in ExecutionPlan

// Option B: layer() — returns raw Layer<LanguageModel>
const AnthropicLive = AnthropicLanguageModel.layer({
	model: 'claude-sonnet-4-20250514'
}).pipe(Layer.provide(AnthropicClientLayer));

Anthropic capability detection preserves the lower output limits and structured-output support of known legacy Claude models. Unknown and newly released model identifiers default to modern capabilities: native structured outputs and 128_000 output tokens. Override capability detection with structuredOutputs: false (or true) when a model or compatible endpoint differs from that default; set max_tokens separately when the provider's output limit differs.

const compatibleClaude = AnthropicLanguageModel.model('future-claude-model', {
	structuredOutputs: false,
	max_tokens: 8192
});

OpenAI Provider

import { OpenAiClient, OpenAiLanguageModel } from '@effect/ai-openai';
import { Config, Layer } from 'effect';
import { FetchHttpClient } from 'effect/unstable/http';

const OpenAiClientLayer = OpenAiClient.layerConfig({
	apiKey: Config.redacted('OPENAI_API_KEY')
}).pipe(Layer.provide(FetchHttpClient.layer));

// model() constructor (preferred)
const gptModel = OpenAiLanguageModel.model('gpt-5.2');

// layer() constructor
const OpenAiLive = OpenAiLanguageModel.layer({
	model: 'gpt-4.1'
}).pipe(Layer.provide(OpenAiClientLayer));

Public OpenAI modules:

• OpenAiSchema — typed Responses API request/response and SSE event schemas
• OpenAiClient — handwritten service with createResponse, createResponseStream, and createEmbedding
• OpenAiClientGenerated — generated direct endpoint access when you need raw OpenAI API coverage
• OpenAiTool — OpenAI provider-defined tools for native capabilities

OpenAiSchema.ResponseStreamEvent accepts both flat OpenAI error events and compatible-provider events with details nested under error, normalizing both to the same decoded error event shape.

const client = yield* OpenAiClient.OpenAiClient;
const [body] = yield* client.createResponse({
	model: 'gpt-4.1',
	input: 'Say hello'
});

OpenAI Provider-Defined Tools

Use OpenAiTool for OpenAI-native tools instead of hand-rolling provider-defined descriptors:

const NativeTools = Toolkit.make(
	OpenAiTool.WebSearch({}),
	OpenAiTool.FileSearch({ vector_store_ids: ['vs_123'] }),
	OpenAiTool.Mcp({
		server_label: 'docs',
		server_url: 'https://mcp.example.com/mcp'
	})
);

Available hosted/provider tools include WebSearch, CodeInterpreter, FileSearch, ImageGeneration, and Mcp (customName: "OpenAiMcp"). MCP tool approval requests/results use this canonical OpenAiMcp name and the normal Effect AI approval request/response parts. Handler-required local tools such as Shell, LocalShell, and ApplyPatch run in your environment; provide handlers only behind explicit sandboxing, authorization, and audit policy.

OpenAI-Compatible Providers

Use apiUrl with @effect/ai-openai for OpenAI-compatible APIs (Azure OpenAI, local models, etc.):

import { OpenAiClient, OpenAiConfig } from '@effect/ai-openai';
import { Config, Layer } from 'effect';
import { FetchHttpClient, HttpClient, HttpClientRequest } from 'effect/unstable/http';

const CompatibleClientLayer = OpenAiClient.layerConfig({
	apiKey: Config.redacted('OPENAI_COMPAT_API_KEY'),
	apiUrl: Config.succeed('https://my-provider.example.com/v1')
}).pipe(Layer.provide(FetchHttpClient.layer));

// Keep withClientTransform for middleware/proxy/tracing/header transforms.
const withAuditHeader = OpenAiConfig.withClientTransform(
	HttpClient.mapRequest(HttpClientRequest.setHeader('x-audit-source', 'writer'))
);

const program = myEffect.pipe(withAuditHeader);

OpenRouter Provider

Multi-provider access through unified interface:

import {
	OpenRouterClient,
	OpenRouterLanguageModel
} from '@effect/ai-openrouter';
import { Config, Layer } from 'effect';
import { FetchHttpClient } from 'effect/unstable/http';

const OpenRouterClientLayer = OpenRouterClient.layerConfig({
	apiKey: Config.redacted('OPENROUTER_API_KEY')
}).pipe(Layer.provide(FetchHttpClient.layer));

// model() constructor — use provider-prefixed model IDs
const routerModel = OpenRouterLanguageModel.model('anthropic/claude-sonnet-4');

ExecutionPlan (Multi-Provider Fallback)

ExecutionPlan defines a strategy for trying multiple providers with different configurations and retry counts:

import { AnthropicClient, AnthropicLanguageModel } from '@effect/ai-anthropic';
import { OpenAiClient, OpenAiLanguageModel } from '@effect/ai-openai';
import { Effect, ExecutionPlan, Layer } from 'effect';
import { LanguageModel } from 'effect/unstable/ai';

// Try cheaper model first, fall back to more expensive one
const DraftPlan = ExecutionPlan.make(
	{
		provide: OpenAiLanguageModel.model('gpt-5.2'),
		attempts: 3 // retry up to 3 times before falling back
	},
	{
		provide: AnthropicLanguageModel.model('claude-opus-4-6'),
		attempts: 2
	}
);

// Inside a Layer.effect, call captureRequirements to capture current services
const draftsModel = yield* DraftPlan.captureRequirements;
// This satisfies the plan's client requirements from the current context

// Apply the plan to an effect
const result = yield* myEffect.pipe(Effect.withExecutionPlan(draftsModel));

Observing ExecutionPlan Lifecycle

Effect.withExecutionPlan and Stream.withExecutionPlan accept an optional onEvent observer. Events are the ExecutionPlan.Event tagged union: AttemptStart, AttemptSuccess, and AttemptFailure.

const result = yield* myEffect.pipe(
	Effect.withExecutionPlan(draftsModel, {
		onEvent: (event) =>
			Effect.logInfo('AI provider attempt').pipe(
				Effect.annotateLogs({
					event: event._tag,
					attempt: event.attempt,
					stepAttempt: event.stepAttempt,
					stepIndex: event.stepIndex
				})
		})
);

• attempt is cumulative and 1-based across the plan; stepAttempt is 1-based within a step; stepIndex is 0-based.
• Success and failure events include attempt duration; failure includes the full Cause, including defects and interruption.
• Every start is paired with one terminal event, including interruption. Observers are awaited in order, should stay cheap, and their defects are ignored so telemetry cannot change the attempt outcome.
• For Stream.withExecutionPlan, an attempt truncated by downstream cancellation is reported as AttemptSuccess; use preventFallbackOnPartialStream when mixing partial output with fallback output is unacceptable.

Chat Service (Stateful Conversations)

Maintain conversation history with automatic context management:

import { Effect, Ref } from 'effect';
import { Chat, Prompt } from 'effect/unstable/ai';

// Create with system prompt
const session =
	yield*
	Chat.fromPrompt(
		Prompt.empty.pipe(Prompt.setSystem('You are a helpful assistant.'))
	);

// Or create empty
const emptySession = yield* Chat.empty;

// Or from raw messages
const agentSession =
	yield*
	Chat.fromPrompt([
		{ role: 'system', content: 'You are an assistant.' },
		{ role: 'user', content: 'Hello' }
	]);

// Generate text (history is maintained automatically)
const response =
	yield*
	session
		.generateText({
			prompt: 'What is Effect?'
		})
		.pipe(Effect.provide(modelLayer));

// Access conversation history
const history = yield* Ref.get(session.history);

// Export for persistence
const json = yield* session.exportJson;

// Restore from persisted state
const restored = yield* Chat.fromJson(json);

Config Override Pattern

Runtime configuration adjustment on a per-effect basis using withConfigOverride (dual API):

import { AnthropicLanguageModel } from '@effect/ai-anthropic';

// Apply overrides to any effect that uses the LanguageModel
const result =
	yield*
	model.generateText({ prompt: '...' }).pipe(
		AnthropicLanguageModel.withConfigOverride({
			temperature: 0.7,
			max_tokens: 4096
		})
	);

// Also available for OpenAI:
import { OpenAiLanguageModel } from '@effect/ai-openai';

const result2 =
	yield*
	model.generateText({ prompt: '...' }).pipe(
		OpenAiLanguageModel.withConfigOverride({
			temperature: 0.9,
			reasoning: { effort: 'medium', summary: 'auto' },
			text: { verbosity: 'low' },
			strictJsonSchema: true,
			fileIdPrefixes: ['file-']
		})
	);

OpenAiLanguageModel.Config accepts Responses API request fields plus fileIdPrefixes, text.verbosity, restored reasoning config, and strictJsonSchema. Do not manually send library-only fields (fileIdPrefixes, strictJsonSchema) to OpenAI APIs; the language model strips them before request construction.

Reasoning effort also accepts 'max' for OpenAI-compatible providers that expose that level, in addition to the standard OpenAI effort values.

OpenAI error classification distinguishes temporary rate limits from exhausted account quota. HTTP 402 responses, and HTTP 429 responses whose code or type is insufficient_quota or billing_insufficient_balance, become AiError values with reason QuotaExhaustedError; error.isRetryable is false, so do not retry them without explicit user action. Ordinary 429 responses remain retryable RateLimitError values and preserve retry metadata when available.

Model.make — Model Abstraction

Wrap provider layers with metadata. Takes 3 positional arguments: (providerName, modelId, layer):

import { Model } from 'effect/unstable/ai';

// This is what ProviderLanguageModel.model() calls internally:
const Claude = Model.make(
	'anthropic', // provider name
	'claude-sonnet-4-20250514', // model identifier
	AnthropicLanguageModel.layer({ model: 'claude-sonnet-4-20250514' })
);

In practice, use the provider's .model() shorthand instead of calling Model.make directly:

// Preferred — equivalent to Model.make("anthropic", "claude-opus-4-6", layer)
const claudeModel = AnthropicLanguageModel.model('claude-opus-4-6');

Context.Service Pattern

Define services using the shape-as-type-parameter pattern:

import { Effect, Context, Stream } from 'effect';

export class AiWriter extends Context.Service<
	AiWriter,
	{
		draftAnnouncement(
			product: string
		): Effect.Effect<string, AiWriterError>;
		streamHighlights(version: string): Stream.Stream<string, AiWriterError>;
	}
>()('myapp/AiWriter') {
	static readonly layer = Layer.effect(
		AiWriter,
		Effect.gen(function* () {
			const model = AnthropicLanguageModel.model('claude-opus-4-6');
			const modelLayer = yield* model.captureRequirements;

			const draftAnnouncement = Effect.fn('AiWriter.draftAnnouncement')(
				function* (product: string) {
					const lm = yield* LanguageModel.LanguageModel;
					const response = yield* lm.generateText({
						prompt: `Write a launch announcement for ${product}`
					});
					return response.text;
				},
				Effect.provide(modelLayer),
				Effect.mapError((e) => AiWriterError.fromAiError(e))
			);

			return AiWriter.of({ draftAnnouncement, streamHighlights });
		})
	).pipe(Layer.provide(AnthropicClientLayer));
}

Custom Error Wrapping

In rc.112, AiError.AuthenticationError accepts an optional description and appends it after the kind-based remediation message. Anthropic, OpenAI, OpenAI-compatible, and OpenRouter adapters propagate provider error text from 401/403 responses. Preserve this reason rather than replacing it with a generic authentication string. AiError.buildErrorDescription is available to adapter authors; inspect its signature before building custom provider mappings. Authentication failures generally require corrected credentials/permissions, not a blanket transient retry. Keep redacted credentials out of logs.

OpenAI Responses additionally supports GPT-5.6+ explicit prompt cache breakpoints through Prompt metadata and prompt_cache_options model config; see effect-ai-prompt for the complete construction example.

Wrap AiError into domain-specific tagged errors:

import { Schema } from 'effect';
import { AiError } from 'effect/unstable/ai';

export class MyAiError extends Schema.TaggedError<MyAiError>()(
	'MyAiError',
	{
		reason: AiError.AiErrorReason
	}
) {
	static fromAiError(error: AiError.AiError) {
		return new MyAiError({ reason: error.reason });
	}
}

// Usage: Effect.mapError((e) => MyAiError.fromAiError(e))

Available Providers

Note: There are no @effect/ai-google or @effect/ai-amazon-bedrock packages. Use OpenRouter to access Google/Bedrock models.

Complete Working Example

Full application with ExecutionPlan, Chat, and streaming:

import { AnthropicClient, AnthropicLanguageModel } from '@effect/ai-anthropic';
import { OpenAiClient, OpenAiLanguageModel } from '@effect/ai-openai';
import {
	Config,
	Effect,
	ExecutionPlan,
	Layer,
	Ref,
	Schema,
	Context,
	Stream
} from 'effect';
import {
	AiError,
	Chat,
	LanguageModel,
	Model,
	Prompt,
	type Response
} from 'effect/unstable/ai';
import { FetchHttpClient } from 'effect/unstable/http';

// ---------------------------------------------------------------------------
// Provider client layers
// ---------------------------------------------------------------------------

const AnthropicClientLayer = AnthropicClient.layerConfig({
	apiKey: Config.redacted('ANTHROPIC_API_KEY')
}).pipe(Layer.provide(FetchHttpClient.layer));

const OpenAiClientLayer = OpenAiClient.layerConfig({
	apiKey: Config.redacted('OPENAI_API_KEY')
}).pipe(Layer.provide(FetchHttpClient.layer));

// ---------------------------------------------------------------------------
// ExecutionPlan — try cheap model first, fall back to expensive
// ---------------------------------------------------------------------------

const DraftPlan = ExecutionPlan.make(
	{ provide: OpenAiLanguageModel.model('gpt-5.2'), attempts: 3 },
	{ provide: AnthropicLanguageModel.model('claude-opus-4-6'), attempts: 2 }
);

// ---------------------------------------------------------------------------
// Custom error type
// ---------------------------------------------------------------------------

export class WriterError extends Schema.TaggedError<WriterError>()(
	'WriterError',
	{
		reason: AiError.AiErrorReason
	}
) {
	static fromAiError(error: AiError.AiError) {
		return new WriterError({ reason: error.reason });
	}
}

// ---------------------------------------------------------------------------
// Service definition
// ---------------------------------------------------------------------------

export class AiWriter extends Context.Service<
	AiWriter,
	{
		draft(
			product: string
		): Effect.Effect<{ provider: string; text: string }, WriterError>;
		chat(message: string): Effect.Effect<string, WriterError>;
		streamHighlights(version: string): Stream.Stream<string, WriterError>;
	}
>()('app/AiWriter') {
	static readonly layer = Layer.effect(
		AiWriter,
		Effect.gen(function* () {
			const draftsModel = yield* DraftPlan.captureRequirements;
			const chatModel = OpenAiLanguageModel.model('gpt-4.1');
			const chatModelLayer = yield* chatModel.captureRequirements;

			// --- Chat session with history ---
			const session = yield* Chat.fromPrompt(
				Prompt.empty.pipe(
					Prompt.setSystem('You are a helpful writing assistant.')
				)
			);

			const draft = Effect.fn('AiWriter.draft')(
				function* (product: string) {
					const provider = yield* Model.ProviderName;
					const lm = yield* LanguageModel.LanguageModel;
					const response = yield* lm.generateText({
						prompt: `Write a launch announcement for ${product}.`
					});
					return { provider, text: response.text };
				},
				Effect.withExecutionPlan(draftsModel),
				Effect.mapError((e) => WriterError.fromAiError(e))
			);

			const chat = Effect.fn('AiWriter.chat')(
				function* (message: string) {
					const response = yield* session
						.generateText({ prompt: message })
						.pipe(Effect.provide(chatModelLayer));
					const history = yield* Ref.get(session.history);
					yield* Effect.logInfo(
						`History: ${history.content.length} messages`
					);
					return response.text;
				},
				Effect.mapError((e) => WriterError.fromAiError(e))
			);

			const streamHighlights = (version: string) =>
				LanguageModel.streamText({
					prompt: `Release highlights for v${version} as bullets.`
				}).pipe(
					Stream.filter(
						(part): part is Response.TextDeltaPart =>
							part.type === 'text-delta'
					),
					Stream.map((part) => part.delta),
					Stream.provide(chatModelLayer),
					Stream.mapError((e) => WriterError.fromAiError(e))
				);

			return AiWriter.of({ draft, chat, streamHighlights });
		})
	).pipe(Layer.provide([OpenAiClientLayer, AnthropicClientLayer]));
}

// ---------------------------------------------------------------------------
// Usage
// ---------------------------------------------------------------------------

const program = Effect.gen(function* () {
	const writer = yield* AiWriter;
	const result = yield* writer.draft('Effect Cloud');
	yield* Effect.logInfo(`Provider: ${result.provider}, Text: ${result.text}`);
});

Effect.runPromise(program.pipe(Effect.provide(AiWriter.layer)));

Anti-Patterns

// WRONG: Hardcoded API keys
AnthropicClient.layerConfig({ apiKey: 'sk-...' });

// RIGHT: Config.redacted for secrets
AnthropicClient.layerConfig({ apiKey: Config.redacted('ANTHROPIC_API_KEY') });

// WRONG: Missing FetchHttpClient layer
AnthropicClient.layerConfig({ apiKey: Config.redacted('KEY') });
// Will fail at runtime — providers require an HttpClient

// RIGHT: Always provide an HTTP client layer
AnthropicClient.layerConfig({ apiKey: Config.redacted('KEY') }).pipe(
	Layer.provide(FetchHttpClient.layer)
);

// WRONG: Old Chat.make API
const chat = yield* Chat.make({ system: 'You are helpful' });

// RIGHT: Chat.fromPrompt with Prompt composition
const chat =
	yield*
	Chat.fromPrompt(Prompt.empty.pipe(Prompt.setSystem('You are helpful')));

// WRONG: Old Model.make API with object arg
Model.make({ name: 'claude', layer: AnthropicLive });

// RIGHT: Model.make with 3 positional args (or use .model() shorthand)
Model.make('anthropic', 'claude-opus-4-6', anthropicLayer);
// Better: AnthropicLanguageModel.model("claude-opus-4-6")

// WRONG: Importing non-existent providers
import { GoogleClient } from '@effect/ai-google'; // Does NOT exist
import { BedrockClient } from '@effect/ai-amazon-bedrock'; // Does NOT exist

Quality Checklist

• Use Config.redacted for API keys (never hardcode)
• Provide FetchHttpClient.layer to all client layers
• Use .model() constructor for ExecutionPlan and Effect.provide
• Use ExecutionPlan for multi-provider fallback with retry
• Use withConfigOverride for per-effect config adjustments
• Use apiUrl for OpenAI-compatible base URLs; reserve client transforms for middleware/proxy/tracing/headers
• Use OpenAiTool for OpenAI provider-defined tools
• Use Chat.fromPrompt / Chat.empty / Chat.fromJson (not Chat.make)
• Wrap AiError into a domain-specific Schema.TaggedError
• Use Context.Service with shape type parameter for service definitions

Related Skills

• effect-ai-language-model - Using LanguageModel service for text/object/stream generation
• effect-ai-prompt - Building prompts with Prompt composition operators
• effect-ai-tool - Defining tools and toolkits for agentic loops
• effect-ai-streaming - Streaming response patterns and accumulation
• effect-layer-design - General Effect layer composition patterns

References

• packages/ai/anthropic/src/AnthropicLanguageModel.ts
• packages/ai/openai/src/OpenAiLanguageModel.ts
• packages/ai/openrouter/src/OpenRouterLanguageModel.ts
• ai-docs/src/71_ai/10_language-model.ts
• ai-docs/src/71_ai/30_chat.ts