Effect AI

Effect AI

Overview

Effect AI (@effect/ai) provides type-safe integration with AI/LLM services:

• Provider abstraction - Unified API for OpenAI, Anthropic, etc.
• Tool use - Type-safe function calling with Schema
• Execution planning - Multi-step AI workflows
• Structured output - Schema-validated responses

Installation

npm install @effect/ai @effect/ai-openai
# or
npm install @effect/ai @effect/ai-anthropic

Basic Usage

Creating a Provider

import { AiChat } from "@effect/ai";
import { OpenAiChat } from "@effect/ai-openai";
import { Effect, Layer } from "effect";

const OpenAiLive = OpenAiChat.layer({
  apiKey: Config.redacted("OPENAI_API_KEY"),
  model: "gpt-4",
});

import { AnthropicChat } from "@effect/ai-anthropic";

const AnthropicLive = AnthropicChat.layer({
  apiKey: Config.redacted("ANTHROPIC_API_KEY"),
  model: "claude-3-opus-20240229",
});

Simple Completion

const program = Effect.gen(function* () {
  const ai = yield* AiChat.AiChat;

  const response = yield* ai.generateText({
    prompt: "Explain functional programming in one sentence.",
  });

  return response.text;
});

const result = yield * program.pipe(Effect.provide(OpenAiLive));

Chat with Messages

const chat = Effect.gen(function* () {
  const ai = yield* AiChat.AiChat;

  const response = yield* ai.generateText({
    messages: [
      { role: "system", content: "You are a helpful assistant." },
      { role: "user", content: "What is Effect-TS?" },
    ],
  });

  return response.text;
});

Tool Use

Define tools that AI can call:

Defining Tools with Schema

import { AiTool } from "@effect/ai";
import { Schema } from "effect";

const WeatherInput = Schema.Struct({
  city: Schema.String,
  unit: Schema.optional(Schema.Literal("celsius", "fahrenheit")),
});

const getWeather = AiTool.make({
  name: "get_weather",
  description: "Get current weather for a city",
  input: WeatherInput,
  handler: (input) =>
    Effect.succeed({
      city: input.city,
      temperature: 22,
      unit: input.unit ?? "celsius",
      conditions: "sunny",
    }),
});

Using Tools in Chat

const programWithTools = Effect.gen(function* () {
  const ai = yield* AiChat.AiChat;

  const response = yield* ai.generateText({
    prompt: "What's the weather in Tokyo?",
    tools: [getWeather],
  });

  return response.text;
});

Multiple Tools

const searchTool = AiTool.make({
  name: "search",
  description: "Search the web",
  input: Schema.Struct({ query: Schema.String }),
  handler: ({ query }) => performSearch(query),
});

const calculatorTool = AiTool.make({
  name: "calculator",
  description: "Perform calculations",
  input: Schema.Struct({
    expression: Schema.String,
  }),
  handler: ({ expression }) => evaluate(expression),
});

const response =
  yield *
  ai.generateText({
    prompt: "Search for Effect-TS and calculate 2+2",
    tools: [searchTool, calculatorTool],
  });

Structured Output

Get typed, validated responses:

const ProductReview = Schema.Struct({
  sentiment: Schema.Literal("positive", "negative", "neutral"),
  score: Schema.Number.pipe(Schema.between(1, 5)),
  summary: Schema.String,
  keywords: Schema.Array(Schema.String),
});

const analyzeReview = Effect.gen(function* () {
  const ai = yield* AiChat.AiChat;

  const review = yield* ai.generateObject({
    prompt: "Analyze this product review: 'Great product, highly recommend!'",
    schema: ProductReview,
  });

  return review;
});

Execution Planning

For complex multi-step AI workflows:

import { AiPlan } from "@effect/ai";

const researchPlan = AiPlan.make({
  name: "research",
  description: "Research a topic and summarize findings",
  steps: [
    {
      name: "search",
      description: "Search for relevant information",
      tool: searchTool,
    },
    {
      name: "analyze",
      description: "Analyze search results",
      handler: (context) =>
        Effect.gen(function* () {
          const ai = yield* AiChat.AiChat;
          return yield* ai.generateText({
            prompt: `Analyze these results: ${context.previousResults}`,
          });
        }),
    },
    {
      name: "summarize",
      description: "Create final summary",
      handler: (context) =>
        Effect.gen(function* () {
          const ai = yield* AiChat.AiChat;
          return yield* ai.generateObject({
            prompt: `Summarize: ${context.analysis}`,
            schema: ResearchSummary,
          });
        }),
    },
  ],
});

const result =
  yield *
  AiPlan.execute(researchPlan, {
    topic: "Effect-TS benefits",
  });

Streaming Responses

import { Stream } from "effect";

const streamProgram = Effect.gen(function* () {
  const ai = yield* AiChat.AiChat;

  const stream = yield* ai.streamText({
    prompt: "Write a short story about a robot.",
  });

  yield* Stream.runForEach(stream, (chunk) => Effect.sync(() => process.stdout.write(chunk)));
});

Provider Configuration

OpenAI Options

const OpenAiLive = OpenAiChat.layer({
  apiKey: Config.redacted("OPENAI_API_KEY"),
  model: "gpt-4-turbo",
  temperature: 0.7,
  maxTokens: 1000,
  organizationId: Config.string("OPENAI_ORG_ID").pipe(Config.option),
});

Anthropic Options

const AnthropicLive = AnthropicChat.layer({
  apiKey: Config.redacted("ANTHROPIC_API_KEY"),
  model: "claude-3-opus-20240229",
  maxTokens: 4096,
});

Error Handling

import { AiError } from "@effect/ai";

const safeChat = program.pipe(
  Effect.catchTag("AiRateLimitError", (error) =>
    Effect.gen(function* () {
      yield* Effect.sleep(error.retryAfter);
      return yield* program;
    }),
  ),
  Effect.catchTag("AiAuthenticationError", () => Effect.fail(new ConfigurationError())),
  Effect.catchTag("AiError", (error) =>
    Effect.gen(function* () {
      yield* Effect.logError("AI error", error);
      return "Sorry, I couldn't process that request.";
    }),
  ),
);

Testing

const MockAiLive = Layer.succeed(AiChat.AiChat, {
  generateText: () => Effect.succeed({ text: "Mock response" }),
  generateObject: (options) => Effect.succeed(mockData),
  streamText: () => Effect.succeed(Stream.make("Mock", " ", "stream")),
});

const testProgram = program.pipe(Effect.provide(MockAiLive));

Best Practices

1. Use Schema for tools - Type-safe tool definitions
2. Handle rate limits - Implement retry with backoff
3. Validate responses - Use generateObject with Schema
4. Stream long responses - Better UX for long generations
5. Mock in tests - Don't call real APIs in tests

Additional Resources

For comprehensive Effect AI documentation, consult ${CLAUDE_PLUGIN_ROOT}/references/llms-full.txt.

Search for these sections:

• "Introduction to Effect AI" for overview
• "Tool Use" for function calling
• "Execution Planning" for multi-step workflows