Golem Add LLM Effect

Adding LLM Capabilities to an Effect Golem Agent

Use the provider-independent LanguageModel service from effect/unstable/ai, an exactly version-aligned Effect AI provider package, and FetchHttpClient.layer from effect/unstable/http. For OpenAI or an OpenAI-compatible endpoint, use @effect/ai-openai.

┌────────────────────┐    provides     ┌───────────────────────────┐
│ Application method │◀───────────────│ LanguageModel service     │
└────────────────────┘                 └─────────────┬─────────────┘
                                                    │ OpenAiLanguageModel.layer
                                      ┌─────────────▼─────────────┐
                                      │ OpenAiClient              │
                                      └─────────────┬─────────────┘
                                                    │ FetchHttpClient.layer
                                      ┌─────────────▼─────────────┐
                                      │ WASI-backed global fetch  │
                                      └───────────────────────────┘

The currently validated contract is non-streaming generateText. Do not add structured generation, streaming, embeddings, or tool calling unless the task explicitly requires it and the installed pinned versions are verified for that capability.

Install the Matching Provider Version

Effect packages use matching versions. Inspect the exact installed effect version, then install the same @effect/ai-openai version. For the current template:

npm install --save-exact @effect/ai-openai@4.0.0-beta.98

Do not add the provider to @golemcloud/effect-golem or the common project template. Install it only in applications that select OpenAI. A version mismatch can create incompatible Effect service identities or types.

Imports

import { OpenAiClient, OpenAiLanguageModel } from "@effect/ai-openai";
import { Effect, Layer, Schema } from "effect";
import { LanguageModel } from "effect/unstable/ai";
import { FetchHttpClient } from "effect/unstable/http";
import { defineAgent, defineConfig, method } from "@golemcloud/effect-golem";

Application methods depend on LanguageModel.LanguageModel, not provider-specific response types. Provider modules appear only in layer construction.

Store Provider Configuration and Secrets

Declare the API key as a typed redacted secret. Keep the provider URL and model configurable so development and tests can target an OpenAI-compatible service:

export class LlmConfig extends defineConfig("Llm.Config", {
  apiKey: Schema.Redacted(Schema.String),
  apiUrl: Schema.String,
  model: Schema.String,
}) {}

Attach config: LlmConfig to the agent. Supply apiKey through Golem secrets and apiUrl/model through ordinary config in golem.yaml. The OpenAI client's option is named apiUrl, and an OpenAI-compatible base normally includes its /v1 prefix.

Never log, return, snapshot, or interpolate the secret. cfg.apiKey.get already returns the Redacted<string> required by OpenAiClient.layer; do not unwrap it in application code.

Compose the Layers

The provider client requires the canonical HttpClient service, and the language-model layer requires the provider client:

const OpenAiClientLive = OpenAiClient.layer({
  apiKey,
  apiUrl,
}).pipe(Layer.provide(FetchHttpClient.layer));

const LanguageModelLive = OpenAiLanguageModel.layer({ model }).pipe(
  Layer.provide(OpenAiClientLive),
);

This composition provides the same LanguageModel.LanguageModel tag imported by application code without bundling another Effect runtime.

Generate Text

LanguageModel.generateText accepts a plain string prompt and returns a normalized, provider-independent response. Read response.text:

const answer = (question: string) =>
  LanguageModel.generateText({
    prompt: question,
    toolChoice: "none",
  }).pipe(
    Effect.map((response) => response.text),
    Effect.provide(LanguageModelLive),
  );

Provider status, transport, and response-decoding failures remain typed AiError failures. Map them to a method's declared error schema when callers should handle them. Use Effect.orDie only when the method intentionally exposes no recoverable provider failure.

Complete Effect-Golem Agent

This example resolves typed Golem config when constructing the agent instance, builds the provider layers from the resolved values, and keeps the handler provider-independent:

import { OpenAiClient, OpenAiLanguageModel } from "@effect/ai-openai";
import { Effect, Layer, Schema } from "effect";
import { LanguageModel } from "effect/unstable/ai";
import { FetchHttpClient } from "effect/unstable/http";
import { defineAgent, defineConfig, method } from "@golemcloud/effect-golem";

export class LlmConfig extends defineConfig("Llm.Config", {
  apiKey: Schema.Redacted(Schema.String),
  apiUrl: Schema.String,
  model: Schema.String,
}) {}

export const LlmAgent = defineAgent({
  name: "LlmAgent",
  mode: "durable",
  id: { name: Schema.String },
  config: LlmConfig,
  methods: {
    ask: method({
      input: { question: Schema.String },
      success: Schema.String,
    }),
  },
}).implement({
  init: () =>
    Effect.gen(function* () {
    const cfg = yield* LlmConfig;
    const apiKey = yield* cfg.apiKey.get;
    const apiUrl = yield* cfg.apiUrl;
    const model = yield* cfg.model;

    const OpenAiClientLive = OpenAiClient.layer({ apiKey, apiUrl }).pipe(
      Layer.provide(FetchHttpClient.layer),
    );
    const LanguageModelLive = OpenAiLanguageModel.layer({ model }).pipe(
      Layer.provide(OpenAiClientLive),
    );

    return LanguageModelLive;
  }),
  methods: (LanguageModelLive) => ({
    ask: ({ question }) =>
      LanguageModel.generateText({
        prompt: question,
        toolChoice: "none",
      }).pipe(
        Effect.map((response) => response.text),
        Effect.provide(LanguageModelLive),
        Effect.orDie,
      ),
  }),
});

Import the implementation module from src/main.ts using its emitted .js suffix so the top-level registration runs.

Durability and Provider Side Effects

FetchHttpClient.layer uses the same WASI-backed outgoing HTTP path as other Effect HTTP requests. Golem records the call for durable replay of the same invocation. Replay is not cross-invocation deduplication: invoking ask twice can send two provider requests and incur two charges.

Do not log request headers or provider configuration. Public oplog conversion redacts recognized credential-bearing HTTP headers, but application code must still keep secrets out of logs, return values, error strings, snapshots, and diagnostics.

When testing against a mock provider, verify the request URL, bearer authorization, model, prompt, and returned text. Also verify that replay does not unexpectedly resend a recorded request.

Do Not Substitute Other Capabilities

• Do not call the provider with Effect.tryPromise(globalThis.fetch); use LanguageModel and the provider layer.
• Do not return canned or stubbed text while claiming a provider request occurred.
• Do not use typed Golem RPC as a substitute for a requested external provider call.
• Do not install the Node openai package or another transport that bypasses Effect's canonical HttpClient service.
• Do not invent Llm.chat, GolemLlm, or EffectGolemHttpClient helpers.
• Do not import @golemcloud/golem-ts-sdk into an Effect component.

Key Constraints

• Pin @effect/ai-openai exactly to the installed effect version.
• Import LanguageModel from effect/unstable/ai.
• Provide FetchHttpClient.layer to OpenAiClient.layer.
• Provide the client layer to OpenAiLanguageModel.layer.
• Keep application logic provider-independent and return response.text.
• Keep API keys as Redacted<string> from typed Golem secrets.
• Start with non-streaming generateText and toolChoice: "none".
• Preserve typed provider errors unless the method intentionally converts them to defects.

Authoritative API Sources

• Effect v4 LanguageModel
• Effect v4 fetch client
• @effect/ai-openai client
• @effect/ai-openai language model
• Effect-Golem typed config