Output Dev LLM Streaming

LLM Text Streaming

When to Use This Skill

• Adding token or chunk progress to an LLM-powered step
• Choosing between completed generation and direct stream access
• Using onChunk, or onEnd / onError on streamText() / Agent.stream()
• Making stream failures trigger Temporal activity retries
• Streaming Agent responses or persisting streamed conversations

Choose the API

In workflow steps, prefer generateTextWithStreaming() or Agent.generateWithStreaming() when onChunk progress is sufficient. They consume the stream internally, return complete results like generateText() or Agent.generate(), and reject on provider, transport, or abort errors. Rejection allows Temporal to record the failed activity attempt and apply the step retry policy.

streamText() and Agent.stream() remain supported for code that needs direct control over stream consumption.

generateTextWithStreaming()

import { generateTextWithStreaming } from '@outputai/llm';

const result = await generateTextWithStreaming( {
  prompt: 'draft@v1',
  variables: { topic },
  onChunk( { chunk } ) {
    if ( chunk.type === 'text-delta' ) {
      process.stdout.write( chunk.text );
    }
  }
} );

return result.result;

The result has the same complete response fields as generateText(), including result, text, output, usage, finishReason, and cost. Structured output passed with aiSdk.Output.* is available through result.output.

Agent.generateWithStreaming()

const result = await agent.generateWithStreaming( {
  onChunk( { chunk } ) {
    if ( chunk.type === 'text-delta' ) {
      process.stdout.write( chunk.text );
    }
  }
} );

generateWithStreaming() returns a complete Agent response and automatically stores messages when the Agent has a messageStore.

Direct stream error handling

AI SDK streaming delivers provider and transport failures through onError. Iterating textStream does not reliably throw the original error. When using streamText() in a workflow step, capture the error and throw it after consumption:

import { streamText } from '@outputai/llm';

const captured: { error: unknown } = { error: null };
const result = streamText( {
  prompt: 'draft@v1',
  variables: { topic },
  onError( { error } ) {
    captured.error = error;
  }
} );

const chunks: string[] = [];
for await ( const chunk of result.textStream ) {
  chunks.push( chunk );
}

if ( captured.error ) {
  throw captured.error;
}

return chunks.join( '' );

Registering onError without throwing the captured error can let the step return an empty successful result, preventing Temporal from retrying it. Awaiting a completion property may also produce a generic no-output error instead of the original provider error.

Agent.stream() stores conversation messages in its wrapped onEnd when finishReason is not 'error'. Use Agent.generateWithStreaming() when a complete stored response meets the requirement.

Streaming call arguments: prompt, promptDir, variables, tools, output, toolChoice, stopWhen, abortSignal, plus onChunk (generateTextWithStreaming) or onChunk / onEnd / onError (streamText). Agent methods: messages, abortSignal, toolChoice, plus those same stream callbacks.

Rules

• Prefer the completed streaming APIs in Temporal steps unless direct stream access is required.
• Do not rely on onError alone to fail a step using streamText().
• Throw the captured error only after stream consumption finishes.
• Keep onChunk side effects bounded. A Temporal signal per token creates a history event per signal, so batch high-frequency updates.
• Do not describe streamText() or Agent.stream() as deprecated.

Related Skills

• output-dev-step-function - Put LLM calls inside Temporal activity steps
• output-dev-agent-class - Construct and use reusable Agents
• output-dev-prompt-file - Create prompt files for generation
• output-error-try-catch - Handle step and workflow failures