Output Meta Project Context

Output.ai Framework - Complete Project Context

What is Output.ai?

Output.ai provides infrastructure for building production-grade AI workflows: fact checkers, content generators, data extractors, research assistants, and multi-step agents. Built on Temporal, it guarantees durable execution - if execution fails mid-run, it resumes from the last successful step.

Core Philosophy

Separation of orchestration from I/O:

• Workflows orchestrate execution (must be deterministic - no I/O)
• Steps/Evaluators handle all I/O operations (HTTP, LLM, database calls)

This separation enables automatic retries, resumption, and debugging.

Component Taxonomy

Project Structure

config/
├── credentials.yml.enc          # Global encrypted credentials
├── credentials.key              # Global decryption key (DO NOT COMMIT)
└── credentials/                 # Environment-specific credentials
    ├── production.yml.enc
    └── production.key
src/
├── shared/                      # Shared code across workflows
│   ├── clients/                 # API clients (e.g., jina.ts, stripe.ts)
│   └── utils/                   # Utility functions (e.g., string.ts)
└── workflows/                   # Workflow definitions
    └── {workflow_name}/
        ├── workflow.ts          # Orchestration logic (deterministic)
        ├── steps.ts             # I/O operations
        ├── types.ts             # Zod schemas (input, output, internal)
        ├── evaluators.ts        # Quality checks (optional)
        ├── utils.ts             # Local utilities (optional)
        ├── credentials.yml.enc  # Workflow-specific credentials (optional)
        ├── prompts/             # LLM templates (optional)
        │   └── generate@v1.prompt
        ├── scenarios/           # Test inputs (optional)
        │   └── happy_path.json
        └── tests/               # Offline eval tests (optional)
            ├── datasets/        # YAML test datasets
            │   └── happy_path.yml
            └── evals/           # Eval evaluators and workflow
                ├── evaluators.ts
                └── workflow.ts

Code Reuse Rules

Shared directory (src/shared/):

• shared/clients/ - API clients using @outputai/http for external services
• shared/utils/ - Helper functions and utilities

Allowed imports:

• Workflows/steps can import from ../../shared/clients/*.js and ../../shared/utils/*.js
• Workflows/steps can import from local files (./types.js, ./utils.js)

Forbidden:

• Importing from sibling workflow folders (../other_workflow/steps.js)
• Steps importing other steps (activity isolation requirement)

Critical Rules

Determinism violations (never in workflows):

• Date.now(), new Date()
• Math.random(), crypto.randomUUID()
• Direct HTTP/fetch calls
• File system operations
• Environment variable reads

Available Tools Inventory

Agents

Skills

Workflow Authoring

Workflow Operations

Monitoring & Debugging

Error Diagnosis

Meta/Lifecycle

Development

Evals

Credentials

CLI Quick Reference

# Development
npx output dev                              # Start dev environment

# List & inspect
npx output workflow list                    # List available workflows

# Execute
npx output workflow run <name> --input '{}'  # Run synchronously (waits)
npx output workflow start <name> --input '{}' # Run async (returns ID)
npx output workflow status <id>              # Check async status
npx output workflow result <id>              # Get async result

# Debug
npx output workflow debug <id>               # Debug failed workflow
npx output workflow debug <id> --json # Machine-readable output

# Rerun from a step (replays up to <stepName>, re-executes everything after)
npx output workflow reset <id> --step <stepName>
npx output workflow reset <id> --step <stepName> --reason "why"

# Eval Testing
npx output workflow test <name>              # Run eval tests against datasets
npx output workflow test <name> --cached     # Use cached output (fast)
npx output workflow test <name> --save       # Run fresh and save results
npx output workflow dataset list <name>      # List datasets for a workflow
npx output workflow dataset generate <name> --input '{}'  # Generate dataset

# Credentials
npx output credentials init                  # Initialize encrypted credentials
npx output credentials edit                  # Edit credentials (decrypts, opens $EDITOR)
npx output credentials show                  # Show decrypted credentials
npx output credentials get <path>            # Get single credential value

Naming Conventions

Common Patterns

Workflow Pattern

import { workflow, z } from '@outputai/core';
import { fetchData, processData } from './steps.js';

export const inputSchema = z.object( { url: z.string().url() } );
export const outputSchema = z.object( { result: z.string() } );

export default workflow( {
  name: 'my_workflow',
  description: 'Processes data from URL',
  inputSchema,
  outputSchema,
  fn: async input => {
    const data = await fetchData( input.url );
    const result = await processData( data );
    return { result };
  }
} );

See output-dev-workflow-function for comprehensive patterns.

Step Pattern

import { step, z } from '@outputai/core';
import { createKyClient } from '@outputai/http';

export const fetchData = step(
  { name: 'fetchData', inputSchema: z.string(), outputSchema: z.any() },
  async url => {
    const client = createKyClient( { prefix: url } );
    const response = await client.get( '' );
    return response.json();
  }
);

See output-dev-step-function for comprehensive patterns.

HTTP Client Pattern (Shared)

Clients live in src/shared/clients/ and are shared across all workflows.

// src/shared/clients/example.ts
import { FatalError, ValidationError } from '@outputai/core';
import { createKyClient } from '@outputai/http';
import { credentials } from '@outputai/credentials';

const API_KEY = credentials.require( 'example.api_key' );

const client = createKyClient( {
  prefix: 'https://api.example.com',
  headers: { Authorization: `Bearer ${API_KEY}` },
  timeout: 30000,
  retry: { limit: 3, statusCodes: [ 408, 429, 500, 502, 503, 504 ] }
} );

export async function fetchFromExample( query: string ): Promise<ExampleResponse> {

  try {
    const response = await client.get( 'endpoint', { searchParams: { q: query } } );
    return response.json();
  } catch ( error: unknown ) {
    const err = error as { status?: number; message?: string };
    if ( err.status === 401 || err.status === 403 ) {
      throw new FatalError( `Auth failed: ${err.message}` );
    }
    throw new ValidationError( `Request failed: ${err.message}` );
  }
}

Error type guidelines:

• FatalError: 401, 403, 404 (won't succeed on retry)
• ValidationError: 429, 5xx (may succeed on retry)

See output-dev-http-client-create for comprehensive patterns.

Evaluator Pattern

Evaluators return confidence-scored results. Three result types available:

import { evaluator, z, EvaluationBooleanResult, EvaluationNumberResult, EvaluationStringResult } from '@outputai/core';

// Boolean evaluator - pass/fail checks
export const evaluateCompleteness = evaluator( {
  name: 'evaluate_completeness',
  description: 'Check if content meets minimum length',
  inputSchema: z.object( { content: z.string(), minLength: z.number() } ),
  fn: async ( { content, minLength } ) => {
    return new EvaluationBooleanResult( {
      value: content.length >= minLength,
      confidence: 1.0,
      reasoning: `Content has ${content.length} chars (min: ${minLength})`
    } );
  }
} );

See output-dev-evaluator-function for comprehensive patterns.

Prompt File Pattern

Prompts use YAML frontmatter + Liquid.js templating. Location: src/workflows/{name}/prompts/

---
provider: anthropic
# current as of 2026-05-04 — run output-dev-model-selection for the latest
model: claude-sonnet-4-6
temperature: 0.7
maxOutputTokens: 4096
---

<system>
You are an expert content analyzer.

{% if context %}
Additional context: {{ context }}
{% endif %}
</system>

<user>
Analyze the following content:

<content>
{{ content }}
</content>

Provide {{ numberOfPoints | default: 3 }} key insights.
</user>

Using in steps:

import { generateText, aiSdk } from '@outputai/llm';
import { z } from '@outputai/core';

// Structured output
const { output } = await generateText( {
  prompt: 'analyze@v1',
  variables: { content: 'Article text...', numberOfPoints: 5 },
  output: aiSdk.Output.object( {
    schema: z.object( { insights: z.array( z.string() ) } )
  } )
} );

// Text output
const { result } = await generateText( {
  prompt: 'summarize@v1',
  variables: { content: 'Article text...' }
} );

Call arguments: prompt, promptDir, variables, tools, output, toolChoice, stopWhen, abortSignal, plus streaming callbacks.

For progress callbacks inside a step, prefer generateTextWithStreaming() or Agent.generateWithStreaming(). They return complete results and reject on stream failures. If direct streamText() access is required, capture onError and throw the captured error after consuming the stream. See output-dev-llm-streaming.

Provider & model selection: the SDK supports anthropic, openai, google-vertex, amazon-bedrock, azure, and perplexity. Don't pin specific model IDs in docs — they drift. To pick a current model, run output-dev-model-selection, which queries the AI Gateway model index live.

See output-dev-prompt-file for comprehensive patterns.

Practical Tips

Docker & Services

• Restart worker after adding workflows: docker restart <project>-worker-1
• View worker logs: docker logs -f output-worker-1
• Check services: Use output-services-check skill

Payload Limits

• Temporal: ~2MB per workflow input/output
• gRPC: ~4MB maximum
• For larger data, use file storage and pass references

Debugging Workflow Failures

1. Get the workflow ID from error output
2. Run npx output workflow debug <id> --json
3. Look for: failed step name, error message, input that caused failure
4. Check if issue is determinism, schema validation, or external API