Temporal

Temporal Workflow Orchestration

Expert-level knowledge for building durable, reliable distributed systems with Temporal.io. Covers the Python SDK (temporalio), TypeScript SDK (@temporalio/*), core platform concepts, and AI/LLM orchestration patterns.

References

Platform Overview

Temporal provides durable execution -- application state survives process crashes, infrastructure failures, and deployments through event-sourced replay. The platform consists of the Temporal Service (server + persistence) and application code (Workers + Clients). Workers execute Workflow and Activity code on user infrastructure; the Temporal Service coordinates task dispatch, state persistence, and timer management.

Key Primitives

Determinism Contract

Workflow code replays from event history on recovery. Every replay produces the same sequence of Commands given the same Events. Non-deterministic operations (I/O, randomness, system time, threading) belong in Activities.

Timeout Model

Start-to-Close Timeout is the primary mechanism for detecting worker failures and is the most important timeout to set on activities.

Retry Policy Defaults

Activities retry by default. Workflows do not retry by default.

SDK Quick Reference

Python

Package: temporalio (pip install temporalio)

from temporalio import workflow, activity
from temporalio.client import Client
from temporalio.worker import Worker

@activity.defn
async def greet(name: str) -> str:
    return f"Hello, {name}!"

@workflow.defn
class GreetingWorkflow:
    @workflow.run
    async def run(self, name: str) -> str:
        return await workflow.execute_activity(
            greet, name,
            start_to_close_timeout=timedelta(seconds=10),
        )

# Client
client = await Client.connect("localhost:7233")
result = await client.execute_workflow(
    GreetingWorkflow.run, "World",
    id="greeting-1", task_queue="my-queue",
)

# Worker
worker = Worker(client, task_queue="my-queue",
    workflows=[GreetingWorkflow], activities=[greet])
await worker.run()

Python decorators: @workflow.defn, @workflow.run, @workflow.init, @workflow.signal, @workflow.query, @workflow.update, @activity.defn

Sandbox: Workflow code runs in an isolated sandbox. Use with workflow.unsafe.imports_passed_through(): for importing activity types and dataclasses.

TypeScript

Packages: @temporalio/client, @temporalio/worker, @temporalio/workflow, @temporalio/activity, @temporalio/common

// activities.ts
export async function greet(name: string): Promise<string> {
  return `Hello, ${name}!`;
}

// workflows.ts
import { proxyActivities } from '@temporalio/workflow';
import type * as activities from './activities';

const { greet } = proxyActivities<typeof activities>({
  startToCloseTimeout: '10s',
});

export async function greetingWorkflow(name: string): Promise<string> {
  return await greet(name);
}

// client.ts
const handle = await client.workflow.start(greetingWorkflow, {
  workflowId: 'greeting-1', taskQueue: 'my-queue', args: ['World'],
});

// worker.ts
const worker = await Worker.create({
  workflowsPath: require.resolve('./workflows'),
  activities, taskQueue: 'my-queue',
});
await worker.run();

Key pattern: Workflows import only types from activities (import type * as activities). Actual implementations are registered on the Worker. Activities are accessed via proxyActivities<typeof activities>().

Sandbox: Workflow code is bundled with Webpack into a V8 isolate. Math.random(), Date.now(), and setTimeout are replaced with deterministic versions. Node.js built-ins (fs, path, crypto, http) are unavailable in workflow code.

Pattern Selection Guide

AI Orchestration Overview

Temporal is infrastructure for production AI agent systems. The core pattern: a Workflow maintains durable agent state and conversation history, while Activities handle non-deterministic LLM calls, tool invocations, and external API requests.

Key properties for AI workloads:

• Completed Activity results are persisted via event sourcing -- crash recovery replays recorded LLM responses rather than re-querying (saves cost, preserves determinism)
• Retry policies handle rate limits (429), transient failures, and API timeouts automatically
• Signals enable human-in-the-loop approval gates with zero-compute waiting
• Task Queues route inference work to GPU-equipped workers
• Conversation history stored as workflow state is automatically durable

Framework Integrations

AI-Specific Configuration

LLM client libraries have built-in retry logic that conflicts with Temporal's retry management. Disable client-level retries and let Temporal handle them:

# Python - Anthropic
client = AsyncAnthropic(max_retries=0)

# Python - OpenAI
client = AsyncOpenAI(max_retries=0)

// TypeScript - OpenAI
const client = new OpenAI({ maxRetries: 0 });

Recommended retry policy for LLM activities:

RetryPolicy(
    initial_interval=timedelta(seconds=1),
    backoff_coefficient=2.0,
    maximum_interval=timedelta(seconds=120),
    maximum_attempts=50,
    non_retryable_error_types=["InvalidInput", "AuthenticationError"],
)

Parse Retry-After headers from 429 responses using next_retry_delay on ApplicationError / ApplicationFailure.

Development Environment

# Install Temporal CLI
brew install temporal

# Start local dev server (gRPC :7233, Web UI :8233)
temporal server start-dev

# Persistent storage between restarts
temporal server start-dev --db-filename temporal.db

# Common CLI commands
temporal workflow start --task-queue my-queue --type MyWorkflow --input '{"key":"val"}'
temporal workflow list
temporal workflow describe --workflow-id my-id
temporal workflow signal --workflow-id my-id --name my-signal --input '"data"'
temporal workflow query --workflow-id my-id --name my-query
temporal workflow cancel --workflow-id my-id

Versioning Strategy

Both SDKs provide a patched() / deprecate_patch() mechanism for evolving workflow code while existing executions are running:

1. Add patch: Branch on patched("change-id") -- new executions take new path, replaying executions take old path
2. Deprecate patch: Once all old executions complete, call deprecate_patch("change-id") and remove old code path
3. Remove patch: After retention window, remove the deprecate call entirely