# Llmobs Integrations

> dd-trace-py LLMObs integration development guide. Use when creating, modifying, or debugging LLMObs integrations for LLM/AI libraries in the Python tracer. Covers BaseLLMIntegration, stream handling, message extraction, token counting, tool call parsing, and VCR-based testing patterns. Triggers: "llmobs", "LLMObs", "BaseLLMIntegration", "llmobs_set_tags", "_llmobs_set_tags", "BaseStreamHandler", "submit_to_llmobs", "integration.trace", "LLM span", "VCR", "cassette", "anthropic", "openai", "google_genai", "claude_agent_sdk", "generative-ai", "LLM integration", "llmobs_enabled".

- Skill: `datadog/llmobs-integrations` (Agent Skill, multi-file: 4 files)
- Install (CLI): `npx skillmds@latest add datadog/llmobs-integrations`
- Raw SKILL.md: https://api.skillmd.com/api/skills/datadog/llmobs-integrations/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: datadog (https://skillmd.com/u/datadog)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/datadog/llmobs-integrations

---


# dd-trace-py LLMObs Integrations

LLMObs integrations enable Datadog LLM Observability for AI/LLM libraries. They extract model inputs, outputs, token usage, and tool calls from traced spans. This skill should be used in addition to the `apm-integrations` skill.

## Two-Layer Architecture

LLMObs integrations consist of two cooperating layers:

1. **Patch Layer** (`ddtrace/contrib/internal/{name}/patch.py`) -- wraps library functions. Standard request/response LLM integrations construct `LlmRequestEvent` and use `core.context_with_event()` so the LLM tracing subscriber owns span lifecycle and LLMObs tag extraction.
2. **Integration Layer** (`ddtrace/llmobs/_integrations/{name}.py`) -- extends `BaseLLMIntegration`, implements `_set_base_span_tags()` and `_llmobs_set_tags()` to extract and set provider-specific messages, tools, metadata, and token metrics.

Both layers must work together. The patch layer identifies the operation and passes request/response data through the event; the integration layer controls what data is extracted.

## Active Patch Patterns

- **Event-based request spans**: Use `LlmRequestEvent` with `core.context_with_event()` for new standard request/response LLM integrations. Anthropic is the canonical reference. This is the preferred pattern.
- **Direct integration spans**: Some existing or specialized integrations still call `integration.trace()` and `integration.llmobs_set_tags()` directly, especially for child spans, agent/tool spans, or integrations not yet migrated. Google GenAI, OpenAI tool spans, and Claude Agent SDK are useful references.

## Key Files

| Purpose | File |
|---------|------|
| Base LLM integration class | `ddtrace/llmobs/_integrations/base.py` (`BaseLLMIntegration`) |
| Stream handler base classes | `ddtrace/llmobs/_integrations/base_stream_handler.py` (`BaseStreamHandler`, `StreamHandler`, `AsyncStreamHandler`) |
| Shared utilities | `ddtrace/llmobs/_integrations/utils.py` |
| LLMObs annotation helper | `ddtrace/llmobs/_utils.py` (`_annotate_llmobs_span_data`) |
| LLMObs constants | `ddtrace/llmobs/_constants.py` |
| LLMObs types | `ddtrace/llmobs/types.py` (`Message`, `AudioPart`, `ToolCall`, `ToolResult`, `ToolDefinition`) |
| Integration registry | `ddtrace/llmobs/_integrations/__init__.py` |

## Reference Integrations

**Always read 1-2 references before writing or modifying LLMObs code.**

| Provider | Patch File | LLMObs Integration | LLMObs Tests |
|----------|-----------|-------------------|--------------|
| **Anthropic** (canonical) | `ddtrace/contrib/internal/anthropic/patch.py` | `ddtrace/llmobs/_integrations/anthropic.py` | `tests/contrib/anthropic/test_anthropic_llmobs.py` |
| **Claude Agent SDK** (latest, agent pattern) | `ddtrace/contrib/internal/claude_agent_sdk/patch.py` | `ddtrace/llmobs/_integrations/claude_agent_sdk.py` | `tests/contrib/claude_agent_sdk/test_claude_agent_sdk_llmobs.py` |
| OpenAI | `ddtrace/contrib/internal/openai/patch.py` | `ddtrace/llmobs/_integrations/openai.py` | `tests/contrib/openai/test_openai_llmobs.py` |
| Google GenAI | `ddtrace/contrib/internal/google_genai/patch.py` | `ddtrace/llmobs/_integrations/google_genai.py` | `tests/contrib/google_genai/test_google_genai_llmobs.py` |

Use **Anthropic** as the canonical reference for standard LLM integrations. Use **Claude Agent SDK** for agent-pattern integrations (agent spans, tool child spans, thinking blocks).

## Abstract Methods to Implement

Subclass `BaseLLMIntegration` and implement:

### `_set_base_span_tags(span, **kwargs)`
Set provider-specific APM tags on the span (e.g., `{name}.request.model`).

### `_llmobs_set_tags(span, args, kwargs, response, operation)`
Extract and annotate all LLMObs fields on the span:

| Field | Description |
|-------------|-------------|
| `kind` | `"llm"` for LLM calls, `"agent"` for agent calls, `"tool"` for tool calls |
| `model_name` | Model identifier (e.g., `"claude-3-sonnet-20240229"`) |
| `model_provider` | Provider name (e.g., `"anthropic"`, `"openai"`) |
| `input_messages` | List of `Message` objects from request |
| `output_messages` | List of `Message` objects from response |
| `metadata` | Dict of sanitized request parameters (temperature, top_p, etc.), plus response-derived scalars such as `finish_reason` (see Response Metadata below) |
| `metrics` | Token usage dict with `INPUT_TOKENS_METRIC_KEY`, `OUTPUT_TOKENS_METRIC_KEY`, `TOTAL_TOKENS_METRIC_KEY` |
| `tool_definitions` | List of `ToolDefinition` objects if tools are passed |

Fields are usually set via `_annotate_llmobs_span_data(...)`, not raw `span._set_ctx_items(...)`.

### Response Metadata

`metadata` is not request-only. Scalars the provider reports on the *response* belong there too, so
long as they are not already covered by `metrics` (token counts) or `output_messages`.

The stop reason is the established case. Record it under the key `finish_reason` for every provider,
so one facet answers "why did generation stop" regardless of integration, and keep each provider's
own vocabulary as the value (`stop`/`length`/`content_filter`/`tool_calls` for the OpenAI family,
`end_turn`/`max_tokens`/`refusal`/`tool_use` for Anthropic). Read it from:

| Provider | Source |
|---|---|
| openai/litellm chat + legacy completions | `choice.finish_reason` (per choice) |
| openai responses API | `incomplete_details.reason` |
| anthropic | `response.stop_reason`, falling back to `response.finish_reason` for the streamed dict the aggregator rebuilds |

Rules:

- **Merge, do not replace.** Build the request metadata first, then update it with the
  response-derived keys (`parameters.update(...)`), so response data never clobbers request params.
- **Omit the key when absent.** If the provider reports no reason, leave `finish_reason` out
  entirely rather than writing `None` or `""`.
- **Keep the value a single string.** When a request returns multiple choices (`n > 1`), comma-join
  the per-choice reasons in choice order (`"stop,length"`) instead of emitting a list, so the key
  never changes type. `_openai_finish_reason_metadata()` in `_integrations/utils.py` does this.
- **Guard on the response, not `span.error`.** A response can exist on an errored span (see the AI
  Guard case in `_integrations/anthropic.py`); gate on `response is not None`.

Note two already-shipped integrations predate this key: bedrock and the claude-agent-sdk both write
`metadata["stop_reason"]`. Renaming those is a breaking change and has not been done — follow
`finish_reason` for new work.

## Key Constraints

- **`submit_to_llmobs=True`** must be set on `LlmRequestEvent` for event-based request spans or passed to `integration.trace()` for direct LLMObs spans
- **`ctx.dispatch_ended_event()`** must run on success and error paths for event-based patch wrappers
- **Streaming** must use `BaseStreamHandler`/`AsyncStreamHandler` -- never consume streams directly
- **Event-based patch wrappers** should not call `span.set_exc_info()`, `span.finish()`, or `integration.llmobs_set_tags()` directly; the tracing subscriber handles that when the event ends
- **Direct integration spans** must keep `integration.llmobs_set_tags()` and span lifecycle handling aligned with the closest current reference
- **Integration instance** must be stored on the module: `module._datadog_integration = MyLibIntegration(integration_config=config.mylib)`

## Message Types

```python
from ddtrace.llmobs.types import AudioPart, ImagePart, Message, ToolCall, ToolResult, ToolDefinition

# Input/output messages
Message(content="text", role="user")
Message(content="response", role="assistant", tool_calls=[...])

# Audio attachments in multimodal messages
AudioPart(mime_type="audio/wav", content="<base64-audio>")
Message(content="", role="user", audio_parts=[...])

# Image attachments in multimodal messages
ImagePart(mime_type="image/png", content="<base64-image>")
Message(content="", role="user", image_parts=[...])

# Tool calls (in output messages)
ToolCall(name="get_weather", arguments={"city": "NYC"}, tool_id="toolu_123", type="tool")

# Tool results (in input messages)
ToolResult(result="72F sunny", tool_id="toolu_123", type="tool_result")

# Tool definitions (from request parameters)
ToolDefinition(name="get_weather", description="...", schema={...})
```

## Debugging Quick Tips

- **No LLMObs spans** -- check `submit_to_llmobs=True`, `ctx.dispatch_ended_event()`, and `llmobs_enabled`
- **Wrong messages** -- check message extraction handles multi-part content and tool blocks
- **Wrong tokens** -- check field name mapping (libraries use different names for token counts)
- **Streaming broken** -- verify `BaseStreamHandler` subclass, check `finalize_stream()` dispatches the ended event or finishes direct-trace spans according to the reference pattern
- **DD_TRACE_DEBUG=true** to see patching activity and span creation

See [Failure Modes](references/failure-modes.md) for detailed debugging guide.

## Reference Files

- [Implementation Guide](references/implementation-guide.md) -- LLM-specific steps (references apm-integrations guide for the full workflow)
- [Failure Modes](references/failure-modes.md) -- All 8 failure modes with causes and fixes
- [Testing Guide](references/testing-guide.md) -- LLMObs test patterns, VCR cassettes, suitespec