DeepEval OpenTelemetry Export
Use this skill to instrument an AI application — an LLM app, agent, RAG
pipeline, or chatbot — with raw OpenTelemetry so its traces land in
Confident AI's Observatory. No deepeval package is needed — it works with
any OTLP-capable OpenTelemetry SDK. The job is exactly two things: export to
the correct Confident AI OTLP endpoint, and set the confident.* attributes
Confident AI reads off each span.
Scope: AI Applications Only
This skill instruments AI applications only. The confident.* attributes
and span types — agent, llm, retriever, tool — describe AI components,
and Confident AI's Observatory is built to evaluate and monitor AI behavior.
Instrument only the AI parts of the system: agent loops and planning, LLM
calls, retrieval / vector search, and tool calls. Do not apply confident.*
attributes to non-AI software (web servers, CRUD backends, database layers,
infrastructure) or to non-AI spans inside an otherwise-AI app — that data does
not belong in Confident AI and will not render meaningfully. If the target has
no LLM, agent, retrieval, or tool-calling component, this skill does not apply.
When to Use vs the deepeval Skill
Use this skill for vendor-neutral OTLP export to Confident AI — pointing an
OpenTelemetry exporter at Confident AI and setting confident.* attributes.
Use the deepeval skill when the user wants to build a Python pytest eval
suite, generate datasets or goldens, write metrics, run deepeval test run, or
instrument with the deepeval SDK's @observe decorator. The two skills are
complementary, not alternatives.
Prerequisites
- A Confident AI account and a
CONFIDENT_API_KEY.
- An OpenTelemetry SDK for the application's language. For Python:
opentelemetry-sdk and opentelemetry-exporter-otlp-proto-http.
- The Confident AI OTLP endpoint accepts HTTP only — never gRPC.
How It Works
Confident AI exposes an OTLP/HTTP traces endpoint. Point any OpenTelemetry span
exporter at it with the x-confident-api-key header. Confident AI's exporter
then reads confident.* attributes off each span to build the trace and span
structure. Parent/child nesting comes from native OpenTelemetry span context,
not from any attribute.
Workflow
- Confirm the target is an AI application (it has LLM calls, an agent loop,
retrieval, or tool calls). If it has none of these, stop — this skill does
not apply. Then inspect for an existing OpenTelemetry setup (a
TracerProvider, span exporters, or an OpenTelemetry Collector) and prefer
repointing what exists over adding a parallel pipeline.
- Choose the endpoint from the API key's region prefix. Read
references/endpoint-and-exporter.md.
- Wire (or repoint) an OTLP/HTTP span exporter with the
x-confident-api-key
header. For Python, start from templates/confident_otel_setup.py.
- If the process runs other OpenTelemetry instrumentation or an APM agent
(auto-instrumentation for HTTP/DB, Datadog, etc.), isolate the Confident AI
export so only AI spans reach it — a dedicated pipeline or a span filter.
Read "Export Only AI Spans" in
references/endpoint-and-exporter.md.
- Set
confident.span.* attributes on spans; set confident.trace.* for
trace-wide fields. Read references/span-attributes.md and
references/trace-attributes.md.
- Honor the OTLP data-type rules: JSON-encode dicts/metadata, use native
arrays for string lists. See the Data-Type Rules in
span-attributes.md.
- If the app already emits OpenTelemetry GenAI semantic conventions, read
references/gen-ai-fallbacks.md before adding redundant attributes.
- Verify traces appear in the Confident AI Observatory.
Core Principles
- Instrument AI components only — agent, LLM, retriever, and tool spans.
Never apply
confident.* attributes to non-AI software or non-AI spans.
- Export only AI spans. If the process has other OpenTelemetry
instrumentation or an APM agent, isolate the Confident AI pipeline (a
dedicated provider or a span filter) so non-AI spans — HTTP requests, DB
queries, infra — are never exported to Confident AI.
- Prefer repointing an existing OTLP exporter over adding a parallel one.
- The
confident.* attribute keys are the entire contract — they are the
same in every language, so language choice is irrelevant.
- Always use OTLP/HTTP. Confident AI's endpoint does not accept gRPC.
- Honor OTLP data-type rules: attribute values must be primitives or
homogeneous primitive lists; JSON-encode dicts and metadata.
- Set
confident.span.type explicitly when it is known; rely on gen_ai.*
inference only as a fallback.
- Never put secrets, credentials, or raw sensitive data into span attributes.
References
| Topic |
File |
| Endpoints, region selection, auth, exporter wiring |
references/endpoint-and-exporter.md |
Trace-level confident.trace.* attributes |
references/trace-attributes.md |
Span-level confident.span.* attributes and data-type rules |
references/span-attributes.md |
Standard OTel gen_ai.* fallback behavior |
references/gen-ai-fallbacks.md |
Templates
| Purpose |
Template |
| Minimal Python OTLP exporter setup + example trace |
templates/confident_otel_setup.py |
1---2name: deepeval-otel3description: >4 Export raw OpenTelemetry traces from an AI application to Confident AI's5 Observatory. TRIGGER when the user wants to send OpenTelemetry or OTLP6 traces/spans from an LLM app, agent, RAG pipeline, or chatbot to Confident7 AI; configure the Confident AI OTLP endpoint; set confident.span.* or8 confident.trace.* attributes; export AI-app traces without the deepeval9 Python package; wire an OTLPSpanExporter, OpenTelemetry Collector, or10 vendor-neutral OTel SDK to Confident AI; or pick the US vs EU OTLP endpoint.11 Language-agnostic: the mechanism is OTLP attribute keys plus an exporter12 endpoint. DO NOT TRIGGER for building DeepEval pytest eval suites, datasets,13 goldens, metrics, or deepeval test run (use the `deepeval` skill); for14 instrumenting with the DeepEval SDK's @observe decorator or framework15 integrations (use the `deepeval-tracing` skill); or for non-AI software such16 as web servers, CRUD backends, or infrastructure: the confident.* attributes17 describe AI components only.18license: Apache-2.019metadata:20 source: https://skillmd.com/skills/confident-ai/deepeval-otel21 author: Confident AI22 version: "1.0.0"23 category: observability24 tags: "opentelemetry, otel, otlp, tracing, confident-ai, observatory, spans"25 compatibility: "Works with any OpenTelemetry SDK in any language. Requires a Confident AI account and `CONFIDENT_API_KEY`. Confident AI's OTLP endpoint is HTTP only — use OTLP/HTTP, not gRPC. Python examples assume `opentelemetry-sdk` and `opentelemetry-exporter-otlp-proto-http`."26---2728# DeepEval OpenTelemetry Export2930Use this skill to instrument an **AI application** — an LLM app, agent, RAG31pipeline, or chatbot — with **raw OpenTelemetry** so its traces land in32**Confident AI's Observatory**. No `deepeval` package is needed — it works with33any OTLP-capable OpenTelemetry SDK. The job is exactly two things: export to34the correct Confident AI OTLP endpoint, and set the `confident.*` attributes35Confident AI reads off each span.3637## Scope: AI Applications Only3839This skill instruments **AI applications only**. The `confident.*` attributes40and span types — `agent`, `llm`, `retriever`, `tool` — describe AI components,41and Confident AI's Observatory is built to evaluate and monitor AI behavior.4243Instrument only the AI parts of the system: agent loops and planning, LLM44calls, retrieval / vector search, and tool calls. Do **not** apply `confident.*`45attributes to non-AI software (web servers, CRUD backends, database layers,46infrastructure) or to non-AI spans inside an otherwise-AI app — that data does47not belong in Confident AI and will not render meaningfully. If the target has48no LLM, agent, retrieval, or tool-calling component, this skill does not apply.4950## When to Use vs the `deepeval` Skill5152Use **this skill** for vendor-neutral OTLP export to Confident AI — pointing an53OpenTelemetry exporter at Confident AI and setting `confident.*` attributes.5455Use the **`deepeval` skill** when the user wants to build a Python pytest eval56suite, generate datasets or goldens, write metrics, run `deepeval test run`, or57instrument with the `deepeval` SDK's `@observe` decorator. The two skills are58complementary, not alternatives.5960## Prerequisites6162- A Confident AI account and a `CONFIDENT_API_KEY`.63- An OpenTelemetry SDK for the application's language. For Python:64 `opentelemetry-sdk` and `opentelemetry-exporter-otlp-proto-http`.65- The Confident AI OTLP endpoint accepts **HTTP only** — never gRPC.6667## How It Works6869Confident AI exposes an OTLP/HTTP traces endpoint. Point any OpenTelemetry span70exporter at it with the `x-confident-api-key` header. Confident AI's exporter71then reads `confident.*` attributes off each span to build the trace and span72structure. Parent/child nesting comes from native OpenTelemetry span context,73not from any attribute.7475## Workflow76771. Confirm the target is an AI application (it has LLM calls, an agent loop,78 retrieval, or tool calls). If it has none of these, stop — this skill does79 not apply. Then inspect for an existing OpenTelemetry setup (a80 `TracerProvider`, span exporters, or an OpenTelemetry Collector) and prefer81 repointing what exists over adding a parallel pipeline.822. Choose the endpoint from the API key's region prefix. Read83 `references/endpoint-and-exporter.md`.843. Wire (or repoint) an OTLP/HTTP span exporter with the `x-confident-api-key`85 header. For Python, start from `templates/confident_otel_setup.py`.864. If the process runs other OpenTelemetry instrumentation or an APM agent87 (auto-instrumentation for HTTP/DB, Datadog, etc.), isolate the Confident AI88 export so only AI spans reach it — a dedicated pipeline or a span filter.89 Read "Export Only AI Spans" in `references/endpoint-and-exporter.md`.905. Set `confident.span.*` attributes on spans; set `confident.trace.*` for91 trace-wide fields. Read `references/span-attributes.md` and92 `references/trace-attributes.md`.936. Honor the OTLP data-type rules: JSON-encode dicts/metadata, use native94 arrays for string lists. See the Data-Type Rules in `span-attributes.md`.957. If the app already emits OpenTelemetry GenAI semantic conventions, read96 `references/gen-ai-fallbacks.md` before adding redundant attributes.978. Verify traces appear in the Confident AI Observatory.9899## Core Principles1001011. Instrument AI components only — agent, LLM, retriever, and tool spans.102 Never apply `confident.*` attributes to non-AI software or non-AI spans.1032. Export only AI spans. If the process has other OpenTelemetry104 instrumentation or an APM agent, isolate the Confident AI pipeline (a105 dedicated provider or a span filter) so non-AI spans — HTTP requests, DB106 queries, infra — are never exported to Confident AI.1073. Prefer repointing an existing OTLP exporter over adding a parallel one.1084. The `confident.*` attribute keys are the entire contract — they are the109 same in every language, so language choice is irrelevant.1105. Always use OTLP/HTTP. Confident AI's endpoint does not accept gRPC.1116. Honor OTLP data-type rules: attribute values must be primitives or112 homogeneous primitive lists; JSON-encode dicts and metadata.1137. Set `confident.span.type` explicitly when it is known; rely on `gen_ai.*`114 inference only as a fallback.1158. Never put secrets, credentials, or raw sensitive data into span attributes.116117## References118119| Topic | File |120| --- | --- |121| Endpoints, region selection, auth, exporter wiring | `references/endpoint-and-exporter.md` |122| Trace-level `confident.trace.*` attributes | `references/trace-attributes.md` |123| Span-level `confident.span.*` attributes and data-type rules | `references/span-attributes.md` |124| Standard OTel `gen_ai.*` fallback behavior | `references/gen-ai-fallbacks.md` |125126## Templates127128| Purpose | Template |129| --- | --- |130| Minimal Python OTLP exporter setup + example trace | `templates/confident_otel_setup.py` |