Jaeger Knowledge Patch
Use this skill when migrating, configuring, extending, or operating Jaeger v2.
Start with the breaking-change checklist, then open the reference matching the
work at hand. Treat the project's manifests, configuration, generated APIs,
tests, and observed runtime behavior as authoritative.
Reference index
| Reference |
Topics |
| runtime-configuration-and-migration.md |
Unified runtime roles, Collector YAML, server configuration, named storage, base paths, source builds |
| storage-backends-and-contracts.md |
Storage v2 and remote-storage contracts, memory, Badger, Cassandra, ClickHouse |
| elasticsearch-and-opensearch.md |
Backend compatibility, mappings, aliases, rotation, lifecycle, authentication, summaries |
| query-sampling-metrics-and-telemetry.md |
Sampling, query APIs, normalization, Prometheus, SPM, internal telemetry |
| operations-deployment-and-development.md |
Pipelines, endpoints, Compose and Kubernetes assets, dashboards, platforms |
| ui-and-integrations.md |
Search, trace visualization, Monitor, themes, MCP, AI-assisted integrations |
Breaking-change checklist
Distribution and configuration
- Deploy the unified
jaeger binary or image. Do not expect a separately
published jaeger-agent, v1 service image, or v1 image tag.
- Express runtime roles as Collector-style YAML. V1 environment variables are
not automatically translated into v2 settings.
- Review query HTTP/TLS settings and assign separate listen ports to the HTTP
and gRPC query servers.
- Select exactly one backend type in each trace- or metric-storage
configuration.
- Replace Kafka receiver
traces.topic with list-valued traces.topics.
- Use Go 1.25.7 for source builds and pnpm for UI or embedded-UI builds.
Removed contracts and compatibility paths
- Remove the
grpc-plugin storage type and migrate off storage/v1/grpc.
- Replace archive-specific storage interfaces with the unified storage
factories.
- Update remote-storage implementations for streaming
GetTraces, direct OTLP
stream elements, current request/response fields, and
FindTraceIDsRequest.
- Remove the metric-query gRPC service and the metric-store minimum-step API
from custom integrations.
- Remove
sampling.strategies.bugfix-5270; its corrected behavior is stable.
- Update remote-sampling clients that parse the removed legacy response.
- Do not script around
jaeger features; that command was reverted.
- Replace imports that moved under Go
internal packages.
Storage and backend upgrades
- Upgrade Cassandra 3.x and Elasticsearch 6 before upgrading Jaeger.
- Recheck Elasticsearch/OpenSearch TLS, mappings, aliases, authentication,
rotation, trace-duration, and legacy trace-ID settings.
- Plan for materialized
span.kind and span.status, populated
parentspanid, and mapped OTLP scope/link fields.
- Treat API keys, bearer tokens, and SigV4 as mutually constrained
authentication choices; let validation reject incompatible combinations.
- Update custom dependency writers to accept and propagate context.
Query and response behavior
- Expect trace adjusters to modify clock skew, duplicates, identifiers, and
attribute/event ordering in API results.
- Treat invalid OTLP payloads as HTTP 400 request failures.
- Expect empty service queries to return an empty collection.
- Accept camel-case API v3 parameters while migrating away from deprecated
snake-case aliases and
query.num_traces.
- Treat trace IDs as opaque strings; query endpoints may receive base64
identifiers.
- Update dashboards and alerts for the corrected counter suffix behavior and
the newer SPM namespace.
Runtime quick reference
Override Collector YAML
Use environment interpolation for deploy-time values and --set for a dotted
path:
receivers:
otlp:
protocols:
grpc:
endpoint: "${env:JAEGER_LISTEN_HOST:-localhost}:4317"
jaeger --config config.yaml \
--set=receivers.otlp.protocols.grpc.endpoint=0.0.0.0:4317
This also works for paths in the built-in all-in-one configuration. Remember
that the OTLP receiver can listen on all interfaces; bind and filter it
deliberately.
Wire named storage
Register backends once in jaeger_storage, select them for reads in
jaeger_query, and select the trace backend for writes in
jaeger_storage_exporter:
extensions:
jaeger_storage:
backends:
trace_store:
memory:
max_traces: 100000
jaeger_query:
storage:
traces: trace_store
exporters:
jaeger_storage_exporter:
trace_storage: trace_store
Configure exporter queueing and retries when storage failures or bursts must be
buffered.
Configure path-prefixed deployments
jaeger_query.base_path describes the path Jaeger receives:
extensions:
jaeger_query:
base_path: /jaeger
Leave it / if a reverse proxy strips the external prefix. Otherwise use the
forwarded prefix. The path must begin with / and must not contain .. or
duplicate slashes. The UI can discover its browser prefix automatically, and
API v3 requests honor the prefix.
Configure remote sampling
adaptive_sampling calculates service and endpoint probabilities.
remote_sampling serves a static file or adaptive store over HTTP or gRPC:
extensions:
remote_sampling:
adaptive:
sampling_store: sampling_store
initial_sampling_probability: 0.1
http:
grpc:
Review inherited per-operation defaults and use a reload interval when a
strategy file must update without restart.
Storage quick reference
Remote storage
Use the v2 factory and streaming contracts for trace, service, operation,
dependency, and trace-ID operations.
Forward tenant or authentication headers only when the backend expects them.
Set max_recv_msg_size_mib when the default gRPC receive limit is too small:
max_recv_msg_size_mib: 64
Keep the remote-storage server's write path on the normal OTLP
receiver/exporter pipeline.
Elasticsearch and OpenSearch
Use the mapping generator embedded in jaeger. Select OpenSearch output with:
jaeger esmapping-generator --backend opensearch
Rollover initialization is idempotent, but rotation configuration has moved
toward the new schema; legacy rotation and cleaner flags are deprecated.
Use a disabled health check or bounded startup timeout only when the backend's
startup behavior requires it.
Apply custom_headers consistently, including host overrides, and ensure
signed writes have replayable bodies.
Prefer native trace summaries where supported; query falls back to aggregating
full traces otherwise.
Cassandra, memory, and ClickHouse
- Let Cassandra create its schema at session initialization, set replication as
needed, and configure a non-default compaction window explicitly.
- Memory searches cover OTLP fields, link attributes, and string-form error
filters.
- Treat ClickHouse support as experimental. Check the exact read, write,
metadata, SPM, dependency, TTL, and TLS capabilities needed by the deployment.
Query and UI quick reference
- Use
GET /api/v3/trace-summaries for lightweight searches and
query.attributes on GET /api/v3/traces for attribute filtering.
- Use storage capabilities, not configuration guesses, to decide whether SPM
and the Monitor tab are available.
- Account for input sanitization of invalid UTF-8, empty services, negative
durations, and empty span names.
- Search results default to a sortable table. Monitor filters persist through
URL parameters, and trace views include aggregate logs, service filtering,
and a native flamegraph.
- AI-assisted features are opt-in with
ai.enabled: true. MCP tooling lives in
jaegerquery, is session-scoped, and may enforce tenancy and response limits.
Verification checklist
- Start each configured role and confirm component validation succeeds.
- Verify HTTP and gRPC query listeners use distinct, intended addresses.
- Ingest OTLP over every enabled protocol and confirm normalized spans are
searchable.
- Exercise trace retrieval, summary search, services, operations,
dependencies, and attribute filters against the selected backend.
- Test archive behavior through the unified factory rather than old archive
interfaces.
- Validate proxy-prefix routing from browser URL through API v3.
- Check storage authentication, forwarded headers, TLS, health timeout,
queueing, retries, and message-size limits.
- Reconcile dashboards, alerts, and scrape targets with current metric names,
namespaces, ports, and storage capabilities.
- Confirm the UI's browser, package-manager, theme, search, and optional AI
requirements match the deployment policy.
1---2name: jaeger-knowledge-patch3description: Jaeger4license: MIT5---678# Jaeger Knowledge Patch910Use this skill when migrating, configuring, extending, or operating Jaeger v2.11Start with the breaking-change checklist, then open the reference matching the12work at hand. Treat the project's manifests, configuration, generated APIs,13tests, and observed runtime behavior as authoritative.1415## Reference index1617| Reference | Topics |18| --- | --- |19| [runtime-configuration-and-migration.md](references/runtime-configuration-and-migration.md) | Unified runtime roles, Collector YAML, server configuration, named storage, base paths, source builds |20| [storage-backends-and-contracts.md](references/storage-backends-and-contracts.md) | Storage v2 and remote-storage contracts, memory, Badger, Cassandra, ClickHouse |21| [elasticsearch-and-opensearch.md](references/elasticsearch-and-opensearch.md) | Backend compatibility, mappings, aliases, rotation, lifecycle, authentication, summaries |22| [query-sampling-metrics-and-telemetry.md](references/query-sampling-metrics-and-telemetry.md) | Sampling, query APIs, normalization, Prometheus, SPM, internal telemetry |23| [operations-deployment-and-development.md](references/operations-deployment-and-development.md) | Pipelines, endpoints, Compose and Kubernetes assets, dashboards, platforms |24| [ui-and-integrations.md](references/ui-and-integrations.md) | Search, trace visualization, Monitor, themes, MCP, AI-assisted integrations |2526## Breaking-change checklist2728### Distribution and configuration2930- Deploy the unified `jaeger` binary or image. Do not expect a separately31 published `jaeger-agent`, v1 service image, or `v1` image tag.32- Express runtime roles as Collector-style YAML. V1 environment variables are33 not automatically translated into v2 settings.34- Review query HTTP/TLS settings and assign separate listen ports to the HTTP35 and gRPC query servers.36- Select exactly one backend type in each trace- or metric-storage37 configuration.38- Replace Kafka receiver `traces.topic` with list-valued `traces.topics`.39- Use Go 1.25.7 for source builds and pnpm for UI or embedded-UI builds.4041### Removed contracts and compatibility paths4243- Remove the `grpc-plugin` storage type and migrate off `storage/v1/grpc`.44- Replace archive-specific storage interfaces with the unified storage45 factories.46- Update remote-storage implementations for streaming `GetTraces`, direct OTLP47 stream elements, current request/response fields, and48 `FindTraceIDsRequest`.49- Remove the metric-query gRPC service and the metric-store minimum-step API50 from custom integrations.51- Remove `sampling.strategies.bugfix-5270`; its corrected behavior is stable.52- Update remote-sampling clients that parse the removed legacy response.53- Do not script around `jaeger features`; that command was reverted.54- Replace imports that moved under Go `internal` packages.5556### Storage and backend upgrades5758- Upgrade Cassandra 3.x and Elasticsearch 6 before upgrading Jaeger.59- Recheck Elasticsearch/OpenSearch TLS, mappings, aliases, authentication,60 rotation, trace-duration, and legacy trace-ID settings.61- Plan for materialized `span.kind` and `span.status`, populated62 `parentspanid`, and mapped OTLP scope/link fields.63- Treat API keys, bearer tokens, and SigV4 as mutually constrained64 authentication choices; let validation reject incompatible combinations.65- Update custom dependency writers to accept and propagate context.6667### Query and response behavior6869- Expect trace adjusters to modify clock skew, duplicates, identifiers, and70 attribute/event ordering in API results.71- Treat invalid OTLP payloads as HTTP 400 request failures.72- Expect empty service queries to return an empty collection.73- Accept camel-case API v3 parameters while migrating away from deprecated74 snake-case aliases and `query.num_traces`.75- Treat trace IDs as opaque strings; query endpoints may receive base6476 identifiers.77- Update dashboards and alerts for the corrected counter suffix behavior and78 the newer SPM namespace.7980## Runtime quick reference8182### Override Collector YAML8384Use environment interpolation for deploy-time values and `--set` for a dotted85path:8687```yaml88receivers:89 otlp:90 protocols:91 grpc:92 endpoint: "${env:JAEGER_LISTEN_HOST:-localhost}:4317"93```9495```sh96jaeger --config config.yaml \97 --set=receivers.otlp.protocols.grpc.endpoint=0.0.0.0:431798```99100This also works for paths in the built-in all-in-one configuration. Remember101that the OTLP receiver can listen on all interfaces; bind and filter it102deliberately.103104### Wire named storage105106Register backends once in `jaeger_storage`, select them for reads in107`jaeger_query`, and select the trace backend for writes in108`jaeger_storage_exporter`:109110```yaml111extensions:112 jaeger_storage:113 backends:114 trace_store:115 memory:116 max_traces: 100000117 jaeger_query:118 storage:119 traces: trace_store120121exporters:122 jaeger_storage_exporter:123 trace_storage: trace_store124```125126Configure exporter queueing and retries when storage failures or bursts must be127buffered.128129### Configure path-prefixed deployments130131`jaeger_query.base_path` describes the path Jaeger receives:132133```yaml134extensions:135 jaeger_query:136 base_path: /jaeger137```138139Leave it `/` if a reverse proxy strips the external prefix. Otherwise use the140forwarded prefix. The path must begin with `/` and must not contain `..` or141duplicate slashes. The UI can discover its browser prefix automatically, and142API v3 requests honor the prefix.143144### Configure remote sampling145146`adaptive_sampling` calculates service and endpoint probabilities.147`remote_sampling` serves a static file or adaptive store over HTTP or gRPC:148149```yaml150extensions:151 remote_sampling:152 adaptive:153 sampling_store: sampling_store154 initial_sampling_probability: 0.1155 http:156 grpc:157```158159Review inherited per-operation defaults and use a reload interval when a160strategy file must update without restart.161162## Storage quick reference163164### Remote storage165166- Use the v2 factory and streaming contracts for trace, service, operation,167 dependency, and trace-ID operations.168- Forward tenant or authentication headers only when the backend expects them.169- Set `max_recv_msg_size_mib` when the default gRPC receive limit is too small:170171 ```yaml172 max_recv_msg_size_mib: 64173 ```174175- Keep the remote-storage server's write path on the normal OTLP176 receiver/exporter pipeline.177178### Elasticsearch and OpenSearch179180- Use the mapping generator embedded in `jaeger`. Select OpenSearch output with:181182 ```sh183 jaeger esmapping-generator --backend opensearch184 ```185186- Rollover initialization is idempotent, but rotation configuration has moved187 toward the new schema; legacy rotation and cleaner flags are deprecated.188- Use a disabled health check or bounded startup timeout only when the backend's189 startup behavior requires it.190- Apply `custom_headers` consistently, including host overrides, and ensure191 signed writes have replayable bodies.192- Prefer native trace summaries where supported; query falls back to aggregating193 full traces otherwise.194195### Cassandra, memory, and ClickHouse196197- Let Cassandra create its schema at session initialization, set replication as198 needed, and configure a non-default compaction window explicitly.199- Memory searches cover OTLP fields, link attributes, and string-form error200 filters.201- Treat ClickHouse support as experimental. Check the exact read, write,202 metadata, SPM, dependency, TTL, and TLS capabilities needed by the deployment.203204## Query and UI quick reference205206- Use `GET /api/v3/trace-summaries` for lightweight searches and207 `query.attributes` on `GET /api/v3/traces` for attribute filtering.208- Use storage capabilities, not configuration guesses, to decide whether SPM209 and the Monitor tab are available.210- Account for input sanitization of invalid UTF-8, empty services, negative211 durations, and empty span names.212- Search results default to a sortable table. Monitor filters persist through213 URL parameters, and trace views include aggregate logs, service filtering,214 and a native flamegraph.215- AI-assisted features are opt-in with `ai.enabled: true`. MCP tooling lives in216 `jaegerquery`, is session-scoped, and may enforce tenancy and response limits.217218## Verification checklist2192201. Start each configured role and confirm component validation succeeds.2212. Verify HTTP and gRPC query listeners use distinct, intended addresses.2223. Ingest OTLP over every enabled protocol and confirm normalized spans are223 searchable.2244. Exercise trace retrieval, summary search, services, operations,225 dependencies, and attribute filters against the selected backend.2265. Test archive behavior through the unified factory rather than old archive227 interfaces.2286. Validate proxy-prefix routing from browser URL through API v3.2297. Check storage authentication, forwarded headers, TLS, health timeout,230 queueing, retries, and message-size limits.2318. Reconcile dashboards, alerts, and scrape targets with current metric names,232 namespaces, ports, and storage capabilities.2339. Confirm the UI's browser, package-manager, theme, search, and optional AI234 requirements match the deployment policy.