Elasticsearch Knowledge Patch

Elasticsearch Knowledge Patch

Use this skill when upgrading, configuring, querying, securing, or operating recent Elasticsearch clusters. Determine the exact server version, index mode, license, deployment type, and client behavior first. Then open the topic reference that matches the task; several defaults and APIs changed more than once.

Reference index

Upgrade checks that prevent breakage

Treat partial results as an explicit policy

ES|QL responses may be partial by default. Inspect is_partial; set allow_partial_results=false per request or esql.query.allow_partial_results: false cluster-wide when completeness is required. EQL likewise defaults allow_partial_search_results to true. With remote-cluster skip_unavailable: true, runtime failures, including a missing index, can produce a skipped or partial cluster instead of a fatal response.

Revalidate index patterns and removed request options

Quote an entire ES|QL remote pattern or none of it: both FROM "remote:index" and FROM remote:index are valid, while FROM remote:"index" is not. Parentheses require quoting. Remove these legacy uses:

• highlighting force_source;
• alias API local;
• the unfreeze endpoint and reads from frozen indices;
• the technical-preview _knn_search API;
• Watcher search types;
• user_agent.ecs and the removed GeoIP fallback option;
• metadata-field type, fields, copy_to, and boost definitions.

The _source.mode mapping attribute is a no-op. random_score without a field now uses _seq_no, and date_histogram rejects boolean values.

Account for changed defaults

• New indices exclude vectors from _source by default.
• LogsDB and TSDB text fields omit norms.
• Eligible logs-*-* data streams enable LogsDB by default.
• ES|QL and EQL permit partial results by default.
• Timeouts return HTTP 429, and byte sizes accept at most two decimal places.
• JDK 24 removes TLS_RSA ciphers and TLSv1.1 from defaults.
• A bind DN without its bind password prevents LDAP or Active Directory startup.
• API request secret_parameters cannot be overridden in 9.3.8 and 9.4.4.

Make downsampling policy explicit

Starting in 9.4.0, ILM downsampling no longer force-merges its result by default. Add a force-merge action or set force_merge_index: true when the old behavior is required. OTLP histograms now map to exponential_histogram, and normalized keyword fields use native synthetic source.

Rework EC2 discovery configuration

discovery-ec2 uses AWS SDK v2 and requires IMDSv2. It ignores discovery.ec2.protocol; put http:// in discovery.ec2.endpoint when needed. Remove aws.secretKey and com.amazonaws.sdk.ec2MetadataServiceEndpointOverride. Configure both discovery.ec2.access_key and discovery.ec2.secret_key, or neither.

Review platform and allocation removals

Remove cluster.routing.allocation.disk.watermark.enable_for_single_data_node. Do not expect cluster state in /_cluster/reroute responses. Also remove client.type, tracing.apm.*, and xpack.searchable.snapshot.allocate_on_rolling_restart. Machine learning is disabled on macOS x86_64, and the old data_frame_transforms roles are gone.

Deprecation priorities

• Do not build new automation around ES|QL query logging; its use is deprecated from 9.4.2.
• Avoid the deprecated logs data-stream type and aggregate_metric_double.default_metric mapping parameter.
• Replace ILM max_size rollover conditions and remove indices.merge.scheduler.use_thread_pool.
• Supply strict true or false values in plugin analysis settings and boolean system properties.
• In ES|QL, write METADATA _id, _index without brackets.
• Migrate from the machine-learning flush API, the elser inference service, and Behavioral Analytics CRUD APIs.
• Recheck authorization assumptions around the built-in reporting_user role, which now derives authorization from reserved Kibana privileges.

Known-version hazards

Upgrade rather than preserve faulty behavior

• Upgrade 9.3.6 to 9.3.7 when valid trained-model requests hit overly strict field limits.
• Upgrade repository-gcs users of Application Default Credentials from 9.2.8 to 9.2.9 or from 9.3.3 to 9.3.4; the compatibility reference contains the temporary entitlement-policy values.
• Upgrade mixed-GPU clusters from 9.3.1 to 9.3.2 to stop repeated usage serialization warnings.
• Do not upgrade directly from 9.1.10 to 9.2.4; use 9.2.5 or later.
• Upgrade shrunk TSDB or LogsDB indices from 9.1.0/9.1.1 to 9.1.2 before relying on force merge.
• On 9.1.0, disable vector.rescoring.directio when in-memory bbq_hnsw searches regress; remove the override in 9.1.1.
• On 9.0.3, keep indices.merge.disk.check_interval at 0s to avoid shard-close hangs under low disk.

Query correctness and old-cluster history

Two-key ES|QL STATS grouping can be wrong before 8.17.9, 8.18.7, and 9.0.4 when the first keyword has more than 65,000 values. Upgrade, put the lower-cardinality key first, or filter cardinality. Match Windows filesystem casing exactly under 9.0 entitlements. A cluster that once ran 7.10.0–7.12.1 may need stale Watcher templates deleted before Watcher can start.

High-value capabilities

Failure-store workflow

Enable a data-stream failure store through PUT _data_stream/<name>/_options or template.data_stream_options.failure_store.enabled. Query failed documents with ::failures. New log, OTel, and APM streams may enable failure stores by default; use recover_failure_document to remediate documents. Failure stores have dedicated lifecycle controls and can participate in cross-cluster search.

ES|QL joins, branches, views, and metrics

Use LOOKUP JOIN to enrich rows from lookup indices; later behavior supports aliases, mixed numeric keys, multiple join fields, remote input, and expression predicates. FORK runs rows through branches and adds _fork. Views expose reusable pipelines as virtual indices, but cannot be queried with document- or field-level security. PROMQL, METRICS_INFO, and TS_INFO support metrics workflows; verify technical-preview status before making compatibility promises.

Time-series storage and downsampling

Use exponential_histogram for OTel histograms and the T-Digest field type for time-series metrics. Doc-values skippers default on for TSDB and LogsDB but are off generally. Synthetic TSDB IDs avoid indexing _id; new TSDB indices disable sequence numbers. The aggregate downsampling method preserves counter resets, while last_value keeps its storage-oriented behavior.

Vector search and semantic defaults

semantic_text integrates mapping and inference and supports standard text, sparse-vector, kNN, highlighting, multi-fields, and configurable chunking. Recent new fields default to DiskBBQ, BFloat16, and the Jina v5 inference configuration. DiskBBQ is designed for lower memory use; tune kNN with num_candidates or visit_percentage, and avoid it for low-dimensional vectors. Check licensing when carrying bbq_disk indices forward from 9.2.

Retriever and reranking choices

Choose RRF for rank fusion, linear retrievers for weighted normalized scores, generic rescorers for request rescoring, text_similarity_reranker for semantic reranking, and MMR for diversification. Search also supports pinned and scripted retrievers, multi-vector rank_vectors, quantized-vector rescoring, and contextual chunk_rescorer snippets.

Working checklist

1. Read the server version from the cluster, not just a client dependency.
2. Identify index mode: standard, LogsDB, TSDB, lookup, failure store, or synthetic source.
3. Check whether the feature is generally available, technical preview, or experimental in the applicable release.
4. For distributed queries, decide whether partial results are acceptable and inspect per-cluster metadata.
5. For mapping changes, verify defaults on newly created indices separately from existing mappings.
6. For inference or vector work, verify endpoint task, chunking, vector element type, quantization, rescoring, memory, and license.
7. For lifecycle work, distinguish main backing indices from failure-store indices and make force merge explicit.
8. For repository or security changes, test credentials, secure settings, entitlements, TLS, and cloud SDK migration in a non-production cluster.
9. Before an affected upgrade, read the compatibility reference and choose a fixed target release instead of carrying a temporary workaround forward.