Elasticsearch Knowledge Patch

Elasticsearch Knowledge Patch

Use this skill when implementing, upgrading, or operating Elasticsearch and the answer may depend on recent API, mapping, query-language, inference, vector-search, lifecycle, security, or platform behavior. Start with the breaking-change and deprecation checks, then load every topic reference that matches the task.

Reference index

Triage breaking changes first

Before an upgrade or rollout, inspect breaking-changes.md in full. In particular:

• Treat ES|QL and EQL results as potentially partial unless the request or cluster setting explicitly requires completeness; always inspect the partial-result indicator.
• Quote an ES|QL remote index pattern as one unit or leave the entire pattern unquoted. Do not quote only the remote or index component.
• Expect timeouts to return HTTP 429, invalid ingest simulation to return HTTP 400, and byte-size parsing to accept at most two decimal places.
• Plan for vectors to be excluded from _source by default on new indices, while reindexing still carries vector values.
• Reconfigure discovery-ec2 for AWS SDK v2 and IMDSv2, remove unsupported credentials and protocol settings, and supply both access-key fields or neither.
• Add explicit force merging after ILM downsampling when the policy requires merged output; downsampling no longer force-merges by default.
• Remove APIs and options that no longer exist: the unfreeze endpoint, _knn_search, highlighting force_source, alias local, ingest user_agent.ecs, and the removed GeoIP fallback option.
• Remove retired settings including client.type, tracing.apm.*, and xpack.searchable.snapshot.allocate_on_rolling_restart.
• Validate TLS configurations without TLSv1.1 or TLS_RSA, and provide a bind password whenever LDAP or Active Directory uses a bind DN.

Migrate deprecated behavior

Before introducing or retaining a dependency, read deprecations-and-known-issues.md. High-priority migrations include:

• stop building operations around the ES|QL query log;
• replace the deprecated logs data-stream type with a supported stream type;
• omit aggregate_metric_double.default_metric from new mappings;
• replace ILM's deprecated max_size rollover condition;
• pass strict true or false values to plugin analysis settings and boolean system properties;
• remove indices.merge.scheduler.use_thread_pool;
• list ES|QL METADATA fields directly, without brackets;
• retire dependencies on the machine-learning flush API, the elser inference service, and Behavioral Analytics CRUD APIs.

Handle partial and asynchronous results deliberately

• Async ES|QL can return partial results on demand, formats async-get output, and reports cross-cluster metadata while work is in progress.
• ES|QL may return partial results by default. Set allow_partial_results=false per request or esql.query.allow_partial_results: false cluster-wide when correctness requires every shard or cluster.
• With skip_unavailable: true, remote runtime failures—including a missing index—become non-fatal and the remote cluster is marked skipped or partial.
• EQL defaults allow_partial_search_results to true; opt out for all-shards-required workflows.
• Async result retrieval can use return_intermediate_results, and async task status exposes keep_alive.

Use current ES|QL primitives

Read esql.md before composing recent pipelines.

• Use LOOKUP JOIN for lookup-index enrichment; it supports aliases, mixed numeric keys, multiple fields, remote input, and preview expression predicates, subject to the documented index-mode and remote-enrich rules.
• Use FORK to branch every input row and merge results with an _fork discriminator. Cross-cluster branches are supported.
• Use INLINE STATS for inline grouped analytics, LIMIT BY for per-group limits, and SET approximate for approximate analysis where preview status is acceptable.
• Use METRICS_INFO and TS_INFO after TS to discover metrics and series.
• Use PROMQL and the Prometheus-compatible plugin endpoints only with their technical-preview status in mind.
• Use TEXT_EMBEDDING, RERANK, vector functions, and dense-vector expressions with their current availability and usage constraints.
• Expect RENAME to process sequentially and text output to omit null columns.

Configure semantic and vector search consciously

Read search-and-vectors.md and inference.md together for semantic search.

• semantic_text is generally available and participates in match, sparse_vector, kNN, highlighting, multi-fields, and the text field family.
• New semantic_text fields use current inference, DiskBBQ, BFloat16, and chunking defaults; pin choices explicitly when stable behavior matters.
• DiskBBQ is designed for lower-memory search, accepts floating-point vectors, uses quantization, and should not be the automatic choice for low-dimensional data.
• Tune DiskBBQ with num_candidates or visit_percentage; configure its quantization level where supported.
• Use on_disk_rescore when raw vectors exceed RAM. Use direct I/O only after evaluating whether vectors fit in memory, because it can trade page-cache pressure for slower searches.
• Set oversample: 0 to bypass quantized-vector oversampling and rescoring.
• Account for the Enterprise-license requirement before creating new DiskBBQ indices migrated from affected earlier versions.

Build reliable data-stream and ingest flows

Read data-mappings-and-ingest.md.

• Enable failure stores through data-stream options for existing streams or template.data_stream_options.failure_store.enabled for new streams.
• Query rejected documents with the ::failures selector and remediate them with recover_failure_document.
• Do not assume a failure-store write is an error response: eligible new log, telemetry, and APM streams can return 201 Created with "failure_store": "used".
• Use include_source_on_error to control source exposure in parse errors.
• Use ingest simulation's effective mapping and merge behavior to validate pipelines; malformed processors now fail simulation with HTTP 400.
• Treat OTLP histograms as exponential_histogram by default and use the field's supported ES|QL aggregations.
• Check LogsDB, TSDB, synthetic-source, doc-values-skipper, sequence-number, and synthetic-ID defaults instead of carrying assumptions from older indices.

Plan lifecycle and repository changes

Read lifecycle-snapshots-and-storage.md before changing retention, migration, or backup policy.

• Use index.lifecycle.skip to exclude a single index from ILM.
• Account for automatic unfollow ordering before downsampling and for the leader time-series end-time wait.
• Choose the downsampling method explicitly and decide whether an ILM force merge is needed.
• Test S3 repository configuration after the AWS SDK v2 migration. Use IMDSv2, conditional-write protections, idle-timeout controls, and API-call timeouts as appropriate.
• Filter snapshots by state and use replicate_for when searchable-snapshot replication must persist for a defined duration.
• Review the known GCS ADC, S3 analysis, upgrade, merge, and low-disk defects before operating on affected versions.

Verify security and platform assumptions

Read security-cluster-and-operations.md for cluster or deployment work.

• Store secure settings through Elasticsearch's secure-settings mechanism, never in YAML.
• Treat Entitlements—not the Java SecurityManager—as the runtime permission system, and preserve exact filesystem path casing on Windows.
• Recheck connector roles, cross-cluster key trust and signing, SAML/JWT behavior, API-key cloning, and service-account-token availability.
• Expect the bundled JDK, Lucene baseline, container base, and FIPS defaults to affect plugins, images, and compliance validation.
• Inspect thread-pool utilization and queue latency, indexing pressure, document expansion, document-size limits, query logs, and watchdog hot threads when diagnosing load.

Check affected-version defects

Do not deploy a workaround without matching its exact affected version in deprecations-and-known-issues.md. That reference contains exact fixed versions and, where required, JVM policy or logger overrides for trained-model limits, GCS ADC, mixed GPU clusters, node-shutdown metadata, DiskBBQ licensing, TSDB/LogsDB merges, direct I/O, low-disk shard closure, ES|QL grouping, Windows entitlements, Active Directory, Watcher templates, and S3 repository analysis.