Opensearch Knowledge Patch

OpenSearch Knowledge Patch

Use this skill when designing, upgrading, configuring, or debugging OpenSearch clusters, OpenSearch Dashboards, search pipelines, vector and semantic search, ML Commons, agents, Query Insights, Alerting, Anomaly Detection, Security, SQL, or PPL.

How to use this skill

1. Determine the deployed OpenSearch and Dashboards versions from manifests or cluster APIs.
2. Identify the affected subsystem and read its topic reference below.
3. Apply guidance only when the referenced version is at or below the deployed version.
4. Treat disabled-by-default and experimental features as opt-in; verify their settings before relying on them.
5. For upgrades, inventory indexes, node roles, plugins, clients, settings, and saved Dashboards workflows before changing the cluster.
6. Prefer explicit engines, query options, and compatibility settings when behavior changed between releases.
7. Validate migrations on representative documents and queries, including security-filtered and multi-tenant paths.

Reference index

Upgrade blockers and breaking changes

Prepare for the 3.x runtime and index boundary

• Run the cluster on JDK 21 or later and account for the Lucene 10.1.0 baseline.
• Reindex every index created before 2.x, including system indexes, before a 3.0 upgrade.
• Verify 3.x artifacts with release@opensearch.org; the older signing key applies to 2.x artifacts.
• Replace removed system-index REST access with supported plugin APIs.
• Audit documents for JSON nesting beyond 1,000 levels, oversized property names, and _id values beyond 512 bytes.
• Remove _bulk batch_size, mmap.extensions, the transport-nio plugin, and the main-response compatibility override.

Review node and storage roles

• Configure coordinating-only nodes with an empty role list:

node.roles: []

• Assign warm rather than search to every node that serves searchable-snapshot shards.
• For remote-store indexes, use _scale and the search-only lifecycle only after verifying reader/writer separation and recovery procedures.

Update renamed and removed behavior

• Replace query-group API paths, response fields, and wlm.query_group settings with workload-group equivalents.
• Replace CatIndexTool with ListIndexTool.
• Use path_hierarchy, not the deprecated PathHierarchy tokenizer name.
• Replace Security whitelist settings with allowlist settings and grant cluster:monitor/shards where _cat/shards requires it.
• Move SQL pagination to Point in Time; legacy Scroll pagination and OpenDistro endpoints/settings are obsolete.
• Remove dependencies on the old Discover experience setting and DataGrid table.
• Reindex Romanian text after the analyzer's cedilla-to-comma Unicode normalization change.
• Revalidate Blake2b hash fixtures after the Security salt-handling correction.

Check platform and plugin dependencies

• OpenSearch 2.0 Alerting notification actions require notifications-core and notifications; Dashboards management also requires notificationsDashboards.
• Plan migrations away from retired or announced-for-deprecation operating-system baselines, including Ubuntu 20.04 and Amazon Linux 2 where applicable.
• Review Dashboards plugins against the Node.js 22 and Rspack transition; Node.js 20 is deprecated.

Search and vector quick reference

Make engine and rescore choices explicit

• The implicit k-NN engine changed to Faiss in 2.18. Explicitly choose Faiss, Lucene, or a supported legacy engine when index behavior must be stable.
• With Faiss cosine space, indexing normalizes vectors; use inner product for already-normalized vectors when equivalent scoring is desired without implicit normalization.
• A mapping backed by a trained vector artifact must not declare both its identifier and dimension.
• index.knn is immutable after index creation, and derived vector source cannot be enabled when it is false.
• New OnDisk 4x-compressed indexes rescore by default; specify rescore: false when preserving the prior behavior.
• A terminal remote-vector-build failure does not fall back to a CPU build.

Respect hybrid-query constraints

• Do not nest hybrid under compound queries such as function_score, constant_score, or script_score.
• Do not use dfs_query_then_fetch with a hybrid query.
• Invalid nested hybrid structures are rejected; test parent-join, collapse, and inner_hits combinations explicitly.
• Choose normalization deliberately: min-max supports lower and upper bounds, Z-score is available, and RRF supports rank-based fusion and custom weights.

Control vector payloads

• Derived vector source removes vectors from stored _source and reconstructs them on reads; validate supported flat, object, and single-level nested mappings.
• Use docvalue_fields when retrieving float, byte, or binary vectors without reindexing; the default representation is Base64 binary.
• Base64 vector ingestion avoids JSON arrays, and a search request processor can exclude vector fields from _source responses.

Validate semantic and sparse retrieval

• A neural sparse query cannot specify both an analyzer and a model identifier.
• Semantic fields expose chunking, analyzers, generated dense-vector parameters, ingest batching, prune strategies, embedding reuse, and sparse output format controls.
• SEISMIC supports sparse approximate retrieval, nested fields, explanation, and optional method parameters.
• Batch semantic highlighting of nested inner_hits requires request-level opt-in.

Query language and transport quick reference

Account for Calcite routing

• Calcite is the default PPL path. Unsupported commands may route to the v2 path, but general Calcite query failures do not fall back by default.
• query.size_limit limits final results, not intermediate processing.
• With plugins.ppl.syntax.legacy.preferred=false, join defaults to max=1.
• Zero or negative subsearch.maxout means unlimited; set an intentional positive ceiling when resource bounds matter.
• PPL final structs are maps, missing or null JSON_EXTRACT paths return null, and overflow to double infinity returns null.

Treat pagination and cancellation as protocol choices

• SQL pagination defaults to Point in Time.
• PPL accepts fetch_size and can be cancelled through _tasks/_cancel.
• The query-only unified V2 SQL path blocks DML and DDL.
• SQL cursor continuation remains confined to the original query indexes under fine-grained access control.

Gate optional transports

• Protobuf-over-gRPC bulk ingestion and search are production-ready, with encryption and Security authentication options.
• Apache Arrow Flight streaming remains disabled by default.
• Pull-based Kafka and Kinesis ingestion is production-ready only after its later lifecycle transition; configure warmup and adaptive shard selection.

Agents and ML quick reference

Select current agent protocols

• Agentic search and persistent agentic memory are production-ready; preserve memory identifiers and session metadata when building multi-turn experiences.
• Unified registration and conversational_v2 are production-ready and honor inferenceConfig.model_parameters.
• Use Streamable HTTP for MCP; SSE transport is deprecated for MCP even though separate prediction and agent streaming APIs may still use SSE.
• Treat AG-UI, retention policies, and other explicitly experimental capabilities as disabled unless enabled.

Validate connectors and guardrails

• Connector actions can use custom names plus PUT and DELETE, and headers can substitute per-request parameters.
• Enforce trusted endpoint patterns and private-IP/ReDoS protections on outbound connector paths.
• ModelGuardrail and LocalRegexGuardrail fail closed when evaluation fails; design error handling accordingly.
• Inline connectors need no name, but schema-declared strings must remain strings during validation.

Operations and security quick reference

Preserve observability and alert semantics

• Query Insights spans historical top-N, live queries, profiling, recommendations, identities, workloads, and remote export; apply authorization filters to its records.
• Alerting's temporary list-of-findings publication was reverted; consumers should expect individual findings.
• Multi-tenant Alerting disables unsupported email, findings, chained actions, scheduler-index, and other paths, returning 501 where specified.
• PPL Alerting assets and API routes changed during their transition; do not assume the experimental assets remain installed.

Apply access controls deliberately

• Scoped API keys carry their own cluster and index permissions and support expiration and synchronous revocation.
• plugins.security.dls.write_blocked can block all writes under document-level restrictions.
• Security configuration supports dynamic resource settings, versioned views and rollback, resource sharing, and explicit protected resources.
• Resource-access configuration uses resource-access-levels.yml; review notification prefixes and multi-tenancy settings during upgrade.
• Security plugin PUT and PATCH requests limit every text input to 256 characters.

Verification checklist

• Confirm cluster, Dashboards, plugin, Java, and index-creation versions.
• Run representative full-text, hybrid, vector, semantic, PPL, SQL, and gRPC requests.
• Compare scores, rescoring, collapse, inner_hits, pagination, and null behavior before and after upgrade.
• Exercise DLS, FLS, field masking, tenant isolation, API keys, JWT, certificate, and audit paths.
• Test Alerting, Anomaly Detection, Query Insights, workload groups, schedulers, and replication under failure.
• Verify experimental flags and disabled-by-default settings explicitly.
• Read all relevant topic references before shipping a migration or compatibility-sensitive change.