# Elasticsearch Reindex > Guide Elasticsearch reindex for performance: local and remote, slicing, throttling, task API. Use when copying or migrating indices, changing mappings, or transforming during reindex. - Skill: `gabrielmoreira/elasticsearch-reindex` (Agent Skill, multi-file: 5 files) - Install (CLI): `npx skillmds@latest add gabrielmoreira/elasticsearch-reindex` - Raw SKILL.md: https://api.skillmd.com/api/skills/gabrielmoreira/elasticsearch-reindex/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - Author: gabrielmoreira (https://skillmd.com/u/gabrielmoreira) - Updated: 2026-09-09 - Page: https://skillmd.com/skills/gabrielmoreira/elasticsearch-reindex --- # Elasticsearch Reindex Copy documents from source indices or data streams to a destination using `POST /_reindex`. An expert reindex workflow prepares the destination explicitly, chooses local versus remote execution, filters at the source when only a subset is needed, runs long copies asynchronously, tracks the task to completion, and verifies the destination document count before reporting results. ## Environment Configuration This skill executes Elasticsearch operations through the `elastic` CLI. If the [`elastic` CLI](https://github.com/elastic/cli#configuration) is not installed, tell the user what it is needed for. Do not guess credentials, call the HTTP API directly, or attempt other workarounds. This skill references operations in HTTP-shorthand form (e.g., `GET /`, `GET /_cat/indices`, `GET /{index}/_mapping`, `GET /{index}/_settings/index.mode`, `POST /_query`). The [Operations](#operations) table at the end of this document maps each shorthand to the equivalent `elastic` CLI command — always use the CLI rather than calling the HTTP API directly. ## Process 1. **Confirm connectivity and deployment type.** Call `GET /`. Read `build_flavor` and `version.number` to know whether shard, replica, and cluster-settings APIs are available (Serverless manages shards/replicas internally and blocks most `_cluster/*` APIs). The decision: continue only when the cluster is reachable. If the call fails, stop — do not guess endpoints or credentials. 2. **Decide local versus remote reindex.** Compare where the source and destination live. - **Same cluster** — use local reindex: `source.index` and `dest.index` only. Do **not** add `source.remote` when both indices are on the cluster you are connected to. - **Different cluster** — use reindex from remote: add `source.remote` with the remote cluster URL and credentials. Remote reindex does not support slicing; compensate with query-based partitioning (date ranges, term filters) across parallel requests. Confirm the remote host is allowlisted on Self-Managed / ECH (`reindex.remote.whitelist` in cluster config); Serverless manages allowlisting internally (ECH remotes only, Tech Preview). Data needed: source index name(s), destination index name, and whether they share a cluster. 3. **Inspect the source — never guess field names or counts.** Call `GET /{source}/_mapping` to ground field names and types. Call `GET /{source}/_count` (or `GET /_cat/count/{source}?h=count` on Self-Managed / ECH) to learn how many documents exist. The decision: **full copy** versus **filtered subset**. - Full copy — omit `source.query` (match-all behavior). - Filtered subset — add `source.query` with Query DSL. For time ranges, use a `range` filter on the timestamp field (commonly `@timestamp`), e.g. `"gte": "2025-01-01", "lt": "2025-02-01"` for January 2025. Do **not** run a full-index copy when the user asked for a date range or other filter. Data needed: the user's filter criteria and the mapping-confirmed field names. 4. **Prepare the destination index before copying.** `_reindex` does **not** copy mappings, shard counts, or analyzers. Create the destination with explicit settings and mappings derived from the source mapping via `PUT /{dest}`. - On Self-Managed / ECH: set `number_of_replicas: 0` and `refresh_interval: "-1"` on the destination during the copy for write throughput; restore production values afterward with `PUT /{dest}/_settings`. - On Serverless: omit `number_of_shards` and `number_of_replicas` (managed by Elastic); you may set `refresh_interval: "-1"` during the copy. - For **data stream** destinations: ensure an index template with `data_stream: {}` exists, create the data stream, and set `dest.op_type` to `"create"` (append-only). The decision: create/prepare the target rather than relying on auto-creation with dynamic mapping. Wrong or missing mappings cause partial failures or silent type coercion. Data needed: destination name, corrected or compatible mappings, and deployment-specific settings constraints. 5. **Build and submit the reindex request.** Call `POST /_reindex?wait_for_completion=false` for any copy that may take more than a few seconds or when the user says the index is large — the response returns a **task id** immediately instead of blocking. Request body essentials: - `source.index` — source index or data stream (correct name, not reversed with `dest.index`). - `dest.index` — prepared destination from step 4. - `source.query` — include only when step 3 chose a filtered subset. - `conflicts: "proceed"` — when retrying a partially complete reindex. - Optional tuning: `source.size` (batch size), `requests_per_second` (throttle), `slices=auto` on local reindex only (parallelize per primary shard — never for remote), `scroll` (increase keep-alive on slow clusters), `max_docs` (test runs), `script` (transform), `dest.pipeline` (ingest enrichment). Example filtered subset (January 2025 only): ```json { "source": { "index": "eval-reindex-src", "query": { "range": { "@timestamp": { "gte": "2025-01-01", "lt": "2025-02-01" } } } }, "dest": { "index": "eval-reindex-jan" } } ``` Do **not** reach for `_split`, `_shrink`, or snapshot/restore when the task is a filtered subset copy or a straight document migration — those APIs solve different problems. 6. **Track the task to completion.** Store the task id from the reindex response. Poll `GET /_tasks/{task_id}` until `completed` is `true`. Read `status.total`, `status.created`, and `response.failures`. On Self-Managed / ECH you may also list active reindex tasks with `GET /_tasks?actions=*reindex&detailed`; on Serverless, query by task id only (list/cancel are not available). Adjust throttling mid-flight with `POST /_reindex/{task_id}/_rethrottle?requests_per_second=N` without canceling. 7. **Verify and report the destination count.** Call `GET /{dest}/_count` (works on all deployment types). On Self-Managed / ECH you may also use `GET /_cat/count/{dest}?h=count`. Compare source filter expectations to the destination count. Report the **exact** count from the destination — do not estimate or guess. After a successful full copy, restore production settings on the destination with `PUT /{dest}/_settings` (replicas and refresh interval on Self-Managed / ECH; refresh interval only on Serverless). ## Deployment constraints | Capability | Self-Managed / ECH | Serverless | | --------------------------- | ------------------ | ----------------------------------------- | | Local reindex | Full support | Full support | | Reindex from remote | Full support | Tech Preview — ECH remotes only | | `number_of_shards/replicas` | User-configurable | Managed — omit on index creation | | `slices=auto` (local only) | Supported | Supported for local reindex | | `GET /_cat/count/{index}` | Supported | Not available — use `GET /{index}/_count` | | `GET /_tasks` (list/cancel) | Full | Get by task id only | | `PUT /_cluster/settings` | Supported | Blocked | | `_split` / `_shrink` | Supported | Not available | ## Consider alternatives first - **Runtime fields** — fix field-type mismatches or add computed fields without reindexing when stored values need not change. - **Aliases** — redirect queries transparently; combine with reindex for zero-downtime mapping changes. - **Snapshot and restore** (Self-Managed / ECH) — faster whole-index transfer when no transformation is needed. See the decision tree in [references/patterns.md](references/patterns.md#decision-tree-do-i-need-to-reindex). ## Reference material - [API parameter reference](references/api-reference.md) — full `POST /_reindex` body and query parameters - [Multi-step patterns](references/patterns.md) — mapping changes, remote migration, merge, ingest pipeline, performance - [Tuning](references/tuning.md) — batch size (`source.size`), timestamps, versioning - [Troubleshooting](references/troubleshooting.md) — mapping conflicts, scroll timeouts, count mismatches ## Examples **"Copy `logs-2024` into a new index with a corrected mapping"** — create the destination first, then reindex: ```json POST /_reindex { "source": { "index": "logs-2024" }, "dest": { "index": "logs-2024-v2" } } ``` **"Reindex a large index in parallel and throttle it"** — slice automatically and cap the request rate: ```json POST /_reindex?slices=auto&requests_per_second=2000 { "source": { "index": "events" }, "dest": { "index": "events-v2" } } ``` **"Migrate only recent documents"** — filter the source with a query: ```json POST /_reindex { "source": { "index": "metrics", "query": { "range": { "@timestamp": { "gte": "now-30d" } } } }, "dest": { "index": "metrics-recent" } } ``` ## Guidelines - **Confirm deployment type first.** Call `GET /` and read `build_flavor`; shard, replica, cluster-settings, and task APIs differ between Self-Managed / ECH and Serverless (see Deployment constraints). - **Prefer an alternative when it fits.** Runtime fields, aliases, or snapshot-and-restore often avoid a full reindex. - **Tune the destination for the copy.** On Self-Managed / ECH set `number_of_replicas: 0` and `refresh_interval: "-1"` during the copy, then restore production settings afterward; on Serverless these are managed. - **Parallelize large copies.** Use `slices=auto` for local reindex and throttle with `requests_per_second` to protect the cluster. - **Run big jobs asynchronously.** Submit with `wait_for_completion=false` and poll the task instead of blocking. - **Verify by count.** Compare the source filter expectation to the exact destination `GET /{dest}/_count` — never estimate. ## Operations | HTTP API (shorthand) | `elastic` CLI command | | ------------------------------------------------------------ | ------------------------------------------------------------------------------------- | | `GET /` | `elastic es info` | | `GET /{index}/_mapping` | `elastic es indices get-mapping --index ''` | | `GET /{index}/_count` | `elastic es count --index ''` | | `GET /_cat/count/{index}?h=count` | `elastic es cat count --index '' --h count` | | `PUT /{index}` | `elastic es indices create --index '' --mappings '' --settings ''` | | `PUT /{index}/_settings` | `elastic es indices put-settings --index '' --settings ''` | | `POST /_reindex?wait_for_completion=false` | `elastic es reindex --wait-for-completion false --source '' --dest ''` | | `GET /_tasks/{task_id}` | `elastic es tasks get --task-id ''` | | `GET /_tasks?actions=*reindex&detailed` | `elastic es tasks list --actions '*reindex' --detailed` | | `POST /_tasks/{task_id}/_cancel` | `elastic es tasks cancel --task-id ''` | | `POST /_reindex/{task_id}/_rethrottle?requests_per_second=N` | `elastic es reindex-rethrottle --task-id '' --requests-per-second ` |