# Hotdata Search > Use this skill when the user wants full-text search, BM25 keyword search, vector similarity search, semantic search, embeddings, or retrieval indexes in Hotdata. Activate for "hotdata search", "BM25", "full-text", "vector search", "semantic search", "similarity", "embedding", "embedding provider", "create an index" (bm25 or vector), "list indexes" for search, or SQL using bm25_search or vector_distance. Do not load for general SQL analytics (aggregations, GROUP BY) or geospatial work — use hotdata-analytics or hotdata-geospatial instead. Requires the core hotdata skill for auth and workspace basics. - Skill: `hotdata-dev/hotdata-search` (Agent Skill, multi-file: 2 files) - Install (CLI): `npx skillmds@latest add hotdata-dev/hotdata-search` - Raw SKILL.md: https://api.skillmd.com/api/skills/hotdata-dev/hotdata-search/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Data & Analytics - Author: hotdata-dev (https://skillmd.com/u/hotdata-dev) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/hotdata-dev/hotdata-search --- # Hotdata Search Skill Retrieval workloads in Hotdata: **BM25 full-text**, **vector similarity**, and the **indexes** and **embedding providers** that power them. **Prerequisites:** Authenticate, set a workspace, and set an active database (`hotdata databases use `) — see the **`hotdata`** skill. Use fully qualified table names: `..`. **Related sub-skills** (bundled alongside this one — `Read` on demand): **`hotdata-analytics`** ([`../analytics/SKILL.md`](../analytics/SKILL.md) — OLAP SQL, query history, materialized chains), **`hotdata-geospatial`** ([`../geospatial/SKILL.md`](../geospatial/SKILL.md) — PostGIS-style functions). --- ## Search CLI Both run server-side. The search action addresses an index **by name** (`--index`, alias `--in`); the index's type, column, and provider come from the index itself. ```bash # BM25 / text (requires a text index; address it by name) hotdata search "" --index \ [--database ] [--select ] [--limit ] [--workspace-id ] [--output table|json|csv] # Vector (requires a vector index; server auto-embeds the query text) hotdata search "" --index \ [--database ] [--select ] [--limit ] [--workspace-id ] [--output table|json|csv] # --in is an accepted alias for --index hotdata search "" --in ``` | Type | Behavior | |------|----------| | **`bm25`** | Server generates `bm25_search(table, col, 'text')`. Results sort by score (descending). | | **`vector`** | Pass plain-text query; name the **source text column** (e.g. `title`). Server embeds using the same provider/metric/dimensions as the index. SQL uses `vector_distance(col, 'text')`. Results sort by distance (ascending). | - **Index name:** the index carries its own type, column, and provider — you only name the index. Use `search list` to see available index names. - **`--select`:** defaults to the table's own columns. An auto-embed vector index materialises a `{column}_embedding` column on the table, and a search leaves it out — it is a 1536-float list in every row, tens of kilobytes per search nobody asked for. Ask for it back with `--select '*'`, or name it (`--select 'id,body_embedding'`); pair either with `--output json`, since `--output table` abbreviates long lists for display. - **Custom embedding model, raw query vector, or no vector index?** Use `hotdata query` directly (e.g. `cosine_distance(col, [])`) — `search` only auto-embeds the query text via the index's own provider. - **Before search:** create the right index (`search create --type text` or `--type vector`). See [references/INDEXES.md](references/INDEXES.md). - Default `--limit` is 10. - **Database:** the search commands resolve the index in the **active** database (`hotdata databases use `). Pass `-d/--database ` to target a different database explicitly — it is required when no active database is set. The same `-d/--database` works on `search show` and `search remove`. - **Active database:** with `hotdata databases use `, an index created with a `schema.table` `--from` resolves the active database's catalog automatically. Or create it with the full `catalog.schema.table` form. Do **not** use the internal `__db_` label or raw catalog ID prefix — `bm25_search`/`vector_distance` resolve a catalog attached to the active database, so an `__db_…` or `conn…` prefix errors with *catalog … is not attached*. --- ## Indexes (text and vector) Indexes are an **instant-database** concept. Create names the index (positional) and attaches to a table on an instant database via `--from` — `catalog.schema.table` (the instant database's catalog), or `schema.table` with an active database set. A plain connection catalog is rejected. `list` narrows to the **active database** when one is set; without one it scans the whole workspace. `show`/`remove` resolve the index by name in the active database (or `--database `). ```bash # List — active-database scope when a DB is set, else whole-workspace scan hotdata search list [--workspace-id ] [--output table|json|yaml] # Create — index name is positional; --from is an instant database's table hotdata search create --type text|vector --from \ --column \ [--metric l2|cosine|dot] [--async] \ [--provider ] [--dimensions ] [--output-column ] [--description ] # Show — by index name, in the active database (or -d/--database ) hotdata search show [-d ] [--output table|json|yaml] # Remove — by index name, in the active database (or -d/--database ) hotdata search remove [-d ] ``` - **`--type` is required** on create: `text` (BM25; one or more text columns, comma-separated in `--column`) or `vector` (exactly one column; often embeddings or auto-embedded text). (`sorted` is also a valid `--type`, covered in **`hotdata-analytics`** — [`../analytics/SKILL.md`](../analytics/SKILL.md).) - **`sorted`** indexes (range/equality for OLAP filters) are documented in **`hotdata-analytics`** ([`../analytics/SKILL.md`](../analytics/SKILL.md)) — this skill focuses on retrieval types. - **`--async`:** poll with `hotdata jobs ` (see **`hotdata`** skill **Jobs**). - **Auto-embedding:** `--type vector` on a **text** column generates embeddings server-side. Optional `--provider`; default output column `{column}_embedding` (override with `--output-column`). The generated column becomes part of the table, so it shows up in `information_schema.columns` and in a hand-written `SELECT *` — `hotdata search` leaves it out of its own projection (see `--select` above). Full workflow (gather workload → compare existing → create → verify): [references/INDEXES.md](references/INDEXES.md). --- ## Embedding providers ```bash hotdata search embeddings list [--workspace-id ] [--output table|json|yaml] hotdata search embeddings show [--workspace-id ] [--output table|json|yaml] hotdata search embeddings add --name --provider-type service|local \ [--config ''] [--provider-api-key | --secret-name ] [--workspace-id ] [--output table|json|yaml] hotdata search embeddings update [--name ] [--config ''] [--provider-api-key | --secret-name ] [--workspace-id ] [--output table|json|yaml] hotdata search embeddings remove [--workspace-id ] ``` - System providers (e.g. `sys_emb_openai`) are pre-configured; use `list` for IDs to pass to `--provider`. - `--provider-api-key` is the **embedding service** key (not Hotdata `--api-key`). `--secret-name` references an existing secret. --- ## Quick workflow 1. `hotdata databases use ` — set an active database, then `hotdata databases tables list` to confirm column types. 2. `hotdata search list` — avoid duplicate indexes (scoped to active DB automatically). 3. `hotdata search create --type text|vector --from --column ` (add `--async` if large). 4. `hotdata search "..." --index ` — address the index you created by name. 5. Record what exists in **context:DATAMODEL** (core skill) when the workspace should remember index choices.