# Vss Search Index

> Search a video library with natural language via the VSS Pipeline Manager - upload a video (POST /videos), generate its embeddings (POST /videos/search-embeddings/{id}), then run a query (POST /search/query) with optional tag and time filters and read the ranked clip results. Use when the user says "search my videos", "find something in the videos", "when did X happen", or wants to ingest/index a video for search. Requires a search-capable deployment (--search, --dual, or --unified).

- Skill: `open-edge-platform/vss-search-index` (Agent Skill, multi-file: 14 files)
- Install (CLI): `npx skillmds@latest add open-edge-platform/vss-search-index`
- Raw SKILL.md: https://api.skillmd.com/api/skills/open-edge-platform/vss-search-index/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- License: Apache-2.0
- Author: open-edge-platform (https://skillmd.com/u/open-edge-platform)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/open-edge-platform/vss-search-index

---


<!--
SPDX-FileCopyrightText: (C) 2026 Intel Corporation
SPDX-License-Identifier: Apache-2.0
-->

# VSS Search

Natural-language search over the indexed video library. Call the documented API
yourself and report only observed responses. Endpoints use the nginx `/manager`
prefix.

Set `HOST=http://${HOST_IP:-localhost}:${APP_HOST_PORT:-12345}`.

## Answer contract when VSS is not reachable

The user may be away from the deployment, or `$HOST` may refuse connections. In
that case **do not stall and do not invent responses.** Answer with the exact
call sequence instead: full endpoint paths, request bodies / form fields, the
field each step carries over from the previous response, and the condition that
says a step is finished. State plainly that the commands were not executed.
Never end the answer by asking whether to run them.

## Environment setup (run first)

This skill drives the Video Search & Summarization app through its real source
files, so the VSS application must be present and you must run commands from its
app root. **Do this before anything else**, and it works whether or not the VSS
source is already in your workspace.

Run the bundled bootstrap. It resolves the app root in this order and prints it
as the only line on stdout:

1. **Walk up from the current directory** looking for a VSS app root - a
   directory carrying all three markers `setup.sh`, `docker/`, and
   `pipeline-manager/`.
2. **Ask git for the enclosing repository** (`git rev-parse --show-toplevel`) and
   check whether it holds `sample-applications/video-search-and-summarization`,
   or is itself a VSS app root. This is what makes your own clone - or a fork -
   work unchanged.
3. **Reuse a checkout a previous bootstrap already placed** in
   `${XDG_CACHE_HOME:-$HOME/.cache}/vss-src/edge-ai-libraries`.

If any of those hit, that checkout is **reused and NO clone is performed**. Only
when all three miss does it clone - and then only a **shallow (`--depth 1`),
single-branch, sparse** checkout of just
`sample-applications/video-search-and-summarization` from `main`:

```bash
# SKILL_DIR is THIS skill's own directory (shown to you when the skill loads);
# in-repo it is .github/skills/vss-search-index. Works the same if the skill is installed standalone.
SKILL_DIR=".github/skills/vss-search-index"
APP_ROOT="$(bash "$SKILL_DIR/scripts/vss-bootstrap.sh")"
cd "$APP_ROOT"
```

Every command below assumes the working directory is this `APP_ROOT`. To pull
from a fork/branch or reuse a specific checkout dir, override `VSS_REPO_URL`,
`VSS_REPO_BRANCH`, or `VSS_CLONE_DIR` before running it. The bootstrap refuses
to overwrite an existing non-VSS clone destination.

## Preconditions

Backend healthy and **search enabled** - probe first; if not, use the installed
`vss-troubleshoot` or `vss-deploy` skill by name:
```bash
curl -sf "$HOST/manager/health" >/dev/null && \
curl -s "$HOST/manager/app/features" | jq -e '(.search // .) == "FEATURE_ON"'
```

## 1. Upload a video (if not already ingested)

`POST /manager/videos` - `multipart/form-data`, field name **`video`**, optional
comma-separated `tags`. File must be a streamable MP4 (server rejects otherwise).
```bash
curl -s -X POST "$HOST/manager/videos" \
  -F "video=@/path/to/clip.mp4" \
  -F "tags=outdoor,daytime" | jq .
# → { "videoId": "<VIDEO_ID>" }
```
When the user says a video is **already uploaded** or **just uploaded**, list
videos first and match the exact real filename. Reuse that record's `videoId`;
do not search the local filesystem and upload another copy. Upload only when no
exact filename match exists and the user actually supplied a local file to
ingest.

The list response is an **object**
`{ "videos": [...] }`, **not a bare array**; `name` is a generated hash, so use
`url` / `dataStore.fileName` for the real filename:
```bash
curl -s "$HOST/manager/videos" | jq '.videos[] | {videoId, file: .dataStore.fileName}'
curl -s "$HOST/manager/videos/<VIDEO_ID>" | jq '.video'   # single record is wrapped under .video
```

## 2. Generate search embeddings

A video is **not searchable until embeddings exist**. Trigger them after upload
(or to retry a failed run):
```bash
curl -s -X POST "$HOST/manager/videos/search-embeddings/<VIDEO_ID>" | jq .
```
Wait for completion (re-check the video record) before querying.

## 3. Query

One-off query - `POST /manager/search/query`. **The response is an object
`{ "results": [ { "query_id", "results": [ … ] } ] }`** - wrapped, NOT a bare
array - so the ranked clips are at `.results[].results[]`:
```bash
curl -s -X POST "$HOST/manager/search/query" \
  -H 'Content-Type: application/json' \
  -d '{
    "query": "person wearing a hat",
    "tags": "indoor",
    "timeFilter": { "value": 7, "unit": "days" }
  }' | jq -r '.results[].results[]
      | "score=\(.metadata.relevance_score)  clip=\(.metadata.segment_start)-\(.metadata.segment_end)s  seek=\(.metadata.seek_timestamp)s  video_id=\(.metadata.video_id)"'
```
- `query` (required): natural language.
- `tags` (optional): comma-separated, intersected with the query.
- `timeFilter` (optional): **either** relative (`value` + `unit` =
  `minutes|hours|days|weeks`) **or** absolute (`start`/`end` ISO-8601). See
  [`references/search-request.md`](./references/search-request.md).

Each clip's `metadata` carries `relevance_score` (0..1; top hit can be exactly
`1`), `video_id`, `video_url`, `segment_start`/`segment_end`, `seek_timestamp`,
`tags`, and `video_metadata` (duration/fps). In search mode `page_content` is a
segment **locator** ("Video segment from Ns to Ms…"), not a caption.

**Filename is NOT in the result** - `metadata` has `video_id` but no `video` /
`file_name`. To show the clip's filename, join `video_id` against the video list
(`.videos[].dataStore.fileName`):
```bash
curl -s "$HOST/manager/videos" \
  | jq '[.videos[] | {key:.videoId, value:.dataStore.fileName}] | from_entries' > /tmp/idmap.json
curl -s -X POST "$HOST/manager/search/query" -H 'Content-Type: application/json' \
  -d '{ "query": "person wearing a hat" }' \
  | jq --slurpfile m /tmp/idmap.json -r '.results[].results[]
      | "score=\(.metadata.relevance_score)  file=\($m[0][.metadata.video_id] // "?")  clip=\(.metadata.segment_start)-\(.metadata.segment_end)s"'
```
Present top hits with their **filename** + clip window + seek time.

If a filtered query returns no results, report the empty result as valid. Then
inspect the Manager video list for the requested tags and indexing readiness,
explain only what the observed state supports, and give a Manager-based next
step that preserves the same filters. To merge missing tags into an existing
record, use the documented embedding operation with a body such as
`POST /manager/videos/search-embeddings/<VIDEO_ID>` plus
`{"tags":"indoor"}`, then rerun the same filtered query.
Never present an unfiltered hit as though it satisfied the requested filter.

## Final answer audit trail

Tool arguments may not be visible to the user or evaluator. The final answer
must therefore name the public Pipeline Manager operations used (method and
`/manager/...` path), the important request fields, the observed status or
response, and any carried identifier such as `videoId`. Include the bootstrap
outcome (resolved app root and whether an existing checkout was reused) plus the
observed health and feature-preflight result. For searches, identify
the exact query/filter payload and say that ranked hits came from
`POST /manager/search/query`; for filename joins, say that the mapping came from
`GET /manager/videos`. If an id cannot be resolved, label it unresolved rather
than inventing a filename.

## 4. Saved / managed queries (optional)

```bash
curl -s -X POST "$HOST/manager/search" -H 'Content-Type: application/json' \
  -d '{"query":"forklift"}' | jq .          # create a persistent query → queryId
curl -s "$HOST/manager/search/<QUERY_ID>" | jq .          # fetch results
curl -s -X POST "$HOST/manager/search/<QUERY_ID>/refetch" | jq .   # re-run
curl -s --request PATCH --json '{"watch":true}' \
  "$HOST/manager/search/<QUERY_ID>/watch" | jq .             # auto-refresh
curl -s "$HOST/manager/search/watched" | jq .
curl -s -X DELETE "$HOST/manager/search/<QUERY_ID>"
```