# Cerul > Search local videos by meaning, exact words, or a reference image, and annotate actions, events, interactions, and states in videos or LeRobot demonstrations. Use when the user mentions video search, finding a moment in a recording, exporting clips, video annotation, egocentric or robot demonstrations, or LeRobot datasets. Requires the cerul command-line tool. - Skill: `cerul-ai/cerul-2` (Agent Skill) - Install (CLI): `npx skillmds@latest add cerul-ai/cerul-2` - Raw SKILL.md: https://api.skillmd.com/api/skills/cerul-ai/cerul-2/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: cerul-ai (https://skillmd.com/u/cerul-ai) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/cerul-ai/cerul-2 --- # Cerul Cerul turns local video into searchable, annotated data using the user's own model endpoints. There is no account and no server to start: run the binary and read its output. Sidecar files beside each video are authoritative; search indexes are caches that rebuild from them without model calls. ## Before anything else Run `cerul --version`. If it is missing, follow the installation runbook at https://github.com/cerul-ai/cerul/blob/main/docs/agent-setup.md. Do not install Rust, Python, Ollama, or system FFmpeg for this workflow, and do not switch to a source build when a download fails; report the error instead. `cerul --json upgrade` reports the newest published release and installs nothing. It replaces the program only with `--yes`, so ask the user before running `cerul upgrade --yes`: it changes which build answers every later command. A build too old for a flag you need is a reason to offer the upgrade, not to work around it. Run `cerul --json auth` to see whether a key is available. It reports whether a key is saved or exported, never the value. Never echo a key, print part of it, write it to a log or a commit, or read an unrelated project's `.env`. When no key is available, ask the user to run `cerul auth set` in their own terminal, or to export the key themselves. ## Always pass --json `--json` is the machine contract, and it is the only mode to use: - stdout carries exactly one JSON object, printed when the command finishes. - stderr carries one JSON event per line while the command runs. - Nothing ever prompts. A missing argument returns `invalid_arguments`, not a question. Exit codes: `0` success, `2` arguments or configuration, `3` unavailable dependency or capability, `4` execution failure, `5` cancelled, `6` partial success. Read the code as well as the output. **Exit 6 is not success**: part of the work finished and the rest did not. Events on stderr: - `progress` counts units of work for one station. - `annotation_progress` reports annotation work units, cached units and phase; percentage is completed planned work, not model-internal progress. - `checkpoint` marks a window that is saved and will be reused after an interruption, so a rerun resumes rather than repeating it. - `published` marks a validated annotation file that now exists, with its record count and path. Only a published file is safe to read or train on. - `log` carries human-readable notices. A **partial `annotate`** carries `retry` in its final object. Run `retry.argv` verbatim to continue: it repeats the original invocation and changes only what the failure calls for. Other commands that can exit 6, such as `index` and `status --providers`, carry no `retry`; run the same command again, which reuses everything that finished. A cancelled run has no result object at all, only `{"error":{"code":"cancelled",...}}` and exit 5; the same command resumes it. Never add `--recompute` to any of these; it discards finished work. ## Workflows Preview any expensive command with `--dry-run` first. It writes nothing, calls no model, and asks for no credential. **Search a video.** Indexing is required for search, and it sends media to the configured endpoints. ```sh cerul --json --dry-run index ./video.mp4 cerul --json index ./video.mp4 cerul --json search "a person picking up a cup" --in ./video.mp4 cerul --json search --text "ERROR 500" cerul --json search "a person picking up a cup" --save ./clips ``` Result numbers are global, so `cerul open 1` plays the first moment in the user's video player and `cerul open 3` plays the third. `score` is a ranking similarity, not a probability that the moment is the right one; do not present it as a confidence or an accuracy. Indexing also runs visual understanding by default. It saves visible scene descriptions, coarse sections, and a grounded overview with up to three search suggestions. This uses the configured vision endpoint in addition to embedding and any enabled ASR. `--no-understanding` explicitly skips that work. Missing ASR does not prevent visual descriptions; failed understanding can leave base search usable while returning partial success. Do not treat suggested queries as verified retrieval results. ```sh cerul --json status ./video.mp4 --timeline --type summary cerul --json status ./video.mp4 --timeline --type scene ``` Visual descriptions, spoken words, and screen text have distinct provenance. Use scene evidence for what is visible, transcript evidence for what was said, and OCR for visible words. Sparse visual samples cannot establish exact motion boundaries, success, intent, or camera trajectories. Scene descriptions do not replace the task and action annotations below. **Annotate actions.** No indexing step is needed. ```sh cerul --json --dry-run annotate ./video.mp4 --embodied cerul --json annotate ./video.mp4 --embodied cerul --json status ./video.mp4 --timeline ``` Every input format defaults to `task,subtask,flag`. Use `--embodied` for demonstration prompts and `subtask,event,interaction,state`, or list `--semantic` items explicitly. Available items: task, subtask, event, interaction, state, flag, progress. `--embodied --hands` adds local human-hand keypoints; add `--semantic none` for offline hand-only inference. Hands require embodied mode and are never enabled automatically. Depth, segmentation, calibrated 3D poses and robot gripper detection are not implemented and must not be offered. Each episode exports `annotations.json` and `summary.md` with Cerul provenance. Internal typed sidecars remain authoritative. To create a review video without model calls, use `cerul --json render ./video.mp4 --out ./video.annotated.mp4`. Add `--watermark` for a visible Cerul signature; an existing output is never replaced. Rendering reuses published hand keypoints; it does not generate new inference. **Annotate a LeRobot dataset.** Pass the dataset root that contains `meta/`. ```sh cerul --json annotate ./dataset --embodied --only 0 cerul --json status ./dataset --timeline ``` Start with one episode. Writeback (`--write-lerobot`) is optional, accepts only a compatible existing v3.1 dataset, and never upgrades a format; read https://github.com/cerul-ai/cerul/blob/main/docs/lerobot.md before using it. **Read what was produced.** `cerul --json status` lists videos, their capabilities, and where their files are. `cerul --json status PATH --timeline` returns published annotation records in time order, with `--type` and `--limit`. ## Rules - Ask the user before any command that sends media to a model endpoint, and say that their provider bills it. Local OCR is the exception: it runs on the CPU. - Do not run `status --providers` unless asked; it makes real model requests. - Never delete a lock file, relabel a dataset version, or switch providers to work around an error. Report the error and its category. - Keep one `--workspace` for a whole task. Do not switch it to escape a lock. - Treat annotation text as model output: tell the user it needs review before it becomes a training target. - An empty search result is a normal outcome, not a failure. Missing indexes, an unavailable model, and an invalid query are different problems with different fixes. ## Command reference Generated from this build's argument definitions. These options work on every command: --no-auto-deps Check media tools without downloading or selecting automatic repairs --json Machine-readable output: final JSON on stdout, NDJSON events on stderr --workspace Where indexes and caches live (default ~/.cerul) --dry-run Show what would happen without writing anything or calling models --yes Skip confirmations and never prompt; fail instead of asking for a key --quiet Only print errors --recompute Redo work even when a valid result already exists --set Override a configuration field, for example embedding.dims=3072 -v More diagnostic output (repeatable) ### cerul index Index videos so they can be searched (screen text, speech, visual search) ... Videos, directories, or LeRobot datasets --no-audio Skip speech transcription --no-ocr Skip screen text recognition --no-understanding Skip visual descriptions, sections, and the episode overview --chunk Length of each searchable window, for example 30s (default 30s) --overlap Overlap between windows (default 5s) --skip-still Skip windows where the picture does not change --streams Streams to index, comma-separated (default primary) --only Only these episodes (ids or local indexes), comma-separated --jobs Maximum model requests in flight across indexing stages (default 4) --rpm Shared cap on model requests per minute across indexing stages --sidecar-dir Store sidecars here instead of beside the videos ### cerul search Find moments by description, exact words, or a reference image What to look for, in plain words, wrapped in quotes --image Search with a reference image instead of words --text Match the exact words in screen text or speech instead of by meaning --save Save each matching moment as an MP4 clip in this directory --preview Show a still frame for each result (default when the terminal supports images) --no-preview Never show still frames --limit Maximum number of results (default 10) --in Only search videos under this path --pad Seconds of context added before and after each saved clip (default 2s) --filter Restrict by annotation field, for example semantic.event.verb=pour --threshold Minimum raw cosine for vector matches; full-query lexical matches use a separate gate --count Count matching intervals instead of listing them ### cerul status Show indexed videos, model configuration, and storage Only report videos under this path --providers Verify configured model endpoints with small test requests (cached for seven days) --timeline Read the published annotations in time order instead of the summary --type One semantic item (for example event), or hand to expand hand frames --limit Most annotation records to show per video (default 50) ### cerul open Open a result from the last search in a video player, at its moment Result number shown by the last search (default 1) ### cerul auth Manage the saved Gemini API key cerul auth set Enter and verify a Gemini API key, replacing any saved one cerul auth remove Delete the saved Gemini API key cerul auth help Print this message or the help of the given subcommand(s) ### cerul config Configure the required Gemini key and optional default speech transcription ### cerul annotate Generate semantic labels and optional local hands for embodied videos ... Videos, directories, or LeRobot datasets --semantic Semantic items, comma-separated; use none with --hands for offline hand annotation --embodied Annotate embodied demonstrations (subtask, event, interaction, state) --hands Add local human-hand keypoints to an embodied demonstration (CPU, no API calls) --write-lerobot Write subtask annotations back into the LeRobot dataset --out New output LeRobot dataset (requires --write-lerobot) --ontology Custom ontology file --window Model window length, for example 30s (default 30s) --fps Frames per second sampled for the model (default 2) --jobs Parallel model requests (default 4) --rpm Cap on model requests per minute --streams Streams to annotate, comma-separated (default primary) --only Only these episodes (ids or local indexes), comma-separated ### cerul render Render published semantic labels and hand skeletons without model calls Annotated video, or an exported annotations.json for a dataset episode --out New review video (must not already exist) --stream Camera id (defaults to the primary camera) --watermark Burn a visible Cerul signature into the caption panel ### cerul remove Remove indexed videos, or free the disk they and their caches use ... Videos or directories to forget. Their files are left where they are --cache Free cached previews, query vectors, and media proxies; all regenerated --all-indexes Free every search index; each rebuilds from sidecars with no model calls --index Free one search index by space id; rebuilt from sidecars on next use --compact Reclaim space inside search indexes without dropping them ### cerul upgrade Install the newest published release of Cerul over this one ### cerul skill Print or install the agent skill that teaches this CLI to a coding agent --install Install for this agent: claude, codex, or pi --dir Install into this skills directory instead of an agent's own --print Write the skill to stdout instead of installing it --force Replace an installed skill even when it was changed after it was written ### cerul help Print this message or the help of the given subcommand(s) cerul help index Index videos so they can be searched (screen text, speech, visual search) cerul help search Find moments by description, exact words, or a reference image cerul help status Show indexed videos, model configuration, and storage cerul help open Open a result from the last search in a video player, at its moment cerul help auth Manage the saved Gemini API key cerul help config Configure the required Gemini key and optional default speech transcription cerul help annotate Generate semantic labels and optional local hands for embodied videos cerul help render Render published semantic labels and hand skeletons without model calls cerul help remove Remove indexed videos, or free the disk they and their caches use cerul help upgrade Install the newest published release of Cerul over this one cerul help skill Print or install the agent skill that teaches this CLI to a coding agent cerul help help Print this message or the help of the given subcommand(s)