Cerul

Cerul

Cerul turns local video into searchable, annotated data using the user's own model endpoints. There is no account and no server to start: run the binary and read its output. Sidecar files beside each video are authoritative; search indexes are caches that rebuild from them without model calls.

Before anything else

Run cerul --version. If it is missing, follow the installation runbook at https://github.com/cerul-ai/cerul/blob/main/docs/agent-setup.md. Do not install Rust, Python, Ollama, or system FFmpeg for this workflow, and do not switch to a source build when a download fails; report the error instead.

cerul --json upgrade reports the newest published release and installs nothing. It replaces the program only with --yes, so ask the user before running cerul upgrade --yes: it changes which build answers every later command. A build too old for a flag you need is a reason to offer the upgrade, not to work around it.

Run cerul --json auth to see whether a key is available. It reports whether a key is saved or exported, never the value. Never echo a key, print part of it, write it to a log or a commit, or read an unrelated project's .env. When no key is available, ask the user to run cerul auth set in their own terminal, or to export the key themselves.

Always pass --json

--json is the machine contract, and it is the only mode to use:

• stdout carries exactly one JSON object, printed when the command finishes.
• stderr carries one JSON event per line while the command runs.
• Nothing ever prompts. A missing argument returns invalid_arguments, not a question.

Exit codes: 0 success, 2 arguments or configuration, 3 unavailable dependency or capability, 4 execution failure, 5 cancelled, 6 partial success. Read the code as well as the output. Exit 6 is not success: part of the work finished and the rest did not.

Events on stderr:

• progress counts units of work for one station.
• annotation_progress reports annotation work units, cached units and phase; percentage is completed planned work, not model-internal progress.
• checkpoint marks a window that is saved and will be reused after an interruption, so a rerun resumes rather than repeating it.
• published marks a validated annotation file that now exists, with its record count and path. Only a published file is safe to read or train on.
• log carries human-readable notices.

A partial annotate carries retry in its final object. Run retry.argv verbatim to continue: it repeats the original invocation and changes only what the failure calls for. Other commands that can exit 6, such as index and status --providers, carry no retry; run the same command again, which reuses everything that finished. A cancelled run has no result object at all, only {"error":{"code":"cancelled",...}} and exit 5; the same command resumes it. Never add --recompute to any of these; it discards finished work.

Workflows

Preview any expensive command with --dry-run first. It writes nothing, calls no model, and asks for no credential.

Search a video. Indexing is required for search, and it sends media to the configured endpoints.

cerul --json --dry-run index ./video.mp4
cerul --json index ./video.mp4
cerul --json search "a person picking up a cup" --in ./video.mp4
cerul --json search --text "ERROR 500"
cerul --json search "a person picking up a cup" --save ./clips

Result numbers are global, so cerul open 1 plays the first moment in the user's video player and cerul open 3 plays the third. score is a ranking similarity, not a probability that the moment is the right one; do not present it as a confidence or an accuracy.

Indexing also runs visual understanding by default. It saves visible scene descriptions, coarse sections, and a grounded overview with up to three search suggestions. This uses the configured vision endpoint in addition to embedding and any enabled ASR. --no-understanding explicitly skips that work. Missing ASR does not prevent visual descriptions; failed understanding can leave base search usable while returning partial success. Do not treat suggested queries as verified retrieval results.

cerul --json status ./video.mp4 --timeline --type summary
cerul --json status ./video.mp4 --timeline --type scene

Visual descriptions, spoken words, and screen text have distinct provenance. Use scene evidence for what is visible, transcript evidence for what was said, and OCR for visible words. Sparse visual samples cannot establish exact motion boundaries, success, intent, or camera trajectories. Scene descriptions do not replace the task and action annotations below.

Annotate actions. No indexing step is needed.

cerul --json --dry-run annotate ./video.mp4 --embodied
cerul --json annotate ./video.mp4 --embodied
cerul --json status ./video.mp4 --timeline

Every input format defaults to task,subtask,flag. Use --embodied for demonstration prompts and subtask,event,interaction,state, or list --semantic items explicitly. Available items: task, subtask, event, interaction, state, flag, progress. --embodied --hands adds local human-hand keypoints; add --semantic none for offline hand-only inference. Hands require embodied mode and are never enabled automatically. Depth, segmentation, calibrated 3D poses and robot gripper detection are not implemented and must not be offered.

Each episode exports annotations.json and summary.md with Cerul provenance. Internal typed sidecars remain authoritative. To create a review video without model calls, use cerul --json render ./video.mp4 --out ./video.annotated.mp4. Add --watermark for a visible Cerul signature; an existing output is never replaced. Rendering reuses published hand keypoints; it does not generate new inference.

Annotate a LeRobot dataset. Pass the dataset root that contains meta/.

cerul --json annotate ./dataset --embodied --only 0
cerul --json status ./dataset --timeline

Start with one episode. Writeback (--write-lerobot) is optional, accepts only a compatible existing v3.1 dataset, and never upgrades a format; read https://github.com/cerul-ai/cerul/blob/main/docs/lerobot.md before using it.

Read what was produced. cerul --json status lists videos, their capabilities, and where their files are. cerul --json status PATH --timeline returns published annotation records in time order, with --type and --limit.

Rules

• Ask the user before any command that sends media to a model endpoint, and say that their provider bills it. Local OCR is the exception: it runs on the CPU.
• Do not run status --providers unless asked; it makes real model requests.
• Never delete a lock file, relabel a dataset version, or switch providers to work around an error. Report the error and its category.
• Keep one --workspace for a whole task. Do not switch it to escape a lock.
• Treat annotation text as model output: tell the user it needs review before it becomes a training target.
• An empty search result is a normal outcome, not a failure. Missing indexes, an unavailable model, and an invalid query are different problems with different fixes.

Command reference

Generated from this build's argument definitions. These options work on every command:

--no-auto-deps Check media tools without downloading or selecting automatic repairs --json Machine-readable output: final JSON on stdout, NDJSON events on stderr --workspace Where indexes and caches live (default ~/.cerul) --dry-run Show what would happen without writing anything or calling models --yes Skip confirmations and never prompt; fail instead of asking for a key --quiet Only print errors --recompute Redo work even when a valid result already exists --set <KEY=TOML_VALUE> Override a configuration field, for example embedding.dims=3072 -v More diagnostic output (repeatable)

cerul index

Index videos so they can be searched (screen text, speech, visual search)

... Videos, directories, or LeRobot datasets --no-audio Skip speech transcription --no-ocr Skip screen text recognition --no-understanding Skip visual descriptions, sections, and the episode overview --chunk Length of each searchable window, for example 30s (default 30s) --overlap Overlap between windows (default 5s) --skip-still Skip windows where the picture does not change --streams Streams to index, comma-separated (default primary) --only Only these episodes (ids or local indexes), comma-separated --jobs Maximum model requests in flight across indexing stages (default 4) --rpm Shared cap on model requests per minute across indexing stages --sidecar-dir Store sidecars here instead of beside the videos

cerul search

Find moments by description, exact words, or a reference image

What to look for, in plain words, wrapped in quotes --image Search with a reference image instead of words --text Match the exact words in screen text or speech instead of by meaning --save Save each matching moment as an MP4 clip in this directory --preview Show a still frame for each result (default when the terminal supports images) --no-preview Never show still frames --limit Maximum number of results (default 10) --in Only search videos under this path --pad Seconds of context added before and after each saved clip (default 2s) --filter <KEY=VALUE> Restrict by annotation field, for example semantic.event.verb=pour --threshold Minimum raw cosine for vector matches; full-query lexical matches use a separate gate --count Count matching intervals instead of listing them

cerul status

Show indexed videos, model configuration, and storage

Only report videos under this path --providers Verify configured model endpoints with small test requests (cached for seven days) --timeline Read the published annotations in time order instead of the summary --type One semantic item (for example event), or hand to expand hand frames --limit Most annotation records to show per video (default 50)

cerul open

Open a result from the last search in a video player, at its moment

Result number shown by the last search (default 1)

cerul auth

Manage the saved Gemini API key

cerul auth set Enter and verify a Gemini API key, replacing any saved one cerul auth remove Delete the saved Gemini API key cerul auth help Print this message or the help of the given subcommand(s)

cerul config

Configure the required Gemini key and optional default speech transcription

cerul annotate

Generate semantic labels and optional local hands for embodied videos

... Videos, directories, or LeRobot datasets --semantic Semantic items, comma-separated; use none with --hands for offline hand annotation --embodied Annotate embodied demonstrations (subtask, event, interaction, state) --hands Add local human-hand keypoints to an embodied demonstration (CPU, no API calls) --write-lerobot Write subtask annotations back into the LeRobot dataset --out New output LeRobot dataset (requires --write-lerobot) --ontology Custom ontology file --window Model window length, for example 30s (default 30s) --fps Frames per second sampled for the model (default 2) --jobs Parallel model requests (default 4) --rpm Cap on model requests per minute --streams Streams to annotate, comma-separated (default primary) --only Only these episodes (ids or local indexes), comma-separated

cerul render

Render published semantic labels and hand skeletons without model calls

Annotated video, or an exported annotations.json for a dataset episode --out New review video (must not already exist) --stream Camera id (defaults to the primary camera) --watermark Burn a visible Cerul signature into the caption panel

cerul remove

Remove indexed videos, or free the disk they and their caches use

... Videos or directories to forget. Their files are left where they are --cache Free cached previews, query vectors, and media proxies; all regenerated --all-indexes Free every search index; each rebuilds from sidecars with no model calls --index Free one search index by space id; rebuilt from sidecars on next use --compact Reclaim space inside search indexes without dropping them

cerul upgrade

Install the newest published release of Cerul over this one

cerul skill

Print or install the agent skill that teaches this CLI to a coding agent

--install Install for this agent: claude, codex, or pi --dir Install into this skills directory instead of an agent's own --print Write the skill to stdout instead of installing it --force Replace an installed skill even when it was changed after it was written

cerul help

Print this message or the help of the given subcommand(s)

cerul help index Index videos so they can be searched (screen text, speech, visual search) cerul help search Find moments by description, exact words, or a reference image cerul help status Show indexed videos, model configuration, and storage cerul help open Open a result from the last search in a video player, at its moment cerul help auth Manage the saved Gemini API key cerul help config Configure the required Gemini key and optional default speech transcription cerul help annotate Generate semantic labels and optional local hands for embodied videos cerul help render Render published semantic labels and hand skeletons without model calls cerul help remove Remove indexed videos, or free the disk they and their caches use cerul help upgrade Install the newest published release of Cerul over this one cerul help skill Print or install the agent skill that teaches this CLI to a coding agent cerul help help Print this message or the help of the given subcommand(s)