# Media Memory > Multimodal long-term memory. Ingest, describe, embed (Gemini Embedding 2 / gemini-embedding-001), and search any media (images, video, audio, documents) stored under /media-memory. Supports semantic similarity search plus structured metadata filtering by type, source, date range, and tag. Use when the user shares any media file, when the assistant generates any media, or when a past asset might be relevant to the current task. Triggers on: log this image, save this media, find that screenshot, do we have a recording of, search media memory, what was that file about. - Skill: `talgacapri/media-memory` (Agent Skill) - Install (CLI): `npx skillmds@latest add talgacapri/media-memory` - Raw SKILL.md: https://api.skillmd.com/api/skills/talgacapri/media-memory/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - Author: talgacapri (https://skillmd.com/u/talgacapri) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/talgacapri/media-memory --- ## Why this exists, and what I'd change **Why it exists.** Stateless chat forgets every screenshot, chart, recording, and doc. Asking "find me the wireframe from last month" becomes "scroll through old Slack." This skill embeds everything so it's actually searchable across sessions and projects. **Design tradeoffs.** - **Local ChromaDB instead of a hosted vector DB.** Everything stays in your workspace, no cloud dependency. Cost: setup friction (GEMINI_API_KEY, Python deps), and search is single-machine only. - **Description-first indexing instead of raw image embedding.** Gemini describes the asset, then we embed the description. Cost: files over 18MB get a placeholder description; full transcription requires routing through the Gemini Files API manually. - **Manual ingest call by default.** I didn't auto-trigger on every Claude session because that creates surprise database writes. Cost: people forget to ingest, and a month of screenshots stays unindexed. **What I'd change.** Auto-ingest on file paste in Claude Code, with a quiet confirmation. Manual ingest is reliable but rarely happens in practice. --- # Media Memory A single source of truth for every piece of media the user shares or the assistant generates. Each asset is described by a multimodal Gemini call, embedded with **Gemini Embedding 2** (`gemini-embedding-001`), and stored in a local **ChromaDB** collection alongside a JSON metadata sidecar. ## Commands ```bash /media-memory ingest # log a single asset /media-memory ingest --source ai_generated --tags chart,roadmap /media-memory search "query" # semantic search /media-memory search "query" --type image --from 2026-01-01 --tag finance /media-memory status # counts and storage health ``` ## Prerequisites ```bash pip install -r scripts/media-memory/requirements.txt export GEMINI_API_KEY=... # or GOOGLE_API_KEY ``` `google-genai` and `chromadb` are the only runtime deps. Vectors persist to `media-memory/chroma/`. ## When to Auto-Trigger Use this skill **without being asked** in any of these cases: | Trigger | Action | |----------------------------------------------------------|-------------------------------------| | User attaches an image, audio, video, or document | `ingest` with `--source user_uploaded` | | You generate an image, diagram, or media artifact | `ingest` with `--source ai_generated` and `--source-detail ""` | | User mentions a past asset ("that chart", "the recording from last week") | `search` with relevant filters before answering | | Question references visual or audio context | `search` first, then answer with citations | If the user explicitly says "don't log this" — skip ingestion and proceed. --- ## Workflow A — Ingest 1. Confirm the file path. If the asset only exists in the chat, save it locally first (e.g. into `media-memory/inbox/`) and pass that path. 2. Pick the right `--source`: - `user_uploaded` (default for shared files) - `ai_generated` (anything you produced) - `screenshot` (UI screenshots) - `web_url` (downloaded from a URL — put the URL in `--source-detail`) 3. Run the ingest command. The script will: - Detect MIME and media type (`image | audio | video | document | other`) - Call `gemini-2.5-flash` to produce a natural language description, OCR/extracted text, and (for audio/video) a transcript - Auto-generate 4-8 semantic tags - Embed the combined description + extract with `gemini-embedding-001` - Persist: - The binary under `media-memory/assets/YYYY/MM/` - The full record under `media-memory/metadata/.json` - The vector + flat metadata in `media-memory/chroma/` - An audit row in `media-memory/index.jsonl` 4. Confirm to the user with the new `id`, type, and tags. Duplicates are detected by SHA-256 checksum and skipped automatically. ### Example ```bash python scripts/media-memory/media_memory.py ingest \ ~/Downloads/product-roadmap-q2.png \ --source ai_generated \ --source-detail "Generated for Q2 roadmap review on 2026-04-25" \ --tags product,roadmap,q2 ``` --- ## Workflow B — Search Use this **before** answering questions that touch on prior assets, designs, recordings, or screenshots. ```bash python scripts/media-memory/media_memory.py search "the dashboard wireframe with the Money In Money Out widget" \ --type image --from 2026-01-01 --tag product -n 5 ``` Filters supported: | Flag | Meaning | |------------|-------------------------------------------------------------| | `--type` | `image | video | audio | document | other` | | `--source` | Exact match (e.g. `user_uploaded`, `ai_generated`) | | `--from` | ISO date — only assets ingested on/after | | `--to` | ISO date — only assets ingested on/before | | `--tag` | Single tag — post-filtered against the asset's tag list | | `-n` | Top-k results (default 5) | Returned JSON contains: `id`, `filename`, `type`, `source`, `timestamp_ingested`, `tags`, `stored_path`, truncated `description`, and similarity `distance`. ### Citing search results When you reference an asset back to the user, use this format: > Found in media memory: `` (``, ingested ``) — ``. Path: ``. If the user wants the actual file, point them at `` (it's repo-relative). --- ## Metadata Schema (canonical record) See `media-memory/README.md`. Fields you will most often reason over: - `type` — coarse media class - `source` — who created it - `description` — natural language summary - `extracted_text` / `transcript` — searchable text content - `tags` — semantic indexing - `timestamp_ingested` / `timestamp_epoch` — recency + range filters - `stored_path` — where the binary lives ChromaDB metadata stores all primitive fields; tag arrays are mirrored as `tags_joined` for filtering. --- ## Quality Checklist (before confirming an ingest) - [ ] File saved under `media-memory/assets/YYYY/MM/` - [ ] Sidecar JSON written under `media-memory/metadata/` - [ ] `description` is concrete (not "an image of something") - [ ] `extracted_text` / `transcript` populated when content has text/audio - [ ] At least 3 semantic tags generated - [ ] Vector added to ChromaDB (script prints `[ingested] id=...`) - [ ] If the file was ai-generated, `source_detail` summarizes the prompt or context ## Edge Cases | Situation | Handling | |--------------------------------------------------------|-----------------------------------------------------------------| | File >18 MB | Description is skipped with a note; consider Files API upload | | Unknown MIME | Stored as `type=other`; description still attempted | | `GEMINI_API_KEY` not set | Script exits with a clear error; tell the user before retrying | | Duplicate (same SHA-256) | Skipped; existing `id` returned | | Empty Chroma collection on first search | Return empty list — never fabricate citations | --- ## Related - **Pairs with** `/research-scout` — when scout finds relevant external assets, ingest them with `--source web_url`. - **Pairs with** `/morning-brief` — screenshots from the brief can be archived for later recall. - **Pairs with** `/sketchnote`, `/frontend-design`, `/prototype` — auto-ingest their outputs as `ai_generated`.