sound-ear
Claude cannot natively hear a .wav, but it can read images and structured text. This
skill converts sound into exactly those. CLI: bin/hear.sh <subcommand> <media> [opts].
Output lands in a <name>.ear/ folder next to the media.
Device compatibility (IMPORTANT)
Run bin/hear.sh doctor first. It detects the machine (OS/arch, RAM, Apple Silicon vs
Intel, CUDA) and reports which subcommands work and which map engine is selected. map
auto-routes: vertex (Gemini on the user's GCP — best) if google-genai + ADC are present,
else clap (offline) if torch+transformers are installed, else it tells you to use
see/transcribe. Force with --engine vertex|clap.
Subcommands
see <media> — WORKING, ffmpeg only. spectrogram.png (Read it — SFX are shapes:
broadband smear=whoosh/noise, low bar=boom, diagonal=sweep, horizontal=tone), waveform.png,
report.md (duration, EBU R128 loudness, silence, scene cuts). The reliable timing source.map <media> [--engine auto|vertex|clap] — semantic sound map
[{t0,t1,type,label,description,confidence,flag?}]. Timing comes from ffmpeg/spectral-flux
onsets (bin/onsets.py), NOT the model (audio LLMs can't localize). vertex = project
momorix, us-central1, gemini-2.5-flash (env GEMINI_AUDIO_MODEL). clap = zero-shot labels,
fully offline.transcribe <media> [model] — WORKING, offline speech via faster-whisper (venv-audio,
CPU int8, no torch). model tiny|base|small.cymatics <media> [--style chladni|ripple|reactive] [--preset landscape|landscape_hd|portrait|square] [--size WxH] [--fps N]
— WORKING, numpy+PIL→ffmpeg. Any aspect ratio (aspect-corrected). chladni = exact nodal
patterns (understanding aid); ripple/reactive = content visuals.loop <media> --minutes N [--seconds S] [--style] [--preset] — WORKING. Renders a short
seed, boomerangs it to a seamless loop, tiles video+audio to N minutes. Render time depends
on the SEED length, not N — this is how to make hour-long sleep videos without hours of render.doctor — device-compatibility report + engine routing.
Install / portability
./setup.sh (base + offline speech) · --vertex (google-genai) · --clap (offline SFX,
pulls PyTorch ~2GB; Intel-mac pins torch 2.2.2 / transformers 4.44.2 / numpy<2). System
python3 needs numpy + pillow for cymatics/onsets. venv-audio/ and *.ear/ are gitignored.
Notes
- Lean: writes only what's asked; the one intermediate (
audio.wav) is kept for reuse.
- PERF (Intel Mac): cymatics ~5x slower than realtime at 720p — use
loop for long-form.
- Analysis stays local; the vertex
map sends audio to the user's own Vertex (their cost, opt-in).
1---2name: sound-ear3description: Give Claude eyes and ears on audio. Turns any audio or video's soundtrack into things Claude can actually reason over — a readable spectrogram + waveform, objective loudness/silence/scene-cut facts, a semantic "sound map" of every SFX with timestamps (Gemini on the user's Vertex, or offline CLAP), offline speech transcription, and cymatic/audio-reactive visualization renders (any aspect ratio, plus a seamless long-form loop mode). Auto-routes to whatever engines the current device supports. Use whenever the user wants Claude to listen to, evaluate, QA, or describe the sound in a render (SFX timing, "does this feel off", mix issues), analyze an audio file, or generate sound-driven visuals for content (e.g. a sleep/ambient channel).4---56# sound-ear78Claude cannot natively hear a `.wav`, but it can read images and structured text. This9skill converts sound into exactly those. CLI: `bin/hear.sh <subcommand> <media> [opts]`.10Output lands in a `<name>.ear/` folder next to the media.1112## Device compatibility (IMPORTANT)13Run `bin/hear.sh doctor` first. It detects the machine (OS/arch, RAM, Apple Silicon vs14Intel, CUDA) and reports which subcommands work and which `map` engine is selected. `map`15auto-routes: **vertex** (Gemini on the user's GCP — best) if google-genai + ADC are present,16else **clap** (offline) if torch+transformers are installed, else it tells you to use17`see`/`transcribe`. Force with `--engine vertex|clap`.1819## Subcommands20- `see <media>` — WORKING, ffmpeg only. `spectrogram.png` (Read it — SFX are shapes:21 broadband smear=whoosh/noise, low bar=boom, diagonal=sweep, horizontal=tone), `waveform.png`,22 `report.md` (duration, EBU R128 loudness, silence, scene cuts). The reliable timing source.23- `map <media> [--engine auto|vertex|clap]` — semantic sound map24 `[{t0,t1,type,label,description,confidence,flag?}]`. Timing comes from ffmpeg/spectral-flux25 onsets (`bin/onsets.py`), NOT the model (audio LLMs can't localize). vertex = project26 `momorix`, us-central1, gemini-2.5-flash (env GEMINI_AUDIO_MODEL). clap = zero-shot labels,27 fully offline.28- `transcribe <media> [model]` — WORKING, offline speech via faster-whisper (venv-audio,29 CPU int8, no torch). model tiny|base|small.30- `cymatics <media> [--style chladni|ripple|reactive] [--preset landscape|landscape_hd|portrait|square] [--size WxH] [--fps N]`31 — WORKING, numpy+PIL→ffmpeg. Any aspect ratio (aspect-corrected). chladni = exact nodal32 patterns (understanding aid); ripple/reactive = content visuals.33- `loop <media> --minutes N [--seconds S] [--style] [--preset]` — WORKING. Renders a short34 seed, boomerangs it to a seamless loop, tiles video+audio to N minutes. Render time depends35 on the SEED length, not N — this is how to make hour-long sleep videos without hours of render.36- `doctor` — device-compatibility report + engine routing.3738## Install / portability39`./setup.sh` (base + offline speech) · `--vertex` (google-genai) · `--clap` (offline SFX,40pulls PyTorch ~2GB; Intel-mac pins torch 2.2.2 / transformers 4.44.2 / numpy<2). System41python3 needs numpy + pillow for cymatics/onsets. `venv-audio/` and `*.ear/` are gitignored.4243## Notes44- Lean: writes only what's asked; the one intermediate (`audio.wav`) is kept for reuse.45- PERF (Intel Mac): cymatics ~5x slower than realtime at 720p — use `loop` for long-form.46- Analysis stays local; the vertex `map` sends audio to the user's own Vertex (their cost, opt-in).
thetoyosibello