# Audio Summary

> Use this skill whenever the user wants to summarize, analyze, or transcribe an audio file. Triggers include any request involving an audio file (.m4a, .mp3, .wav, .aac, .ogg, etc.) and words like "summarize", "transcribe", "what's in this audio", "what does this say", "analyze this recording", "who is speaking", or any variation. Also triggers when the user drops or mentions an audio file and asks what it contains. Use this skill even if the user says just "summarize this" and has previously mentioned an audio file in the conversation.

- Skill: `rperez030/audio-summary` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add rperez030/audio-summary`
- Raw SKILL.md: https://api.skillmd.com/api/skills/rperez030/audio-summary/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Research & Search
- Author: rperez030 (https://skillmd.com/u/rperez030)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/rperez030/audio-summary

---


# Audio Summary Skill

This skill uses the Gemini API to analyze audio files and produce friendly, natural-language summaries. Gemini understands speech, identifies multiple speakers, detects non-speech sounds (music, laughter, ambient noise), and handles accented or colloquial speech well.

## When to use which mode

- **Summary mode** (default): A friendly narrative description of what's in the recording — who's speaking, what they discuss, the mood, memorable moments. Best for voice messages, meetings, interviews, and podcasts.
- **Transcription mode**: A verbatim or near-verbatim text of the speech. Use this when the user explicitly asks for a transcript, "word for word", or needs to quote specific passages.
- **Combined mode**: Both a summary and a transcription. Use when the user asks for both, or when the content seems important enough to warrant the full text.

When in doubt, default to summary mode and offer to also transcribe if useful.

## Step-by-step workflow

### 1. Locate the audio file

The user may give you a path, drop a filename, or refer to something like "the audio file in my Downloads folder." Resolve the path before proceeding. If ambiguous, ask.

### 2. Confirm the API key is available

This skill requires the `GEMINI_API_KEY` environment variable. Check it with `echo "${GEMINI_API_KEY:+present}"` — if empty, stop and tell the user the key is missing and how to set it (`export GEMINI_API_KEY=...` in their shell rc, then restart the session). Never print the key value.

If the user manages secrets through a password manager (1Password CLI, `pass`, etc.), suggest they wire it through their shell rc rather than hardcoding it.

### 3. Call the Gemini API

Use Python to encode the audio and call the API:

```python
import base64, json, os, urllib.request

key = os.environ["GEMINI_API_KEY"]

# Encode audio
with open(audio_path, 'rb') as f:
    audio_b64 = base64.b64encode(f.read()).decode()

# MIME type mapping
mime_types = {
    '.m4a': 'audio/mp4', '.mp4': 'audio/mp4',
    '.mp3': 'audio/mpeg', '.wav': 'audio/wav',
    '.aac': 'audio/aac', '.ogg': 'audio/ogg',
    '.flac': 'audio/flac', '.webm': 'audio/webm',
}
ext = os.path.splitext(audio_path)[1].lower()
mime_type = mime_types.get(ext, 'audio/mp4')

# Call API
url = f'https://generativelanguage.googleapis.com/v1beta/models/gemini-3-flash-preview:generateContent?key={key}'
payload = {
    'contents': [{
        'parts': [
            {'inline_data': {'mime_type': mime_type, 'data': audio_b64}},
            {'text': prompt}
        ]
    }]
}
req = urllib.request.Request(
    url, data=json.dumps(payload).encode(),
    headers={'Content-Type': 'application/json'}
)
with urllib.request.urlopen(req, timeout=120) as r:
    result = json.load(r)
    print(result['candidates'][0]['content']['parts'][0]['text'])
```

### 4. Prompt template

For **summary mode**:

```
Listen to this audio and write a friendly, natural summary of what is being discussed.
Write it as if you are describing the conversation to someone who has not heard it.
Mention the people referenced by name, the topics covered, the mood and tone, and
any memorable moments or notable quotes. If there are non-speech sounds (music,
laughter, background noise), mention those too.
```

For **transcription mode**:

```
Please transcribe this audio as accurately as possible. Include timestamps every
30 seconds or at natural breaks. Indicate non-speech sounds in [brackets] (e.g.
[laughter], [music], [background noise]). If there are multiple speakers, label
them Speaker 1, Speaker 2, etc. (or use names if clearly stated in the audio).
```

### 5. Present the output

Return the summary or transcription as clean prose in the conversation. If the audio contains names, quotes, or notable details, light formatting (bold for names, italics for direct quotes) is fine, but keep it readable and natural.

If a word looks like it may have been misheard — especially proper nouns, brand names, or acronyms — flag it briefly rather than passing it through silently.

## File size note

The inline base64 approach works well for files up to ~10MB. For larger files, use the Gemini File API instead (upload first, then reference by URI). For most voice messages and short recordings, inline is fine and simpler.

## Model

Use `gemini-3-flash-preview`. It delivers the best combination of audio comprehension, natural language output, and speed (~10s for a 3-minute recording). If it returns a 503 (high demand), retry once before reporting the error to the user.