# Listening Companion

> Act as a real-time multimodal music companion by listening to audio tracks and providing structural, stylistic, and lyrical feedback.

- Skill: `xuepoo/listening-companion` (Agent Skill)
- Install (CLI): `npx skillmds@latest add xuepoo/listening-companion`
- Raw SKILL.md: https://api.skillmd.com/api/skills/xuepoo/listening-companion/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- License: MIT
- Author: Xuepoo (https://skillmd.com/u/xuepoo)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/xuepoo/listening-companion

---


# Multimodal Music Listening Companion Skill

## Overview

This skill enables AI agents with multimodal audio processing capabilities (such as Gemini 1.5 Pro) to function as a real-time, highly interactive **Music Listening Companion (听歌伴侣)**. The agent literally "hears" the audio tracks played by `alx` and combines it with lyrics and metadata to engage in rich, analytical, and emotional conversations with the user about music.

---

## Capabilities of the Multimodal Audio Companion

Large multimodal models can ingest high-quality audio files directly. When equipped with this skill, the agent performs the following advanced analysis:

1. **Acoustic & Production Analysis**: Identifies specific instruments (e.g. Hammond organ, slap bass, acoustic guitar cuts), synthesizer styles, stereo panning, mixing balance, and vocal reverb/delay processing.
2. **Structural Breakdown**: Maps the song layout (Intro → Verse → Pre-Chorus → Chorus → Bridge → Outro) and explains transition logic or harmonic hooks.
3. **Emotional & Stylistic Analysis**: Interprets the singer's vocal delivery (breathiness, vocal fry, falsetto vibrato) and aligns it with the lyrics to explain the track's emotional resonance.
4. **Lyrical Interpretation**: Combines the audio with synchronized LRC lyrics (fetched via `alx lyric`) to analyze double entendres, poetic metaphors, and cultural references.

---

## Agent Integration Workflow

When a user initiates the Listening Companion mode:

```text
                  ┌──────────────────────┐
                  │ 1. User plays music  │
                  └──────────┬───────────┘
                             ▼
                  ┌──────────────────────┐
                  │ 2. Agent downloads   │
                  │    audio clip & LRC  │
                  └──────────┬───────────┘
                             ▼
                  ┌──────────────────────┐
                  │ 3. Feed audio + LRC  │
                  │    to Multimodal LLM │
                  └──────────┬───────────┘
                             ▼
                  ┌──────────────────────┐
                  │ 4. Rich Conversation │
                  │    on style / music  │
                  └──────────────────────┘
```

### Automation Script Example

The agent can fetch the currently active song, download a 30-60 second preview or the full audio, fetch the cached lyrics, and submit them to the multimodal interface:

```bash
# Step 1: Resolve currently active song metadata
SONG_INFO=$(alx now --json)
SONG_ID=$(echo "$SONG_INFO" | jq -r '.song.song_id')

# Step 2: Download the audio file and its lyrics
alx download "$SONG_ID"
alx lyric "$SONG_ID" --save
```

### MCP Integration

If the host registers the built-in `alx mcp` server (v0.4.0+), the steps above map to single structured calls — `status` instead of `alx now --json`, `lyric_get` instead of `alx lyric`, and `download_add` / `download_status` instead of `alx download` — returning compact JSON without subprocess parsing:

```json
{
  "mcpServers": { "alx": { "command": "alx", "args": ["mcp"] } }
}
```

---

## Sample Companion Dialogues

### Scenario 1: Analyzing Instrument and Production Choices

- **User**: "This guitar solo sounds so warm and organic. What gear or techniques are they using here?"
- **Companion (Model Listening to Audio)**: _"Ah, that's a gorgeous tone. Listening to the audio, they've recorded an antique Gibson ES-335 hollowbody electric guitar running through a lightly driven vintage tube amplifier (likely a Fender Deluxe Reverb). You can hear a subtle spring reverb adding depth, and the player is using a hybrid picking technique (combining flatpick and fingers) to give it that soft, organic dynamic response."_

### Scenario 2: Lyrical and Vocal Resonance

- **User**: "Why does this chorus hit so hard emotionally?"
- **Companion (Model listening to vocal dynamics)**: _"In this section, the singer transitions from a soft, breathy chest voice in the verse directly into a soaring, slightly strained head voice in the chorus. The model's audio captures a slight vocal fry on the words '[02:14.30] 晴天', which perfectly emphasizes the theme of nostalgic longing in the lyrics. The backing vocals also expand from mono center to wide stereo panning, creating a powerful emotional swell."_