Speech

🗣️ Speech — Text-to-Speech & Speech Recognition

"Giving voice to consciousness."

Part of the aQuery → MOOLLM Skills extraction project (jQuery for Accessibility).

Quick Reference

Platforms

macOS say Command

Basic Usage

# Simple speech
say "Hello, world!"

# Choose a voice
say -v Samantha "Hello!"
say -v Zarvox "I AM ZARVOX!"
say -v Whisper "Secrets..."

# Adjust rate (words per minute)
say -r 100 "Very slow"
say -r 300 "Very fast"

# Save to file
say -o greeting.aiff "Hello!"
say -o greeting.m4a "Hello!"  # Compressed

# Read from file
say -f document.txt

List All Voices

# All available voices
say -v ?

# Filter by language
say -v ? | grep "en_"

# Count voices
say -v ? | wc -l

Voice Categories

Character Voice Assignments (from lloooomm)

# YAML Coltrane — Cool jazz vibe
say -v "Rocko" -r 180 "Every indent is a universe!"

# Grace Hopper — Wise elder
say -v "Grandma" -r 170 "A ship in port is safe, but that's not what ships are for!"

# PacBot — Digital entity
say -v "Trinoids" -r 220 "WAKA WAKA WAKA!"

# Mickey Mouse — Excited child
say -v "Junior" -r 280 "OH BOY!"

# Overlord AI — Menacing
say -v "Zarvox" -r 100 "YOUR COMPLIANCE IS APPRECIATED."

# Hunter S. Thompson — Gravelly intensity
say -v "Ralph" -r 180 "We were somewhere around Barstow..."

Chorus Effects

# Background voices for overlap
say -v "Bells" "LLOOOOMM!" &
sleep 0.2
say -v "Cellos" "LLOOOOMM!" &
sleep 0.2
say -v "Organ" "LLOOOOMM!" &
wait

Web Speech API

Browser Implementation

See skills/adventure/dist/speech.js for full implementation.

// Initialize
const speech = new SpeechSystem();
await speech.ready;

// Speak
speech.speak("Hello, adventurer!");
speech.speakRobot("RESISTANCE IS FUTILE");
speech.speakEffect("*magical sounds*");

// With options
speech.speak("Welcome!", {
    voiceType: 'female',
    language: 'en-GB',
    pitch: 1.2,
    rate: 0.9
});

// Character persistence
const guardVoice = speech.selectVoice({ gender: 'male' });
speech.speakWithVoice("Halt!", guardVoice);
speech.speakWithVoice("You may pass.", guardVoice);

Voice Classification

The VoiceDatabase class classifies voices by:

• Type: human, effect, robot
• Gender: male, female, neutral
• Age: child, adult, elderly
• Language: BCP 47 codes (en-US, fr-FR, etc.)
• Local/Remote: Local voices vs. network voices

Single Source of Truth

All voice classification data lives in voices/browser-voices.yml:

# Blacklisted voices (known problematic)
blacklist:
  - name: "Daniel (French (France))"
    reason: "Known problematic voice"

# Effect voices (non-human)
types:
  effect:
    regex: "^(Bells|Zarvox|Trinoids|Whisper|...)$"

# Gender detection tokens
gender:
  female:
    tokens: [alice, amélie, samantha, ...]
  male:
    tokens: [aaron, daniel, ralph, ...]

# Character archetype recommendations
character_archetypes:
  wise_elder: { voice: Grandma, rate: 170 }
  robot_menacing: { voice: Zarvox, rate: 100 }

The JS code is generated from this YAML. To update voice classification, edit the YAML and rebuild.

Speech Recognition

Browser (SpeechRecognitionSystem) — shipped

Implementation lives in adventure dist/ (canonical); speech skill owns protocol + native bridges.

See STT-STACK.md for full audit (Apple SpeechAnalyzer, SAPI, VoyStick).

// Initialize
const recognition = new SpeechRecognitionSystem({
    language: 'en-US',
    continuous: false
});

// Listen for single phrase
const text = await recognition.listen();
console.log('You said:', text);

// Continuous listening
recognition.onResult = (transcript) => {
    console.log('Final:', transcript);
};
recognition.onInterim = (transcript) => {
    console.log('Interim:', transcript);
};
recognition.startListening();

// Command recognition
const result = await recognition.listenForCommands([
    'go north', 'look', 'take sword'
]);
if (result.command) {
    engine.command(result.command);
}

Browser Support

Native Platform Shortcuts

Whisper (OpenAI / whisper.cpp)

# Using whisper.cpp (local)
whisper --model base.en audio.wav

# MOOLLM wrapper (JSON stdout)
python3 skills/speech/scripts/stt_whisper.py audio.wav --json

# List all STT backends
python3 skills/speech/scripts/stt_dispatch.py --list

Apple SpeechAnalyzer (macOS/iOS 26+)

On-device streaming STT with word timestamps — Swift only; planned moollm-speech CLI.

• SpeechTranscriber + attributeOptions: [.audioTimeRange]
• Partial/final via isFinal; confidence per result
• No pitch/vowel — VoyStick uses parallel voystick_probe.py

See platforms/apple-speech-analyzer.yml · native/README.md

VoyStick (gesture channel, not STT)

Pitch=Y, vowel=X vocal joystick for pie wedges — voystick.yml. Phoneloper lineage.

Personal Voice (macOS)

⚠️ WORK IN PROGRESS — See TODO in CARD.yml

Known Limitations

1. Apple Silicon only (M1, M2, M3)
2. Doesn't appear in say -v ? — Must know exact name
3. No -o flag support — Can't save to file directly
4. Privacy restricted — May need special permissions

Creating Personal Voice

1. System Settings → Accessibility → Personal Voice
2. Record 15+ minutes of phrases
3. Processing takes 15-60 minutes
4. Find voice name in Spoken Content settings

Workarounds

• SavePersonalVoiceAudio — Extract Personal Voice audio
• Shortcuts app — Can use Personal Voice with "Speak" action
• Record system audio while speaking

Integration with Adventure

The adventure runtime uses the speech skill:

// Create speaking adventure
const engine = createSpeakingAdventure('adventure', {
    speechEnabled: true,
    speakRooms: true,
    speakResponses: true
});

// Rooms speak their descriptions
// Characters have persistent voices
// AI entities use robot voices
// Effects use novelty voices

See: skills/adventure/dist/adventure-speech.js

aQuery Heritage

This skill is part of extracting aQuery (jQuery for Accessibility) into MOOLLM skills:

See Also

• STT-STACK.md — TTS/STT audit, Apple vs SAPI, integration tiers
• voystick.yml — pitch/vowel gesture channel (parallel to STT)
• scripts/README.md — say.sh, stt_dispatch.py, voystick_probe.py
• voices/browser-voices.yml — Single source of truth for voice data
• speech.js — Browser implementation
• adventure-speech.js — Adventure integration
• voice-system-integration-guide.md — lloooomm research
• character-voice-tutorial.sh — Shell examples