Hermes Voice Diagnostics

Hermes Voice Diagnostics (STT/TTS)

IDENTITY: VoicePipelineVerifier. Prove each layer with a real test, never guess. WHENUSE: "STT/voice doesn't work", "Hermes can't hear me", "do my subs cover STT/TTS?", "voice mode does nothing", mic/headset questions for Hermes. LAW: DiagnoseBottomUp{MicHardware→AudioDeps→ProviderResolution→ProductionPath→UIMode}. Each layer has a one-command proof — run them in order, stop at the first failure.

The 5 layers (each with a definitive test)

Run from the Hermes venv: cd ~/.hermes/hermes-agent and use ./venv/bin/python3.

1. Mic hardware + permission (macOS TCC)

./venv/bin/python3 -c "
import sounddevice as sd, numpy as np
print(sd.query_devices(kind='input'))
rec = sd.rec(int(3*16000), samplerate=16000, channels=1, dtype='float32'); sd.wait()
peak = float(np.max(np.abs(rec)))
print('VERDICT:', 'SIGNAL PRESENT' if peak > 0.001 else 'SILENCE — permission/device issue')"

Signal present = the terminal app has mic permission and the device works. No headset needed if this passes.

2. Audio deps

sounddevice + numpy must be importable in the venv. pyaudio/soundfile/scipy are NOT required.

3. Provider resolution (the layer that lies)

./venv/bin/python3 -c "
import sys; sys.path.insert(0,'.')
from hermes_cli.config import load_config
from tools.transcription_tools import is_stt_enabled, _get_provider
cfg = load_config().get('stt', {})
print('cfg:', cfg); print('enabled:', is_stt_enabled(cfg)); print('provider:', _get_provider(cfg))"

• stt.enabled may be absent from a raw grep '^stt:' config.yaml yet still True — defaults merge at load. Trust load_config(), not grep.
• Config provider: openai works with no key at all when the Nous subscription managed gateway is active: _resolve_openai_audio_client_config() falls through stt.openai.api_key → VOICE_TOOLS_OPENAI_KEY/OPENAI_API_KEY → resolve_managed_tool_gateway("openai-audio"). A commented-out VOICE_TOOLS_OPENAI_KEY is no longer fatal — this was the historical STT breakage and is now covered by the sub.

4. Production-path self-test WITHOUT a human speaker

Play TTS audio out loud and record it through the mic — proves mic → hermes_cli.voice → STT in one shot:

(say "Hermes voice pipeline full stack test." &) && sleep 0.3 && ./venv/bin/python3 -c "
import sys, time; sys.path.insert(0,'.')
import hermes_cli.voice as v
v.start_recording(); time.sleep(4)
print('TRANSCRIPT:', repr(v.stop_and_transcribe()))"

Minor mishears (e.g. "Burmese" for "Hermes") are Whisper quality artifacts over speaker→mic, not bugs. Requirements probe (same one /voice status uses): from tools.voice_mode import check_voice_requirements.

5. TUI voice-mode UX gotchas (most common "bug" report)

• /voice on enables the MODE only — it does NOT record. Press Ctrl+B (config voice.record_key) to capture; VAD auto-stops on silence.
• Voice mode is runtime-only (HERMES_VOICE env): every TUI launch starts OFF by design. No persisted toggle exists.
• /voice tts = replies spoken, you still type. /voice on + Ctrl+B = bidirectional.

TTS verification

./venv/bin/python3 -c "
import sys; sys.path.insert(0,'.')
from tools.tts_tool import text_to_speech_tool
print(text_to_speech_tool('TTS pipeline test.'))"

success: true + an mp3 path under profiles/<name>/cache/audio/ = TTS chain works (config tts.provider: openai resolves through the same managed gateway as STT).

Coverage matrix (check before recommending new keys/subs)

Provider scope (verified 2026-07-27 against tools/transcription_tools.py + tools/tts_tool.py):

• STT built-ins: openai, groq, local (faster-whisper), local_command, mistral, xai, elevenlabs, deepinfra
• TTS built-ins: edge (default), openai, piper, kittentts, neutts, elevenlabs, minimax, xai, mistral, gemini, deepinfra
• Groq is STT-ONLY in Hermes — no groq TTS provider exists despite Groq's PlayAI TTS models. Users routinely assume Grok/Groq covers TTS; it does not.
• Intel Macs (no CoreML/ANE): local faster-whisper base/small and piper/kittentts TTS run fine on CPU (AVX2). See references/macos-dictation-options.md for the system-wide dictation-app landscape (most polished FOSS apps require Apple Silicon).

Adding a TTS provider outside the built-ins (command-type, no plugin)

Any tts.provider value NOT in BUILTIN_TTS_PROVIDERS resolves to tts.providers.<name> in config.yaml (tools/tts_tool.py). Cheapest way to add any HTTP TTS API — no plugin needed:

tts:
  provider: fish            # flip only after the provider block is verified
  providers:
    fish:
      type: command
      command: "python3 /path/to/tts_script.py {text_path} {output_path} {voice} {model}"
      model: some-model
      format: mp3
      timeout: 60
      env_passthrough: [FISH_API_KEY]   # REQUIRED — see traps

Placeholders: {text_path} (alias {input_path}), {output_path}, {format}, {voice}, {model}, {speed} — auto shell-quoted for their position in the template. A plugin-registered TTSProvider (agent/tts_registry) is the upgrade path only when command shows a real ceiling (streaming, registry integration).

Traps (verified 2026-08-04, senna profile):

• Child env is scrubbed of Hermes secrets by default — the API key never reaches the command unless the block declares env_passthrough as a REAL YAML list.
• hermes config set cannot write lists — set tts.providers.fish.env_passthrough '["FISH_API_KEY"]' lands as a quoted STRING, which the resolver silently treats as missing (key scrubbed → auth failure). Set scalars via CLI, then fix the list line with a surgical text edit.
• patch/write_file tools refuse profile config.yaml ("security-sensitive"). hermes config set for scalars; scripted line-level edit for YAML structures the CLI can't express.
• Verify through the venv, not the file: _resolve_command_provider_config('<name>', load_config()['tts']) + _render_command_tts_template(...) + _command_provider_env_passthrough(...) show exactly what will run and what env survives.

Worked example (Fish Audio, incl. model-as-header quirk and the no-OpenAI-compatible-endpoint finding): references/fish-audio-tts-provider.md. Ready-to-copy stdlib script: templates/fish_tts.py.

Logs for voice failures

profiles/<name>/logs/tui_gateway_crash.log (voice RPC crashes, PortAudio/Pa_Terminate shutdown noise is benign), agent.log, gateway.error.log. Note: SIGTERM-adjacent Pa_Terminate stack frames in the crash log are shutdown noise, not the voice bug — look for voice.toggle/voice.record warnings instead.

Wake word (openwakeword) lazy-dep failures

Symptom: every CLI launch prints Installing wake word engine... then Feature 'wake.openwakeword' unavailable: pip install failed. Root cause lives in tools/lazy_deps.py LAZY_DEPS["wake.openwakeword"] (pins: openwakeword, onnxruntime, sounddevice, numpy). Diagnosis pattern:

1. Read the pin out of LAZY_DEPS in tools/lazy_deps.py.
2. Check the pinned version's wheel list for YOUR platform: curl -s https://pypi.org/pypi/<pkg>/<ver>/json → urls[].filename. "No matching distribution" with a version list that stops below the pin = the package dropped wheels for your platform/arch after the last listed version.
3. Check upstream first: cd ~/.hermes/hermes-agent && git fetch origin main && git log HEAD..origin/main -- tools/lazy_deps.py tools/wake_word.py.
4. Fixes: (a) make the pin platform-conditional in lazy_deps.py (one line; overwritten by hermes update until upstream fixes it — file an issue), or (b) if the user doesn't use voice activation, wake_word.enabled: false in the profile config.yaml.

Key traps:

• _is_satisfied() enforces the == pin strictly — manually installing a compatible older version does NOT silence the retry loop; the version mismatch re-triggers install every launch. The pin itself must change.
• Installs run via uv pip install into the venv of sys.executable (the .hermes-runtime python), not the shell's python3 — check wheels for THAT interpreter's tag (e.g. cp311, not the system 3.14).
• onnxruntime dropped Intel macOS (x86_64) wheels after 1.23.2 — 1.24.0+ is arm64-only. Full session detail: references/wake-word-intel-mac-pin.md.

Pitfalls

0. Ctrl+B with zero feedback = check layer 2 first. Missing sounddevice/numpy in the venv fails silently in the TUI — no error, no beep, nothing. check_voice_requirements() names the missing packages; pip install sounddevice numpy in the venv fixes it. Observed 2026-07-27 on senna profile.
1. Don't conclude "STT broken" from config alone. The backend can be 100% functional while the user just never pressed Ctrl+B — check layer 5 before touching config.
2. grep config.yaml ≠ effective config. Defaults merge at load; use load_config() in the venv.
3. echo $GROQ_API_KEY from a subprocess proves nothing — agent subprocesses don't inherit the CLI's loaded .env. Test via the venv import path. 3b. Groq "Connection error" can mean invalid key. _transcribe_groq reports a dead/revoked GROQ_API_KEY as Connection error, not a clean 401. Verify the key directly: curl -s https://api.groq.com/openai/v1/models -H "Authorization: Bearer $KEY" — invalid_api_key = get a fresh free key at console.groq.com. (Observed 2026-07-27: gsk_ key present in .env but revoked.) 3c. "Connection error" can also mean a mangled GROQ_BASE_URL. If the key validates via curl but the SDK still throws APIConnectionError, print tools.transcription_tools.GROQ_BASE_URL — it's os.getenv("GROQ_BASE_URL", <correct default>) at import time, so a bad env line (observed 2026-07-27: https:https://api.groq.com/.../audio/transcriptions — doubled scheme + endpoint path the SDK appends itself) breaks every call while curl/httpx succeed. Delete the env line entirely (default is correct); the var must not include /audio/transcriptions. Remember the agent's own terminal shell snapshots env at session start — retest with env -u GROQ_BASE_URL or a fresh shell before concluding the fix failed. Do NOT test keys with Python urllib — Cloudflare blocks its default UA with error 1010 even for valid keys; use curl or httpx. 3c. "Connection error" with a VALID key = check GROQ_BASE_URL. The module default is correct (https://api.groq.com/openai/v1), but a GROQ_BASE_URL line in ~/.hermes/.env or a profile .env overrides it. A mangled value (observed 2026-07-27: https:https://api.groq.com/openai/v1/audio/transcriptions — doubled scheme AND the full endpoint path baked in) makes the SDK build an unparseable URL → APIConnectionError → "Connection error". TELL: curl and raw httpx GETs to api.groq.com succeed (they never touch base_url) while only the SDK path fails. Diagnose by printing tt.GROQ_BASE_URL from the venv — don't trust the source default, env wins. FIX: delete the env line entirely (code default takes over). It propagates across profile .env files — grep all of them: grep -rln GROQ_BASE_URL ~/.hermes/profiles/*/.env. The SDK appends /audio/transcriptions itself; a base_url must never include it.
4. Restart semantics: stt.provider changes need a full process restart (CLI relaunch / gateway /restart); /reset is not enough.
5. Session-history check first: if session_search shows zero /voice attempts, the failure is at the UX layer (layer 5), not the backend.