# Android Voice

> Voice input/output on Android with OpenAI audio APIs: RECORD_AUDIO permission, MediaRecorder capture, multipart upload to /v1/audio/transcriptions (STT), /v1/audio/speech (TTS) playback with MediaPlayer, and a VoiceSession state machine for push-to-talk UX. Use when adding speech-to-text, text-to-speech, mic recording, or a voice-driven screen to an Android app.

- Skill: `kathleenmaas/android-voice` (Agent Skill)
- Install (CLI): `npx skillmds@latest add kathleenmaas/android-voice`
- Raw SKILL.md: https://api.skillmd.com/api/skills/kathleenmaas/android-voice/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: KathleenMaas (https://skillmd.com/u/kathleenmaas)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/kathleenmaas/android-voice

---


# Android Voice (OpenAI STT/TTS)

Discrete pipeline: **record → transcribe → text**, and **text → synthesize →
play**. Each stage independently testable; keep a typed-text input path so
agent/network debugging never requires a microphone.

The Realtime speech-to-speech API is the production upgrade (latency,
barge-in) — don't build it in a timed interview; design the voice layer so it
can slot in (interfaces below already isolate it).

## Endpoints (verified Jul 2026)

| Direction | Endpoint | Model | Notes |
|-----------|----------|-------|-------|
| STT | `POST /v1/audio/transcriptions` | `gpt-4o-mini-transcribe` | multipart; `response_format=text` or `json`; accepts m4a/mp3/wav/webm |
| TTS | `POST /v1/audio/speech` | `gpt-4o-mini-tts` | JSON body; returns audio bytes (`response_format: "mp3"`) |

Known issue: current `gpt-4o-mini-tts` snapshots sometimes **truncate the
final sentence** (~5–20%, silent 200). Mitigation for short spoken summaries:
end `input` with a final period and avoid trailing short questions; don't
chase it further in a demo.

## Permission (Compose)

`AndroidManifest.xml`: `<uses-permission android:name="android.permission.RECORD_AUDIO" />`

```kotlin
val permissionLauncher = rememberLauncherForActivityResult(
    ActivityResultContracts.RequestPermission()
) { granted -> if (granted) onMicGranted() else onMicDenied() }

// on mic button press:
when {
    context.checkSelfPermission(RECORD_AUDIO) == PERMISSION_GRANTED -> onMicGranted()
    else -> permissionLauncher.launch(RECORD_AUDIO)
}
```

Denied → keep the app usable: fall back to the text input path, show a short
rationale. Don't loop the request dialog.

## Recording — MediaRecorder → m4a

Simplest reliable capture; AAC in an `.m4a` container is accepted by the
transcription endpoint directly. (`AudioRecord` is for raw PCM / streaming —
only if the prompt demands it.)

```kotlin
class VoiceRecorder @Inject constructor(@ApplicationContext private val context: Context) {
    private var recorder: MediaRecorder? = null
    private var output: File? = null

    fun start(): Result<Unit> = try {
        val file = File(context.cacheDir, "turn_${System.currentTimeMillis()}.m4a")
        recorder = MediaRecorder(context).apply {
            setAudioSource(MediaRecorder.AudioSource.MIC)
            setOutputFormat(MediaRecorder.OutputFormat.MPEG_4)
            setAudioEncoder(MediaRecorder.AudioEncoder.AAC)
            setAudioSamplingRate(16_000) // speech; keeps upload small
            setOutputFile(file.absolutePath)
            prepare()
            start()
        }
        output = file
        Result.success(Unit)
    } catch (t: Throwable) {
        Result.failure(t.toAppError())
    }

    fun stop(): Result<File> = try {
        recorder?.apply { stop(); release() }
        recorder = null
        Result.success(requireNotNull(output))
    } catch (t: Throwable) {
        recorder?.release(); recorder = null
        Result.failure(t.toAppError())
    }
}
```

- `MediaRecorder(context)` ctor needs API 31+; use the deprecated no-arg ctor
  below that
- `stop()` throws if called immediately after `start()` (no audio yet) — treat
  as "too short, try again"
- Cache dir files: delete after successful transcription

## STT — multipart upload (OkHttp)

```kotlin
suspend fun transcribe(file: File): Result<String> = withContext(Dispatchers.IO) {
    try {
        val body = MultipartBody.Builder().setType(MultipartBody.FORM)
            .addFormDataPart("model", "gpt-4o-mini-transcribe")
            .addFormDataPart("response_format", "text")
            .addFormDataPart("file", file.name, file.asRequestBody("audio/m4a".toMediaType()))
            .build()
        val request = Request.Builder()
            .url("https://api.openai.com/v1/audio/transcriptions")
            .header("Authorization", "Bearer ${BuildConfig.OPENAI_API_KEY}")
            .post(body)
            .build()
        client.newCall(request).execute().use { response ->
            if (!response.isSuccessful) return@withContext Result.failure(AppError.Http(response.code))
            Result.success(response.body.string().trim())
        }
    } catch (t: CancellationException) {
        throw t
    } catch (t: Throwable) {
        Result.failure(t.toAppError())
    }
}
```

Optional `prompt` form field improves domain vocabulary ("Kotlin, Gradle,
Hilt, Jetpack Compose") — cheap accuracy win for coding speech.

## TTS — synthesize + play

```kotlin
suspend fun speak(text: String): Result<Unit> = withContext(Dispatchers.IO) {
    try {
        val json = """{"model":"gpt-4o-mini-tts","voice":"alloy","input":${text.toJsonString()},"response_format":"mp3"}"""
        val request = Request.Builder()
            .url("https://api.openai.com/v1/audio/speech")
            .header("Authorization", "Bearer ${BuildConfig.OPENAI_API_KEY}")
            .post(json.toRequestBody("application/json".toMediaType()))
            .build()
        client.newCall(request).execute().use { response ->
            if (!response.isSuccessful) return@withContext Result.failure(AppError.Http(response.code))
            val file = File(context.cacheDir, "speech.mp3")
            file.outputStream().use { response.body.byteStream().copyTo(it) }
            play(file) // suspends until playback completes
        }
        Result.success(Unit)
    } catch (t: CancellationException) {
        throw t
    } catch (t: Throwable) {
        Result.failure(t.toAppError())
    }
}

private suspend fun play(file: File) = suspendCancellableCoroutine { cont ->
    val player = MediaPlayer().apply {
        setDataSource(file.absolutePath)
        setOnCompletionListener { it.release(); cont.resume(Unit) }
        setOnErrorListener { mp, _, _ -> mp.release(); cont.resume(Unit); true }
        prepare() // file is local; prepareAsync unnecessary
        start()
    }
    cont.invokeOnCancellation { player.release() } // barge-in / stop speaking
}
```

- Write bytes to a file then `MediaPlayer` — simplest reliable path; skip
  ExoPlayer/streaming playback unless latency polish time remains
- Cancellation of the `speak` coroutine stops playback → this is your
  "barge-in": cancel speaking when the user presses the mic
- Use Android's built-in `TextToSpeech` only as an offline fallback if asked;
  quality is noticeably worse

## VoiceSession state machine

One enum drives the whole screen; every phase change is a UiState copy.

```kotlin
enum class VoicePhase { Idle, Listening, Transcribing, WaitingOnAgent, Speaking }

// transitions:
// Idle          --mic press-->        Listening   (recorder.start)
// Listening     --mic release-->      Transcribing (recorder.stop → transcribe)
// Transcribing  --transcript-->       WaitingOnAgent (send turn)
// WaitingOnAgent --terminal event-->  Speaking (if speak text) else Idle
// Speaking      --playback done-->    Idle
// any           --stop/cancel-->      Idle (cancel jobs; recorder/player released)
```

- Push-to-talk (press-and-hold or tap-to-toggle) beats always-listening for a
  demo: deliberate, no VAD tuning, no open-mic questions
- Speak **summaries at turn boundaries**, never stream tokens into TTS
- Big touch targets + high contrast if the prompt is glanceable/car-mode

## Emulator

- Extended controls (⋯) → Microphone → enable **"Virtual microphone uses host
  audio input"** — the laptop mic feeds the emulator
- Audio output plays through host speakers by default
- Rehearse once: mic levels through the emulator are quieter than device mics;
  the `prompt` field on STT helps

## Pitfalls

- `MediaRecorder.stop()` immediately after start → `RuntimeException` (treat
  as "too short")
- Forgetting `Authorization` header on the *second* OkHttp client if you build
  a fresh one — share the client, add the key via interceptor
- TTS `input` not JSON-escaped (quotes in summaries) — serialize with Moshi,
  don't hand-build JSON in real code
- Playing TTS while still recording (release order: recorder before player)
- Blocking the main thread with `MediaPlayer.prepare()` on remote sources
  (local file is fine)
- Testing voice before the text path works — voice bugs and agent bugs become
  indistinguishable

## Related

- App structure / Result / AppError:
  [../android-interview-bootstrap/SKILL.md](../android-interview-bootstrap/SKILL.md)
- Agent loop / harness-server client:
  [../android-agent-harness/SKILL.md](../android-agent-harness/SKILL.md)