# Android Agent Harness

> Android agent clients in two topologies: a client-side LLM tool loop (OpenAI-style Responses or Chat Completions, SSE streaming, tool calls, permissions) or a thin client consuming a remote harness server's event stream over WebSocket. Use when building Cursor-like agents, parsing tool_calls, streaming tokens into UiState, reading SSE with OkHttp, or wiring an OkHttp WebSocket to a sealed AgentEvent Flow.

- Skill: `kathleenmaas/android-agent-harness` (Agent Skill)
- Install (CLI): `npx skillmds@latest add kathleenmaas/android-agent-harness`
- Raw SKILL.md: https://api.skillmd.com/api/skills/kathleenmaas/android-agent-harness/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: KathleenMaas (https://skillmd.com/u/kathleenmaas)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/kathleenmaas/android-agent-harness

---


# Android Agent Harness

Align with bootstrap: `Result` / `AppError` at boundaries, concrete classes,
no interface theater.

Use this skill **only if the prompt needs an LLM/tool loop**. Otherwise stay on
[../android-interview-bootstrap/SKILL.md](../android-interview-bootstrap/SKILL.md).

## Two topologies — pick first

| Topology | When | What the client does |
|----------|------|----------------------|
| **A. Client-side loop** | No backend allowed; tools act on the device | Runs the tool loop itself against OpenAI (bulk of this skill) |
| **B. Harness-server client** | Agent operates on a codebase / machine elsewhere; server runs the agent (e.g. Cursor/Codex SDK) | Sends turns, consumes a typed event stream, renders/speaks it — see [Consuming a harness server](#consuming-a-harness-server-websocket) |

In topology B the server owns the loop; **do not duplicate tool-call parsing on
the client**. The client's contract is the event protocol, not the LLM wire
format. Keep the protocol small, turn-scoped, with exactly one terminal event
per turn (`completed` / `failed` / `cancelled`).

For topology A: the **model proposes tools; the Android app executes them** and
returns results.

## Placement

| Scope | Where |
|-------|--------|
| One chat feature | Keep harness in `:feature:<chat>` (`agent/` package) |
| Reused by 2+ features | Promote to `:core:agent` |

Depends on `:core:network` (OkHttp client) + `:core:model` (messages / errors).
Don’t put streaming inside Retrofit unless you already know that path cold —
**OkHttp + SSE event source / ResponseBody.byteStream()** is the interview-safe default.

## Loop

```
messages (source of truth)
    → POST /v1/chat/completions (messages + tools, stream optional)
    → stream delta.content → UI; accumulate delta.tool_calls by index
    → finish_reason == tool_calls?
         yes → permission → execute tools → append assistant + tool msgs → loop
         no  → append assistant text → stop; wait for user
```

Rules:
- Send the **same tools list** on every call while tools are allowed
- Cap iterations (e.g. **5**) — prevent infinite tool loops
- Cancel via `Job` / `CoroutineScope` cancellation; don’t append half-baked
  assistant messages on cancel (or mark cancelled explicitly)
- `tool_choice: "auto"` normally; `"none"` to force a final text answer

## Roles

| role | writer | purpose |
|------|--------|---------|
| system | app | instructions / product personality |
| user | human | input |
| assistant | model | text and/or `tool_calls` |
| tool | app | result; **must** set `tool_call_id` |

## Kotlin models (keep small)

```kotlin
data class ChatMessage(
    val role: Role, // System, User, Assistant, Tool
    val content: String? = null,
    val toolCalls: List<ToolCall>? = null,
    val toolCallId: String? = null, // for Role.Tool
)

data class ToolCall(
    val id: String,
    val name: String,
    val argumentsJson: String, // JSON string
)

data class ToolDefinition(
    val name: String,
    val description: String,
    val parametersJson: String, // JSON Schema; map to provider DTO at API edge
)

sealed interface AgentEvent {
    data class Token(val text: String) : AgentEvent
    data class ToolRequests(val calls: List<ToolCall>) : AgentEvent
    data class Finished(val reason: String) : AgentEvent // stop | tool_calls | length | cancel
    data class Error(val error: AppError) : AgentEvent
}

interface Tool {
    val definition: ToolDefinition
    suspend fun execute(argumentsJson: String): String
}
```

Concrete `Tool` classes + a `Map<String, Tool>` registry is enough — no
`ToolFactoryProvider` stacks.

```kotlin
@Module
@InstallIn(SingletonComponent::class)
object AgentModule {
    @Provides
    fun tools(
        weather: WeatherTool,
        readFile: ReadFileTool,
    ): Map<String, Tool> =
        listOf(weather, readFile).associateBy { it.definition.name }
}
```

This explicit provider is faster than custom Hilt map keys for a small tool set.

## Choose the wire protocol first

The harness loop is provider-independent; DTOs and SSE event names are not.
Confirm the interview's endpoint before coding.

| Contract | Use |
|----------|-----|
| OpenAI **Responses** (`/v1/responses`) | Default for a new OpenAI project; current recommended API |
| **Chat Completions** (`/v1/chat/completions`) | OpenAI-compatible providers or when the prompt specifies it |

Responses uses typed output items/events (`response.output_text.delta`,
`function_call`, `function_call_output`) rather than
`choices[].delta.tool_calls`. Keep provider DTOs behind
`ChatCompletionsClient`/`AgentClient` so the harness consumes only
`StreamOutcome`. Consider `store: false` when prompts should not be retained.

The examples below use **Chat Completions** because its raw tool loop is common
across OpenAI-compatible providers.

## Chat Completions wire format

### Request essentials

```json
{
  "model": "MODEL",
  "stream": true,
  "messages": [/* full history */],
  "tools": [{
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "...",
      "parameters": {
        "type": "object",
        "properties": { "city": { "type": "string" } },
        "required": ["city"]
      }
    }
  }],
  "tool_choice": "auto"
}
```

### Non-stream tool call

- `finish_reason`: `"tool_calls"`
- `message.tool_calls[].id`, `.function.name`, `.function.arguments`
- **`arguments` is a JSON string** — `Moshi`/`JSONObject` parse after assemble
- Append the **assistant** message (with `tool_calls`) to history **before** tool results

### Tool result message

```json
{
  "role": "tool",
  "tool_call_id": "call_abc123",
  "content": "{\"temp\":72}"
}
```

`content` is a **string** (often stringified JSON).

### Streaming (SSE)

- Lines: `data: {...}` then `data: [DONE]`
- Concatenate `choices[0].delta.content` into the in-progress assistant bubble
- Merge `delta.tool_calls` by **`index`** (id / name / arguments arrive fragmented)
- `finish_reason: tool_calls` → run tools; `stop` → done

```kotlin
// Pseudocode accumulator
val args = mutableMapOf<Int, StringBuilder>()
val ids = mutableMapOf<Int, String>()
val names = mutableMapOf<Int, String>()
// on each delta.tool_calls entry:
//   ids[index] = id ?: ids[index]
//   names[index] = name ?: names[index]
//   args.getOrPut(index) { StringBuilder() }.append(arguments.orEmpty())
```

Wire with shared OkHttp client from `:core:network`. Parse with Moshi adapters
for chunk types — keep DTOs next to the API, map to `ChatMessage` / `AgentEvent`
in the harness.

### OkHttp SSE reader (the part you don’t want to derive live)

```kotlin
suspend fun streamCompletion(
    request: Request, // POST with JSON body, Accept: text/event-stream
    onToken: (String) -> Unit,
): StreamOutcome = withContext(Dispatchers.IO) {
    val call = client.newCall(request)
    val cancelHandle = currentCoroutineContext().job.invokeOnCompletion { cause ->
        if (cause is CancellationException) call.cancel()
    }
    try {
        call.execute().use { response ->
            if (!response.isSuccessful) throw AppError.Http(response.code)
            val source = response.body.source()
            val text = StringBuilder()
            val toolAccumulator = ToolCallAccumulator()
            var finishReason: String? = null

            while (!source.exhausted()) {
                currentCoroutineContext().ensureActive()
                val line = source.readUtf8Line() ?: break
                if (!line.startsWith("data: ")) continue
                val payload = line.removePrefix("data: ")
                if (payload == "[DONE]") break

                val chunk = chunkAdapter.fromJson(payload) ?: continue
                val choice = chunk.choices.firstOrNull() ?: continue
                choice.delta?.content?.let { text.append(it); onToken(it) }
                choice.delta?.toolCalls?.forEach(toolAccumulator::accept)
                choice.finishReason?.let { finishReason = it }
            }

            if (finishReason == "tool_calls") {
                StreamOutcome.ToolCalls(
                    text.toString().ifEmpty { null },
                    toolAccumulator.build(),
                )
            } else {
                StreamOutcome.Text(text.toString())
            }
        }
    } finally {
        cancelHandle.dispose()
    }
}

sealed interface StreamOutcome {
    data class Text(val text: String) : StreamOutcome
    data class ToolCalls(val assistantText: String?, val calls: List<ToolCall>) : StreamOutcome
}
```

Notes:
- Set `readTimeout` generously or to zero for the streaming client
  (`client.newBuilder().readTimeout(0, MILLISECONDS)`)
- Chunks can arrive with empty `choices` — skip, don’t crash
- Tie coroutine cancellation to `Call.cancel()`; `withContext(IO)` alone does
  not cancel a blocking OkHttp read
- `response.use { }` guarantees the connection closes on completion/error
- OkHttp’s `okhttp-sse` artifact (`EventSources`) is a fine alternative to
  hand-parsing lines; callbacks instead of a loop

## Harness sketch

```kotlin
class AgentHarness @Inject constructor(
    private val api: ChatCompletionsClient, // OkHttp streaming wrapper
    private val tools: Map<String, Tool>,
) {
    private val toolDefs = tools.values.map { it.definition }

    suspend fun run(
        history: MutableList<ChatMessage>,
        userText: String,
        onEvent: (AgentEvent) -> Unit,
    ): Result<Unit> = try {
        history += ChatMessage(Role.User, content = userText)
        var steps = 0
        while (steps++ < MAX_STEPS) {
            currentCoroutineContext().ensureActive()
            val outcome = api.stream(history, tools = toolDefs) { token ->
                onEvent(AgentEvent.Token(token))
            }
            when (outcome) {
                is StreamOutcome.Text -> {
                    history += ChatMessage(Role.Assistant, content = outcome.text)
                    onEvent(AgentEvent.Finished("stop"))
                    return Result.success(Unit)
                }
                is StreamOutcome.ToolCalls -> {
                    history += ChatMessage(
                        Role.Assistant,
                        content = outcome.assistantText,
                        toolCalls = outcome.calls,
                    )
                    onEvent(AgentEvent.ToolRequests(outcome.calls))
                    for (call in outcome.calls) {
                        val tool = tools[call.name]
                        val result = try {
                            tool?.execute(call.argumentsJson)
                                ?: """{"error":"unknown tool"}"""
                        } catch (t: CancellationException) {
                            throw t
                        } catch (t: Throwable) {
                            // Log t; keep tool output valid JSON and non-sensitive.
                            """{"error":"tool execution failed"}"""
                        }
                        history += ChatMessage(
                            Role.Tool,
                            content = result,
                            toolCallId = call.id,
                        )
                    }
                    // loop for model follow-up
                }
            }
        }
        Result.failure(AppError.Message("Tool loop limit reached"))
    } catch (t: CancellationException) {
        throw t
    } catch (t: Throwable) {
        val err = t.toAppError()
        onEvent(AgentEvent.Error(err))
        Result.failure(err)
    }
}
```

Permissions go **inside** the tool execute path or a wrapper before `execute`.

## Permissions

For `read_file` / device / network side effects:

1. Model returns `tool_call`
2. Allowlist and/or user confirmation dialog
3. Deny → tool `content` = `"permission denied"` (model can recover) **or** abort
4. Allow → execute → string result

Never give the model blind access to the filesystem or tokens.

## Compose chat UiState

```kotlin
data class ChatUiState(
    val messages: List<MessageUi> = emptyList(),
    val input: String = "",
    val isStreaming: Boolean = false,
    val error: String? = null,
)

data class MessageUi(
    val id: String,
    val role: Role,
    val text: String,
    val isPending: Boolean = false, // streaming assistant bubble
)
```

- On `Token`: append/update the pending assistant message
- On `Finished`: clear pending / `isStreaming`
- On `Error`: `error = appError.toUserMessage()`; keep prior messages
- Cancel button → cancel the harness `Job`
- Allow one active run per conversation (disable Send or guard with a `Mutex`);
  concurrent runs corrupt message ordering
- Collect in UI with `collectAsStateWithLifecycle()`

## Errors

Same policy as bootstrap:

- Transport/parse failures → `toAppError()` → `Result.failure` / `AgentEvent.Error`
- Tool failures → usually **string error payload** back to the model (continues loop)
  unless the failure is fatal to the app
- User-facing strings via `toUserMessage()`; log the throwable

## Config / security

- API key from `BuildConfig` / `local.properties` — never hardcode
- Prefer a backend proxy in real products; for interview, header
  `Authorization: Bearer …` on OkHttp interceptor is fine
- Don’t log full prompts if they may contain secrets

## Consuming a harness server (WebSocket)

Topology B: server runs the agent; client is a protocol consumer. Define the
wire events as a sealed type and never let raw server/SDK shapes past the
client's network edge.

```kotlin
sealed interface AgentEvent {
    val turnId: String
    data class TurnStarted(override val turnId: String) : AgentEvent
    data class AssistantText(override val turnId: String, val text: String) : AgentEvent
    data class ToolAction(override val turnId: String, val label: String) : AgentEvent
    data class TurnCompleted(
        override val turnId: String,
        val summary: String,
        val speak: String? = null, // spoken content is server-chosen, distinct from display text
    ) : AgentEvent
    data class TurnFailed(override val turnId: String, val message: String) : AgentEvent
    data class TurnCancelled(override val turnId: String) : AgentEvent
}
```

Moshi: `PolymorphicJsonAdapterFactory.of(AgentEventDto::class.java, "type")`
with one DTO per event type; map DTO → sealed event at the client edge.

### OkHttp WebSocket → Flow

```kotlin
class AgentClient @Inject constructor(private val okHttp: OkHttpClient) {
    private var webSocket: WebSocket? = null

    fun connect(url: String): Flow<AgentEvent> = callbackFlow {
        val request = Request.Builder().url(url).build()
        val ws = okHttp.newWebSocket(request, object : WebSocketListener() {
            override fun onMessage(webSocket: WebSocket, text: String) {
                parseEvent(text)?.let { trySend(it) } // skip unknown types, don't crash
            }
            override fun onFailure(ws: WebSocket, t: Throwable, r: Response?) {
                close(t)
            }
            override fun onClosed(ws: WebSocket, code: Int, reason: String) {
                close()
            }
        })
        webSocket = ws
        awaitClose { ws.close(1000, null) }
    }

    fun sendTurn(text: String) { webSocket?.send(turnJson(text)) }
    fun cancel() { webSocket?.send("""{"type":"cancel"}""") }
}
```

Notes:
- `WebSocketListener` delivers **complete text frames** — no SSE line parsing
- Callbacks arrive on OkHttp threads; `callbackFlow` + `collect` in
  `viewModelScope` gets you back to a coroutine context safely
- Reconnect: retry `connect` with backoff in the VM/repo layer; interview scope
  = show the seam, not a full reconnect state machine
- Unknown event `type` → log and skip (forward compatibility)
- Emulator → laptop server: `adb reverse tcp:PORT tcp:PORT`, then use
  `ws://localhost:PORT` (or `10.0.2.2` without adb reverse)

### Turn state machine in the VM

```kotlin
enum class TurnPhase { Idle, WaitingOnAgent, Speaking }
// events fold into UiState:
//   TurnStarted   → WaitingOnAgent; clear transcript for turn
//   AssistantText → append/replace streaming transcript
//   ToolAction    → update "working on…" status line
//   Terminal      → Idle (or Speaking if speak != null); show summary/error
```

Exactly one terminal event per turn keeps this fold total — no "is it done?"
ambiguity. Guard with one-active-turn (disable send while `WaitingOnAgent`).

## Interview vertical slice

**Topology A (client-side loop):**

1. Non-stream fake JSON → parse assistant text into list
2. Real non-stream completion
3. Streaming tokens into one bubble
4. One tool end-to-end (fake `get_weather` is enough)
5. Cancel + error + iteration cap

Stop adding tools/UI chrome until 1–4 work.

**Topology B (harness-server client):**

1. Server first, alone — drive it with `wscat` until events look right
2. Client connect + render `AssistantText` into one bubble (typed input)
3. Full event fold: status line, terminal events, error states
4. Cancel round-trip
5. Only then: voice or other input modalities on top

## Testing (no slop)

- Unit-test **argument accumulation** / message append ordering with fake chunks
- Fake `ChatCompletionsClient` returning scripted `StreamOutcome`s — no MockK theater
- Optional: one test that unknown tool → error JSON tool message

## Pitfalls

- Forgetting assistant `tool_calls` message before `role: tool` results
- Mismatched `tool_call_id`
- Treating `arguments` as a JSON object in the wire model (it’s a **string**)
- Unbounded tool loops
- Updating UI off the main thread
- Coding against Chat Completions event shapes when the supplied contract is
  Responses (or vice versa)
- Parallel `tool_calls`: execute all, append all tool msgs, then one follow-up
  completion (order preserved)

## Related

- [../android-interview-bootstrap/SKILL.md](../android-interview-bootstrap/SKILL.md)
- [../android-compose/SKILL.md](../android-compose/SKILL.md)
- Voice input/output on top of an agent client:
  [../android-voice/SKILL.md](../android-voice/SKILL.md)