Android Emulator

Android emulator (Flutter)

A bash helper that wraps adb and the qemu emulator console so an AI agent can see (screenshots, accessibility tree) and act on (tap, long-press, swipe, pinch) a Flutter app running on an Android emulator. Computer-use APIs cannot reach the emulator window (qemu has no macOS app bundle), and adb shell input is single-touch only — this script bridges both gaps.

When to use this

The user wants to launch, debug, smoke-test, QA, or screenshot a Flutter app on Android. Reach for this skill before resorting to coordinate-guessing, manual screenshotting via DevTools, or asking the user to run flutter run themselves.

Setup

The script assumes a working Android SDK install (with adb and emulator on $PATH or in ~/Library/Android/sdk/ / ~/Android/Sdk/), at least one AVD created, and Flutter (or fvm) installed. macOS-only utility: sips (used to resample screenshots — ships with macOS). On Linux, swap in convert from ImageMagick if porting.

The script is at scripts/emu.sh relative to this skill. Once the skill is wired into your project (see README.md), agents call it with that relative path.

Quick start (cold boot to interactive)

scripts/emu.sh boot        # start an AVD (cold boot), idempotent
scripts/emu.sh run         # flutter build & install in background (~30–60s first time)
scripts/emu.sh wait-run    # blocks until attached or errors (180s timeout)
scripts/emu.sh health      # sanity check
scripts/emu.sh ui-list     # see what's on screen — start here, not screenshot

run auto-detects the project root (walks up to find pubspec.yaml) and uses fvm flutter if the project pins it. Always pair run with wait-run instead of sleep — the daemon's stdin is gone, so timing is the only signal that the build finished.

Commands

Emulator lifecycle

App control

Input — coordinate-based (all coords in screenshot pixels, 360-wide space)

Input — label-based (preferred — no coordinate guessing)

These read Android's accessibility tree via uiautomator dump and act on the node whose text, content-desc, resource-id, or hint matches the given LABEL. Exact match wins over substring fallback. hint covers empty TextFields (Flutter's InputDecoration.labelText surfaces there).

Coords are in screenshot space. The cx cy columns in ui-list use the same 360-wide frame as the screenshot JPEG and tap X Y, so you can cross-reference the two without rescaling.

Example ui-list output:

 cx   cy  flags          label
 27   70  tap            'Settings'
137  141  tap            'Get started'
 47  254  tap            'New project'
 45  760  tap            'Home\nTab 1 of 4'
135  760  tap            'Search\nTab 2 of 4'
225  760  tap            'Library\nTab 3 of 4'
315  760  tap            'Profile\nTab 4 of 4'

Then tap-label "Search" — substring match is enough; you do not have to include the \nTab 2 of 4 suffix.

Filtering logs by severity

If the app uses package:logging with [F]/[I]/[W]/[S] prefixes, pipe log through grep. Logcat wraps every print() with I/flutter ( PID): , so the level tag isn't at line start — match it after the wrapper:

scripts/emu.sh log 500 | grep -E 'I/flutter.*\[(W|S)\]'    # warnings + severe
scripts/emu.sh log -f  | grep --line-buffered -E 'I/flutter.*\[S\]'   # follow severe

Swap the regex for whatever scheme the app emits (e.g. \[(WARNING|SEVERE)\]).

Choosing screenshot vs ui-list

Screenshots are ~30–60 KB each and add up fast in the conversation context, while ui-list output is ~2 KB. Default to ui-list. If the labels on screen changed, you're on a new screen — that's what most navigation steps need to confirm.

Reach for screenshot only when:

1. Content is inherently visual — a CustomPainter, image/camera preview, color swatches, thumbnails — anything rendered as pixels rather than widgets.
2. Labels don't differentiate — same screen, state change that isn't reflected in the accessibility tree (e.g. a slider dragged to a new value, a toggled chip that re-uses the same text).
3. Debugging a tap-label failure — when a label match fails or taps the wrong thing, a screenshot is faster than reading ui-dump to figure out what's on screen.

Heuristic: after a tap, check ui-list first. Screenshot only if it doesn't answer your question.

Making widgets addressable (Flutter Semantics)

uiautomator dump reads Android's accessibility tree. In a Flutter app, that tree is empty unless the app's semantics tree is exposed. Two ways:

1. Enable semantics in debug builds — add this near the top of main():

   import 'package:flutter/foundation.dart';
   import 'package:flutter/semantics.dart';
   
   void main() {
     if (kDebugMode) {
       SemanticsBinding.instance.ensureSemantics();
     }
     runApp(const MyApp());
   }
   

   Material widgets (BottomNavigationBar, IconButton with tooltip:, TextButton, TextField, etc.) emit Semantics automatically — no per-widget wrapping needed for the nav bar, app-bar buttons, form fields, etc.

2. Enable an accessibility service on the AVD — TalkBack or a similar service forces the platform to materialize the tree. Heavier-handed; option (1) is cleaner.

If ui-list prints (no labelled nodes found — is Flutter semantics enabled?), neither is in effect.

For an icon-only custom widget that needs to be addressable:

Semantics(
  identifier: 'some_stable_id',   // optional, locale-invariant
  label: 'Human readable name',
  child: ...,
)

identifier is preferred for tests because it doesn't change with locale; label is what assistive tech (and tap-label) reads.

Gotchas

• adb shell input swipe X Y X Y MS is not a long-press. It sends DOWN/UP so close together that Flutter's LongPressGestureRecognizer treats it as a cancelled tap and never fires onLongPress. Use hold (or hold-label) — they send a real motionevent DOWN, sleep, motionevent UP pair.
• Pinch does not go through adb. adb shell input is single-touch only. Multi-touch via sendevent /dev/input/event1 requires either root (production AVDs deny adb root) or SELinux permissions the shell user lacks. The pinch command falls back to qemu's host-side console (localhost:5554, auth token at ~/.emulator_console_auth_token), which writes events directly into the virtual input device.
• Without semantics, the app is one opaque rectangle. ui-list on a release build (or a debug build that didn't call ensureSemantics()) returns nothing useful. Fix it in the app, not by guessing coordinates — see the section above.
• ui-list only shows nodes with a label. Unlabelled parents/wrappers won't appear; use ui-dump to see the raw XML if a node you expect is missing.
• Hot-restart vs full rebuild. run always launches a fresh flutter run. If you want hot-reload after a code change, send r\n to the daemon's stdin — but the script backgrounds it (so stdin is gone) and assumes you'll kill-run and run again. For tighter inner loops, run flutter run in your own terminal and use only the input/screenshot commands here.
• First-launch dialogs. Many apps show splash, onboarding, or upsell screens on first launch. Use ui-list to see what's blocking, then tap-label to dismiss (e.g. tap-label "Close", tap-label "Skip"). Don't rely on a fixed sleep — wait until ui-list shows the screen you expect.

Typical workflows

Cold start from nothing

scripts/emu.sh boot
scripts/emu.sh run
scripts/emu.sh wait-run
scripts/emu.sh health
scripts/emu.sh ui-list

Debug something the user is seeing

1. scripts/emu.sh ui-list — what's on screen now?
2. Reproduce the user's path: tap-label "…" for each step.
3. After the suspect action, ui-list again. If it looks right, screenshot only if the bug is visual.
4. scripts/emu.sh log 100 — tail the flutter log to catch exceptions/asserts.

Stop a debug session cleanly

scripts/emu.sh kill-run

Don't try TaskStop from inside the agent — the background task may already be reaped, and either way the app keeps running on the device until force-stopped.

Full teardown

scripts/emu.sh kill-run    # stop flutter + app
scripts/emu.sh exit        # shut down the emulator

Why screenshots are 360px-wide JPEGs

Full-resolution PNGs (1080×2424 on a Pixel-class AVD) are 600 KB–1.5 MB and balloon the conversation context. PNG resize alone barely helps — the encoder isn't aggressive. JPEG q85 at 360px wide gets to ~25–60 KB while staying legible for UI labels and small text. The 360-wide image is the canonical input space for tap/hold/swipe/pinch — the script handles device-side scaling itself, so the resolution is invariant from the caller's point of view. Override the JPEG quality with ANDROID_EMU_SHOT_QUALITY if labels are illegible (try 92) or you need smaller files (try 70).

Concurrent use

Per-invocation scratch paths (suffixed with ANDROID_EMU_TMP_ID, default $$) cover screenshots, UI dumps, and the flutter daemon log + pidfile, so two agents driving different emulator devices don't clobber each other. Pin ANDROID_EMU_DEVICE and ANDROID_EMU_TMP_ID across the full run → wait-run → log → kill-run chain. screenshot echoes its output path — read that, don't hardcode it.

One emulator still hosts only one flutter daemon, so two agents on the same device must coordinate (typically: one owns run/kill-run, both freely log/screenshot/ui-list). Shared state: the emulator input stream (simultaneous taps interleave) and the device-size cache.

Auto-detection

scripts/emu.sh health prints the resolved values — run it first if anything seems off.

Environment overrides