jeanpfs agent-mobile
Automate any mobile app running in an iOS Simulator or Android Emulator. You interact with apps through accessibility tree snapshots (compact text with element refs) and deterministic actions (tap, type, scroll).
Goal
$ARGUMENTS
Prerequisites
Before using any command, verify the environment:
agent-mobile setup
This checks: Maestro CLI installed, simulator/emulator running, app accessible. If it fails, follow the error suggestions.
Install agent-mobile (if missing):
npm install -g @jeanpfs/agent-mobile
If the npm package is not yet published, install from source (the dist is not committed, so a build step is required):
git clone https://github.com/jeanpfs/agent-mobile.git && cd agent-mobile
npm install && npm run build && npm install -g .
Install Maestro (if missing):
curl -Ls install.maestro.dev | bash
Core Workflow
Always follow this loop:
- Snapshot — Capture current UI state
- Decide — Analyze the snapshot, identify target elements by ref
- Act — Execute one action (tap, type, scroll)
- Snapshot — Re-capture to verify the result
- Repeat — Continue until the goal is achieved
CRITICAL: Always snapshot before acting. Refs are invalidated after any action that changes the UI. Never reuse refs from a previous snapshot.
Commands
snapshot — Capture UI state
agent-mobile snapshot [--platform ios|android] [--max 50]
Returns a compact text representation of the screen:
Screen: (12 elements)
- header "Welcome" [ref=m1]
- textbox "Email" [ref=m2]
- textbox "Password" [ref=m3]
- button "Sign In" [ref=m4]
- link "Forgot password?" [ref=m5]
Each [ref=mN] is a unique identifier for that element, valid only for this snapshot.
tap — Tap an element
agent-mobile tap <ref>
Example: agent-mobile tap m4 taps the "Sign In" button.
type — Type text into a field
agent-mobile type <ref> "<text>"
Example: agent-mobile type m2 "user@example.com" types into the Email field. The script taps the field first to focus it, then inputs the text.
scroll — Scroll the screen
agent-mobile scroll <direction>
Directions: up, down, left, right. Default: down. Use when elements aren't visible in the current snapshot.
screenshot — Capture screen image
agent-mobile screenshot [--output path.png]
Saves a PNG screenshot. Use for visual verification when the text snapshot isn't sufficient.
assert — Verify text on screen
agent-mobile assert "<expected text>"
Returns PASS if text is found, FAIL with visible texts for debugging.
logs — Capture device logs
Capture device logs over a time period. Useful for debugging failed requests, crashes, or unexpected behavior.
agent-mobile logs start
agent-mobile logs stop
Start before performing actions that might fail. Stop after to inspect errors, network failures, or crash traces.
Best Practices
- Always snapshot first — Never act without a fresh snapshot
- One action at a time — Tap, then snapshot, then decide next action
- Validate after actions — Use assert or snapshot to confirm results
- Handle missing elements — If an element isn't visible, scroll first
- Report clearly — When something fails, show the user the error and current screen state
- Use logs for debugging — When an action fails unexpectedly, use
logs startbefore retrying to capture device logs - Stop on input failure — If
typeorinputTextfails twice on the same field (custom components, OTP inputs, masked fields), do not keep retrying or inventing workarounds. Instead: stop and ask the user to enter the value manually on the device. Never echo the value back (OTPs, passwords, tokens, card numbers, etc.) — the user already knows what to type. Reference the field by its label or ref (e.g., "please type the OTP into field m3 on the device"), not by its content. Only continue after the user confirms they are done.
Additional resources
- For error codes, platform-specific notes, and troubleshooting, see reference.md