iOS Simulator Skill
Quick Start
# 1. List simulators and find a booted one
python scripts/ios_sim.py list
# 2. Boot (if not already booted)
python scripts/ios_sim.py boot <UDID>
# 3. Build app (if you have source)
python scripts/ios_sim.py build --project MyApp.xcodeproj --scheme MyApp
# 4. Install + launch
python scripts/ios_sim.py install /path/to/MyApp.app --udid <UDID>
python scripts/ios_sim.py launch com.example.MyApp --udid <UDID>
# 5. Inspect the current screen
python scripts/ios_sim.py describe --udid <UDID>
# 6. Tap by element label (recommended — uses live describe-all for coordinates)
python scripts/ios_sim.py tap-element "Sign In" --udid <UDID>
# 7. Tap at explicit coordinates
python scripts/ios_sim.py tap 195 422 --udid <UDID>
# 8. Type text, press Enter
python scripts/ios_sim.py text "hello@example.com" --udid <UDID>
python scripts/ios_sim.py key enter --udid <UDID>
# 9. Scroll / swipe
python scripts/ios_sim.py scroll down --udid <UDID>
python scripts/ios_sim.py swipe 195 700 195 200 --duration 0.5 --udid <UDID>
# 10. Open a URL or deep link
python scripts/ios_sim.py openurl "myapp://home" --udid <UDID>
# 11. Screenshot
python scripts/ios_sim.py screenshot /tmp/screen.png --udid <UDID>
# 12. Set appearance
python scripts/ios_sim.py appearance dark --udid <UDID>
python scripts/ios_sim.py appearance light --udid <UDID>
python scripts/ios_sim.py appearance # get current (uses booted simulator)
describe-all Hook Pattern
Every navigation command (tap, tap-element, swipe, scroll, text, key, button, openurl)
automatically runs idb ui describe-all --json before and after the action.
This achieves two goals:
- Pre-hook: confirms which screen is active and reveals exact element coordinates
- Post-hook: validates the action had the expected effect (screen changed, element appeared/disappeared)
The tap-element command uses the pre-hook describe-all to dynamically locate the element by
AXLabel / title / AXValue and computes its center coordinates — no hardcoded pixel coordinates needed.
See references/describe-all-format.md for the full JSON schema and coordinate math.
All Commands
Simulator Lifecycle
| Command |
Description |
list |
List all available simulators with UDIDs and boot states |
boot <udid> |
Boot a simulator |
shutdown <udid> |
Shut down a simulator |
appearance [light|dark] [--udid U] |
Get or set UI appearance; omit mode to read current value |
App Management
| Command |
Description |
build --project P --scheme S [--udid U] [--derived-data D] |
Build app via xcodebuild for iphonesimulator |
install <app_path> [--udid U] |
Install .app bundle or .ipa |
launch <bundle_id> [--udid U] |
Launch app; shows initial UI state |
terminate <bundle_id> [--udid U] |
Terminate running app |
list-apps [--udid U] |
List installed apps and their running state |
Navigation (all include describe-all hooks)
| Command |
Description |
tap-element <label> [--udid U] |
Preferred: find element by label, tap center |
tap <x> <y> [--duration S] [--udid U] |
Tap at specific coordinates (points) |
swipe <x1> <y1> <x2> <y2> [--duration S] [--delta N] [--udid U] |
Swipe gesture |
scroll <direction> [--distance N] [--speed S] [--udid U] |
Directional scroll (up/down/left/right) |
text <text> [--udid U] |
Type text into focused element |
key <keycode_or_name> [--udid U] |
Press key: enter, backspace, tab, up/down/left/right, or numeric HID code |
button <name> [--udid U] |
Hardware button: HOME, LOCK, SIRI, SIDE_BUTTON, APPLE_PAY |
openurl <url> [--udid U] |
Open URL (http, https, or custom scheme deep link) |
Inspection
| Command |
Description |
describe [--json] [-v] [--udid U] |
Show UI accessibility tree summary (or raw JSON) |
find <label> [--udid U] |
Locate element by label, print frame + tap coordinates |
screenshot <path> [--udid U] |
Save screenshot to file |
Scroll Direction Reference
scroll uses idb ui swipe internally. Directions match the content movement the user sees:
direction |
Finger movement |
Content reveals |
down |
finger swipes up |
content below |
up |
finger swipes down |
content above |
left |
finger swipes right |
content to the left |
right |
finger swipes left |
content to the right |
Adjust --distance (default 300pt) and --speed (default 0.4s) for faster/slower scrolls.
Build Output Location
After build, the .app bundle is at:
<derived-data>/Build/Products/<Configuration>-iphonesimulator/MyApp.app
Default derived-data is /tmp/ios_sim_derived.
For workspaces (CocoaPods/SPM):
python scripts/ios_sim.py build --workspace MyApp.xcworkspace --scheme MyApp
References
references/describe-all-format.md — Full JSON schema for idb ui describe-all output, coordinate math, element field reference, and search patterns. Read this when building custom element-finding logic.
1---2name: ios-simulator3description: Automate iOS simulators: build, install, launch, terminate, set appearance (light/dark mode), and navigate (tap, swipe, scroll, type text, press keys, open URLs, press HOME/LOCK/SIRI) using idb and xcrun simctl. Use when asked to interact with an iOS app in the simulator, automate UI flows, run a navigation sequence, take screenshots, switch dark/light mode, or inspect the UI accessibility tree. All navigation commands use `idb ui describe-all` before and after each action to validate the current UI state and obtain exact tap coordinates.4---56# iOS Simulator Skill78## Quick Start910```bash11# 1. List simulators and find a booted one12python scripts/ios_sim.py list1314# 2. Boot (if not already booted)15python scripts/ios_sim.py boot <UDID>1617# 3. Build app (if you have source)18python scripts/ios_sim.py build --project MyApp.xcodeproj --scheme MyApp1920# 4. Install + launch21python scripts/ios_sim.py install /path/to/MyApp.app --udid <UDID>22python scripts/ios_sim.py launch com.example.MyApp --udid <UDID>2324# 5. Inspect the current screen25python scripts/ios_sim.py describe --udid <UDID>2627# 6. Tap by element label (recommended — uses live describe-all for coordinates)28python scripts/ios_sim.py tap-element "Sign In" --udid <UDID>2930# 7. Tap at explicit coordinates31python scripts/ios_sim.py tap 195 422 --udid <UDID>3233# 8. Type text, press Enter34python scripts/ios_sim.py text "hello@example.com" --udid <UDID>35python scripts/ios_sim.py key enter --udid <UDID>3637# 9. Scroll / swipe38python scripts/ios_sim.py scroll down --udid <UDID>39python scripts/ios_sim.py swipe 195 700 195 200 --duration 0.5 --udid <UDID>4041# 10. Open a URL or deep link42python scripts/ios_sim.py openurl "myapp://home" --udid <UDID>4344# 11. Screenshot45python scripts/ios_sim.py screenshot /tmp/screen.png --udid <UDID>4647# 12. Set appearance48python scripts/ios_sim.py appearance dark --udid <UDID>49python scripts/ios_sim.py appearance light --udid <UDID>50python scripts/ios_sim.py appearance # get current (uses booted simulator)51```5253---5455## describe-all Hook Pattern5657Every navigation command (`tap`, `tap-element`, `swipe`, `scroll`, `text`, `key`, `button`, `openurl`)58automatically runs `idb ui describe-all --json` **before** and **after** the action.5960This achieves two goals:611. **Pre-hook**: confirms which screen is active and reveals exact element coordinates622. **Post-hook**: validates the action had the expected effect (screen changed, element appeared/disappeared)6364The `tap-element` command uses the pre-hook describe-all to dynamically locate the element by65`AXLabel` / `title` / `AXValue` and computes its center coordinates — no hardcoded pixel coordinates needed.6667> See `references/describe-all-format.md` for the full JSON schema and coordinate math.6869---7071## All Commands7273### Simulator Lifecycle7475| Command | Description |76|---|---|77| `list` | List all available simulators with UDIDs and boot states |78| `boot <udid>` | Boot a simulator |79| `shutdown <udid>` | Shut down a simulator |80| `appearance [light\|dark] [--udid U]` | Get or set UI appearance; omit mode to read current value |8182### App Management8384| Command | Description |85|---|---|86| `build --project P --scheme S [--udid U] [--derived-data D]` | Build app via xcodebuild for iphonesimulator |87| `install <app_path> [--udid U]` | Install .app bundle or .ipa |88| `launch <bundle_id> [--udid U]` | Launch app; shows initial UI state |89| `terminate <bundle_id> [--udid U]` | Terminate running app |90| `list-apps [--udid U]` | List installed apps and their running state |9192### Navigation (all include describe-all hooks)9394| Command | Description |95|---|---|96| `tap-element <label> [--udid U]` | **Preferred**: find element by label, tap center |97| `tap <x> <y> [--duration S] [--udid U]` | Tap at specific coordinates (points) |98| `swipe <x1> <y1> <x2> <y2> [--duration S] [--delta N] [--udid U]` | Swipe gesture |99| `scroll <direction> [--distance N] [--speed S] [--udid U]` | Directional scroll (up/down/left/right) |100| `text <text> [--udid U]` | Type text into focused element |101| `key <keycode_or_name> [--udid U]` | Press key: `enter`, `backspace`, `tab`, `up/down/left/right`, or numeric HID code |102| `button <name> [--udid U]` | Hardware button: `HOME`, `LOCK`, `SIRI`, `SIDE_BUTTON`, `APPLE_PAY` |103| `openurl <url> [--udid U]` | Open URL (http, https, or custom scheme deep link) |104105### Inspection106107| Command | Description |108|---|---|109| `describe [--json] [-v] [--udid U]` | Show UI accessibility tree summary (or raw JSON) |110| `find <label> [--udid U]` | Locate element by label, print frame + tap coordinates |111| `screenshot <path> [--udid U]` | Save screenshot to file |112113---114115## Scroll Direction Reference116117`scroll` uses `idb ui swipe` internally. Directions match the **content movement** the user sees:118119| `direction` | Finger movement | Content reveals |120|---|---|---|121| `down` | finger swipes up | content below |122| `up` | finger swipes down | content above |123| `left` | finger swipes right | content to the left |124| `right` | finger swipes left | content to the right |125126Adjust `--distance` (default 300pt) and `--speed` (default 0.4s) for faster/slower scrolls.127128---129130## Build Output Location131132After `build`, the `.app` bundle is at:133```134<derived-data>/Build/Products/<Configuration>-iphonesimulator/MyApp.app135```136Default `derived-data` is `/tmp/ios_sim_derived`.137138For workspaces (CocoaPods/SPM):139```bash140python scripts/ios_sim.py build --workspace MyApp.xcworkspace --scheme MyApp141```142143---144145## References146147- **`references/describe-all-format.md`** — Full JSON schema for `idb ui describe-all` output, coordinate math, element field reference, and search patterns. Read this when building custom element-finding logic.
ninjaproger