# IOS Sim

> Control the iOS Simulator — tap, type, swipe, screenshot, record video, install and launch apps. Use when interacting with the iOS simulator during React Native development. Triggers on "simulator", "tap", "swipe", "screenshot of simulator", "install app", "launch app", "what's on screen", "record video", "accessibility tree", "tap by label", "tap by id", "list elements", "go back", "scroll to top", "scroll to bottom".

- Skill: `ryanthedev/ios-sim` (Agent Skill, multi-file: 6 files)
- Install (CLI): `npx skillmds@latest add ryanthedev/ios-sim`
- Raw SKILL.md: https://api.skillmd.com/api/skills/ryanthedev/ios-sim/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Web & Frontend
- Author: ryanthedev (https://skillmd.com/u/ryanthedev)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/ryanthedev/ios-sim

---


# Skill: ios-sim

**On load:** Read `../../.claude-plugin/plugin.json` from this skill's base directory. Display `ios-sim v{version}` before proceeding.

Control the iOS Simulator through shell scripts wrapping `xcrun simctl` and `AXe`.

```
IMPORTANT: Never load screenshots or accessibility trees in the main context.
Always dispatch a subagent for visual/inspection tasks.
```

---

## Used By

| Skill | What it uses |
|-------|-------------|
| `a11y-audit` | `ui.sh describe-all` for accessibility tree capture |
| `layout-check` | `capture.sh view` + `ui.sh describe-all` for screenshot and element positions |
| `diagnose` | `capture.sh view` to read error text from the simulator screen |
| `deeplink-test` | Screenshot capture to verify the screen after firing a deep link |

---

## Prerequisites

- macOS with Xcode installed
- A booted iOS simulator (`xcrun simctl list devices` to check)
- AXe (`brew install cameroncooke/axe/axe`) — required for all `ui.sh` commands (tap, tap-label, tap-id, type, swipe, describe-all, describe-point, list, back, scroll). `capture.sh view` works without AXe.
- See `${CLAUDE_SKILL_DIR}/references/troubleshooting.md` if anything is missing

## Scripts

All scripts live at `${CLAUDE_SKILL_DIR}/scripts/`. Run them with Bash.

## Routing Table

| Intent | Workflow | Why |
|--------|----------|-----|
| See what's on screen | **view** | Image stays in subagent |
| Find UI elements/coordinates | **inspect** | JSON tree stays in subagent |
| Multi-step UI interaction | **interact** | Entire loop stays in subagent |
| Simple one-shot command | **direct** | No image/tree involved |

### Direct Commands (safe for main context)

| Intent | Script | Example |
|--------|--------|---------|
| Get booted simulator ID | `device.sh booted` | `device.sh booted` |
| Open Simulator app | `device.sh open` | `device.sh open` |
| Save screenshot to file | `capture.sh screenshot <path>` | `capture.sh screenshot /tmp/shot.png` |
| Start video recording | `capture.sh record` | `capture.sh record` |
| Stop video recording | `capture.sh stop` | `capture.sh stop` |
| Install app bundle | `app.sh install <path>` | `app.sh install /path/to/App.app` |
| Launch app by bundle ID | `app.sh launch <id>` | `app.sh launch com.example.app` |
| Tap element by accessibility label | `ui.sh tap-label <label>` | `ui.sh tap-label "Login"` |
| Tap element by accessibility ID | `ui.sh tap-id <id>` | `ui.sh tap-id "submit-button"` |
| List on-screen elements (Controls/Content) | `ui.sh list` | `ui.sh list` |
| Tap the back/navigation button | `ui.sh back` | `ui.sh back` |
| Scroll to top or bottom of list | `ui.sh scroll top\|bottom` | `ui.sh scroll top` |

---

## Workflows

### 1. View (screenshot analysis)

**When:** "What's on the simulator screen?", "How does it look?", "Is there an error?"

Main agent never loads the image. Haiku does the analysis.

```
Dispatch Agent:
  subagent_type: general-purpose
  model: haiku
  description: "ios-sim: analyze screenshot"
  prompt: |
    1. Run: ${CLAUDE_SKILL_DIR}/scripts/capture.sh view
       This outputs a file path to a compressed JPEG.
    2. Read that file path with the Read tool to see the image.
    3. Analyze and return:
       - Overview: What app/screen is visible (1-2 sentences)
       - Key elements: Buttons, text, inputs, navigation items
       - State: Errors, loading, forms filled, current tab
       - Coordinates: Notable interactive elements with approximate point positions
    4. If the user asked something specific, answer that directly.
    Return text only. Be concise.

    USER QUESTION: [insert user's question here]
```

### 2. Inspect (accessibility tree)

**When:** "What elements are on screen?", "Find the login button", "Where should I tap?"

The accessibility tree JSON can be massive. Parse it in a subagent.

```
Dispatch Agent:
  subagent_type: general-purpose
  model: haiku
  description: "ios-sim: inspect UI elements"
  prompt: |
    1. Run: ${CLAUDE_SKILL_DIR}/scripts/ui.sh describe-all
       This outputs the full accessibility tree as JSON.
    2. Parse the JSON and return a structured summary:
       - Screen dimensions (from root frame)
       - Interactive elements: buttons, text fields, switches, links
         Format each as: "Label" [type] at (x, y) — size WxH
       - Current focus/selection state
       - Navigation structure (tabs, headers, back buttons)
    3. If looking for a specific element, report its exact coordinates.
    Return text only. Be concise.

    LOOKING FOR: [insert what the user needs to find]
```

### 3. Interact (multi-step UI automation)

**When:** "Tap the login button", "Fill in the form", "Navigate to settings"

Combines view + inspect + actions in a subagent loop. The entire interaction
stays isolated — main context only gets the final result.

```
Dispatch Agent:
  subagent_type: general-purpose
  description: "ios-sim: UI interaction"
  prompt: |
    You are automating the iOS Simulator. Scripts are at:
    ${CLAUDE_SKILL_DIR}/scripts/

    Available commands:
    - capture.sh view               → compressed screenshot (read the output path to see it)
    - ui.sh describe-all            → full accessibility tree JSON
    - ui.sh describe-point X Y      → element at coordinates
    - ui.sh tap X Y                 → tap at point coordinates
    - ui.sh tap X Y --duration S    → long press
    - ui.sh type "text"             → type ASCII text (max 500 chars)
    - ui.sh swipe X1 Y1 X2 Y2      → swipe gesture
    - ui.sh tap-label "label"       → tap element by accessibility label (no coordinate lookup needed)
    - ui.sh tap-id "id"             → tap element by accessibility ID (no coordinate lookup needed)
    - ui.sh list                    → compact table of on-screen elements grouped by Controls/Content
    - ui.sh back                    → heuristic back-button finder and tap (scores by label/position)
    - ui.sh scroll top|bottom       → repeated swipes with stabilization detection (max 10 swipes)

    TASK: [insert what the user wants to do]

    WORKFLOW:
    1. First capture.sh view to see current state
    2. Use ui.sh describe-all if you need exact coordinates
    3. Perform the requested actions
    4. capture.sh view again to verify the result
    5. Return a text summary of what you did and the final state

    RULES:
    - Use POINT coordinates from the accessibility tree, not pixel coordinates
    - After each action, verify the result before proceeding
    - If something fails, try describe-all to re-orient
    - Return text summary only — do not include base64 image data
```

---

## Tips

- All scripts auto-detect the booted simulator. Pass `--udid <UUID>` to target a specific device.
- Screenshots are 3x pixel resolution. The accessibility tree reports point coordinates. Always use point coordinates for tap/swipe.
- `ui.sh type` only accepts ASCII printable characters (max 500 chars).
- Video recording runs in the background. Use `capture.sh stop` to finish.
- For the **interact** workflow, omit `model` to use the user's current model (better reasoning for complex multi-step tasks).

## Context Efficiency

| Item | Size | In Main Context? |
|------|------|------------------|
| Screenshot JPEG | ~100-300 KB | NEVER — haiku subagent only |
| Accessibility tree JSON | ~10-100 KB | NEVER — subagent only |
| Subagent text summary | ~200-800 chars | YES |
| Direct commands (device, app) | ~50-200 chars | YES |