Ipad CLI

iPad CLI — Real Device & Simulator Automation

Remote control QCut on a real iPad or iOS Simulator from the command line. Build, deploy, navigate, trigger exports, take screenshots, and run E2E tests — all without touching the device.

Quick Reference

Architecture

Mac CLI ──► xcrun devicectl notification post ──► Darwin Notification
                                                        │
                                                        ▼
                                            QCutViewController (Swift)
                                            registerDarwinNotificationBridge()
                                                        │
                                                        ▼
                                            handleDeepLink(url:) ──► webView.evaluateJavaScript()
                                                        │
                                                        ▼
                                            NSLog("[QCut CLI] Result: ...")

Key insight: xcrun devicectl cannot open URLs on real devices (only simulators via simctl openurl). The workaround is Darwin notifications — the Swift app listens for named notifications and maps them to deep link commands.

Device vs Simulator

Step-by-Step: Full E2E on Real iPad

1. Check connection

xcrun xctrace list devices 2>&1 | grep "iPad.*00008"

Look for the UDID (e.g., 00008112-001538520C83601E). If it shows "Connecting", kill stale processes:

sudo pkill -9 -f "pymobiledevice3"
sudo killall -9 remoted 2>/dev/null

Then unplug and replug the USB cable.

2. Build the web app

cd <PROJECT_ROOT>
bun run build
bunx cap sync ios

3. Build the iOS app

cd apps/web/ios/App
xcodebuild -scheme App -destination 'id=<UDID>' -allowProvisioningUpdates build

The built .app bundle lives at:

~/Library/Developer/Xcode/DerivedData/App-*/Build/Products/Debug-iphoneos/App.app

4. Install and launch

xcrun devicectl device install app --device <UDID> \
  ~/Library/Developer/Xcode/DerivedData/App-*/Build/Products/Debug-iphoneos/App.app

xcrun devicectl device process launch --device <UDID> com.qcut.videoeditor

5. Send commands via Darwin notifications

# Navigate to editor (opens first project)
xcrun devicectl device notification post --device <UDID> --name com.qcut.cli.open-editor

# Get editor state
xcrun devicectl device notification post --device <UDID> --name com.qcut.cli.state

# Playback
xcrun devicectl device notification post --device <UDID> --name com.qcut.cli.play
xcrun devicectl device notification post --device <UDID> --name com.qcut.cli.pause

# FPS benchmark
xcrun devicectl device notification post --device <UDID> --name com.qcut.cli.fps

# Console logs
xcrun devicectl device notification post --device <UDID> --name com.qcut.cli.console

# Export status
xcrun devicectl device notification post --device <UDID> --name com.qcut.cli.export-status

# Test share sheet (fake MP4 blob)
xcrun devicectl device notification post --device <UDID> --name com.qcut.test-export

6. Take screenshots (requires tunnel)

# Start tunnel (needs sudo, runs in background)
sudo pymobiledevice3 remote start-tunnel --udid <UDID> --script-mode > /tmp/tunnel_out.txt 2>&1 &
sleep 8
cat /tmp/tunnel_out.txt
# Output: <HOST> <PORT>  (e.g., fd0c:7d3f:3356::1 60660)

# Take screenshot
pymobiledevice3 developer dvt screenshot /tmp/ipad-screenshot.png --rsd <HOST> <PORT>

# View it
open /tmp/ipad-screenshot.png
# Or read it inline (Claude can view images)

7. Clean up tunnel

sudo pkill -f "pymobiledevice3"

Available Darwin Notifications

These are registered in QCutViewController.registerDarwinNotificationBridge():

Deep Link Commands (Simulator only via simctl openurl)

Simulator Workflow

For simulator-only testing (faster iteration, no device needed):

# Boot simulator
xcrun simctl boot "33281421-8240-4977-B61A-27769DF02D04"  # iPad Pro 13-inch
open -a Simulator

# Build for simulator (needs separate build)
xcodebuild -scheme App -destination 'platform=iOS Simulator,id=<SIM-UDID>' build

# Install and launch
xcrun simctl install booted <path-to-Debug-iphonesimulator/App.app>
xcrun simctl launch booted com.qcut.videoeditor

# Send commands directly via URL
xcrun simctl openurl booted "qcut://state"
xcrun simctl openurl booted "qcut://play"
xcrun simctl openurl booted "qcut://panel?panel=ai&subpanel=image"

# Eval arbitrary JS
ENCODED=$(python3 << 'PYEOF'
import urllib.parse
js = 'JSON.stringify({url: location.href, stores: Object.keys(window).filter(k => k.startsWith("__"))})'
print(urllib.parse.quote(js))
PYEOF
)
xcrun simctl openurl booted "qcut://eval?js=$ENCODED"

# Read logs (results appear as NSLog "[QCut CLI] Result: ...")
sleep 2
xcrun simctl spawn booted log show --last 5s --style compact 2>&1 | grep "QCut CLI"

# Screenshot
xcrun simctl io booted screenshot /tmp/sim-screenshot.png

Export Flow on iPad

The export uses navigator.share() which opens the iOS share sheet:

ExportEngineMuxer.export() → Blob
        │
        ▼
saveExportedVideo(blob, filename)
        │
        ├── navigator.share({files: [File]})  ← Primary (opens share sheet)
        │   └── User taps "Save to Files" / "Save Video" / AirDrop / etc.
        │
        ├── Capacitor Filesystem (fallback)   ← Saves to Files > On My iPad > QCut/
        │
        └── <a download> click (last resort)  ← Browser download

Share sheet options on real iPad:

• Save to Files — save to Files app (pick folder)
• AirDrop — send to nearby device
• Copy — copy to clipboard
• Save Video — save to Photos (only for real video blobs, not test data)
• Messages, Notes, etc.

Key Files

Troubleshooting

Device stuck in "Connecting"

sudo pkill -9 -f "pymobiledevice3"
sudo killall -9 remoted 2>/dev/null
# Then unplug and replug USB cable

pymobiledevice3 tunnel won't start

• Needs sudo (creates network interface)
• iPad must trust the computer — look for "Trust This Computer?" dialog
• If stuck on "Waiting user pairing consent", unplug/replug and tap Trust

Build fails with "device is busy"

Kill any pymobiledevice3 tunnels first, then unplug/replug.

Deep links not working on real device

devicectl cannot open URLs — use Darwin notifications instead:

# Wrong (only works on simulator):
xcrun simctl openurl booted "qcut://state"

# Right (works on real device):
xcrun devicectl device notification post --device <UDID> --name com.qcut.cli.state

No "Save Video" in share sheet

Only appears for real video files with valid video data. Test blobs show Copy/Save to Files/More.

Adding New Notifications

To add a new remotely-triggerable command:

1. Add the notification name to the commands array in QCutViewController.registerDarwinNotificationBridge()
2. Add the mapping in handleNotification(name:) — either map to an existing deep link URL or add custom logic
3. Rebuild and deploy to the device

Example — adding a com.qcut.cli.export notification:

// In commands array:
"com.qcut.cli.export",

// In handleNotification:
} else if name == "com.qcut.cli.export" {
    if let url = URL(string: "qcut://export?quality=720p&format=mp4&filename=export") {
        handleDeepLink(url: url)
    }
}

Dependencies