# Macos Agent Ops > Use Homebrew-installed macos-agent to run repeatable app automation checks and routine tasks on macOS. Use when this capability is needed. - Skill: `tomevault-io/macos-agent-ops` (Agent Skill, multi-file: 2 files) - Install (CLI): `npx skillmds@latest add tomevault-io/macos-agent-ops` - Raw SKILL.md: https://api.skillmd.com/api/skills/tomevault-io/macos-agent-ops/raw - Safety review: pending (external: skill-scanner PASS, skillspector PASS) - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - Author: tomevault-io (https://skillmd.com/u/tomevault-io) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/tomevault-io/macos-agent-ops --- # macos-agent ops ## Contract Prereqs: - macOS host with Accessibility and Automation permissions granted for Terminal and target apps. - Homebrew-installed `macos-agent` available on `PATH`. - `cliclick`, `osascript`, and `im-select` available on `PATH`. - `ABC`/`US` input source enabled in macOS Input Sources. - For full AX surface (`ax attr/action/session/watch`), Hammerspoon runtime should be healthy (`hs` + `hs.ipc`). Inputs: - Command entrypoint: `$AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh`. - Optional env vars: - `MACOS_AGENT_OPS_INPUT_SOURCE_ID` (preferred; example: `abc`) for deterministic input source target. - `MACOS_AGENT_REAL_E2E_INPUT_SOURCE` (legacy fallback) if `MACOS_AGENT_OPS_INPUT_SOURCE_ID` is unset. - `MACOS_AGENT_OPS_SKIP_INPUT_SOURCE_SWITCH=1` to bypass automatic input source switch. - `CODEX_MACOS_AGENT_AX_BACKEND=auto|hammerspoon|applescript` to control AX backend preference. Outputs: - macos-agent JSON/text command output. - Artifacts generated by macos-agent under `$AGENT_HOME/out/` (screenshots, traces, debug bundles, step ledgers). Exit codes: - `0`: success - `1`: runtime failure or missing runtime dependency - `2`: usage error Failure modes: - `wait app-active` timeouts because focus was stolen (Control Center/Spotlight/notifications). - `window.activate` failure when app relaunch/update state is stuck; wrapper now uses `--reopen-on-fail`. - missing `im-select` causes `input-source` failure before keyboard-input actions. - `not authorized` / Apple Events denial from TCC permissions. - AX target mismatch (selector too broad / stale node id) during `ax` operations. - AX gate failures (`--gate-*`) when app/window/selector readiness conditions are not met. - AX postcondition failures (`--postcondition-*`) when state does not change as expected after mutation. - `observe screenshot --if-changed-baseline` path missing or unreadable. - trace directory validation failures when using `--trace`/`--trace-dir` and path is not writable. ## Scripts (only entrypoints) - `$AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh` ## Workflow 1. Resolve Homebrew `macos-agent` binary path: ```bash $AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh where ``` 1. Ensure deterministic input source (default target: `abc`): ```bash $AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh input-source # or explicit source id $AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh input-source --id com.apple.keylayout.ABC ``` 1. Run readiness checks (`preflight` + AX list smoke check): ```bash $AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh doctor # optional target for AX smoke $AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh doctor --ax-app Arc $AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh doctor --ax-bundle-id com.google.Chrome ``` 1. Inspect unified permission state directly (`screen_recording` / `accessibility` / `automation` / `ready`): ```bash $AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh run -- \ --format json preflight --include-probes ``` 1. Run quick app foreground check with reopen recovery: ```bash $AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh app-check --app Finder $AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh app-check --app Arc --timeout-ms 15000 $AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh app-check --bundle-id com.google.Chrome ``` 1. Run AX-only health check (AX tree probe): ```bash $AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh ax-check --app Arc --role AXWindow $AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh ax-check --app Arc --role AXTextField --title-contains Search ``` 1. Use AX wait primitives before mutation to stabilize dynamic UI: ```bash $AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh run -- \ --format json wait ax-present --app Arc --role AXButton --title-contains "Compose" \ --match-strategy contains --timeout-ms 2500 --poll-ms 80 $AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh run -- \ --format json wait ax-unique --app Arc --role AXTextField --title-contains "Search" \ --selector-explain --timeout-ms 2500 --poll-ms 80 ``` 1. Run robust mutating AX actions with gating + postconditions: ```bash $AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh run -- \ --format json ax click --app Arc --role AXButton --title-contains "Play" \ --match-strategy contains --selector-explain --reselect-before-click \ --fallback-order ax-press,ax-confirm,frame-center,coordinate \ --gate-app-active --gate-window-present --gate-ax-unique \ --postcondition-focused true --postcondition-timeout-ms 2000 \ --allow-coordinate-fallback ``` ```bash $AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh run -- \ --format json ax type --app Arc --role AXTextField --title-contains "Search" \ --text "lofi" --clear-first --submit --paste \ --gate-app-active --gate-window-present --gate-ax-present \ --postcondition-attribute AXValue --postcondition-attribute-value "lofi" \ --postcondition-timeout-ms 2000 --allow-keyboard-fallback ``` 1. Use selector-frame screenshots for visual proof of target alignment: ```bash $AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh run -- \ --format json observe screenshot --active-window \ --role AXButton --title-contains "Play" --selector-padding 12 \ --path "$AGENT_HOME/out/macos-agent-selector-$(date +%Y%m%d-%H%M%S).png" ``` 1. Use diff-aware screenshots to avoid noisy artifacts when nothing changed: ```bash $AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh run -- \ --format json observe screenshot --active-window \ --path "$AGENT_HOME/out/macos-agent-state.png" \ --if-changed --if-changed-threshold 2 ``` 1. Use one-shot debug bundle for triage artifacts: ```bash $AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh run -- \ --format json debug bundle --active-window \ --output-dir "$AGENT_HOME/out/macos-agent-debug-$(date +%Y%m%d-%H%M%S)" ``` 1. Run routine scripted scenarios: ```bash $AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh scenario --file /path/to/scenario.json ``` 1. Pass through raw macos-agent commands with JSON errors + trace when needed: ```bash $AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh run -- \ --format json --error-format json --trace \ input click --x 200 --y 160 ``` ## Screenshot-Based Triage Rules - During runtime failure triage, the agent SHOULD capture an active-window screenshot before retry/remediation. - For `wait app-active` and `window.activate` failures, screenshot capture SHOULD be first-line diagnostics. - Screenshot artifacts MUST be written under `$AGENT_HOME/out/` and SHOULD use timestamped filenames. - The agent SHOULD include screenshot path + failing command + error message in the final diagnostic summary. Recommended command (via the skill entrypoint): ```bash $AGENT_HOME/skills/tools/macos-agent-ops/scripts/macos-agent-ops.sh run -- \ --format json observe screenshot --active-window \ --path "$AGENT_HOME/out/macos-agent-failure-$(date +%Y%m%d-%H%M%S).png" ``` ## Common Errors And Prevention - `error: input source mismatch after switch attempt ...` - Run `.../macos-agent-ops.sh input-source --id abc` and verify `current` is `com.apple.keylayout.ABC` or `...US`. - Ensure `im-select` is installed: `brew install im-select`. - If you intentionally need non-ABC IME, set `MACOS_AGENT_OPS_SKIP_INPUT_SOURCE_SWITCH=1`. - `error: timed out waiting for app-active ...` - Keep hands off keyboard/mouse while checks are running. - Close Control Center/Spotlight overlays and reduce notification interruptions. - Increase timeout (`--timeout-ms 12000` or higher) for slow app transitions. - `error: ax.list ... timed out` during `wait ax-present` / `wait ax-unique` - Run `app-check` first, then retry AX waits after app focus is stable. - Increase AX wait timeout/poll (`--timeout-ms`, `--poll-ms`) and narrow selectors. - Keep backend on `auto` unless you need to force Hammerspoon behavior (`CODEX_MACOS_AGENT_AX_BACKEND`). - `window activate failed ...` - Wrapper already uses `--reopen-on-fail`; rerun once to allow quit/relaunch recovery. - For flaky apps (e.g., Spotify updater), kill stale relauncher and retry. - `not authorized` / Apple Events failures - Re-run `doctor` and fix Accessibility/Automation in System Settings. - `selector returned zero AX matches` or ambiguous matches - Narrow selectors and use `--match-strategy exact|prefix|suffix|regex` when `contains` is too broad. - Use `--selector-explain` and `wait ax-unique` before `ax click`/`ax type`. - Prefer `--reselect-before-click` in dynamic pages where node ids churn. - gate failure (`--gate-*`) before mutation - Add/adjust readiness gates based on context (`--gate-app-active`, `--gate-window-present`, `--gate-ax-present`, `--gate-ax-unique`). - Increase gate timeout/poll for slower app transitions (`--gate-timeout-ms`, `--gate-poll-ms`). - postcondition failure (`--postcondition-*`) after mutation - Verify you are asserting the correct target/attribute (`AXValue`, focus state, etc.). - Increase postcondition timeout on animation-heavy apps (`--postcondition-timeout-ms`). - `error: --if-changed-baseline path does not exist ...` - Verify baseline path exists before running capture. - If baseline is optional, remove `--if-changed-baseline` and let current output file be baseline. - trace directory is not writable (`trace.write` errors) - Use a writable custom path with `--trace-dir`. - Prefer `$AGENT_HOME/out/...` for reproducible artifacts. - Homebrew macos-agent missing - Install and link with Homebrew: `brew install macos-agent`. ## Practical Notes - Prefer canonical flags in examples: - `--window-title-contains` (instead of legacy `--window-name`) - `input type --submit` (instead of legacy `--enter`) - Keep `--error-format json` enabled in automation loops when you need machine-parseable failures. - Use `debug bundle` first when a run fails and root cause is unclear. - Parse preflight JSON permissions (`screen_recording` / `accessibility` / `automation` / `ready`) before mutating runs. ## References - `skills/tools/macos-agent-ops/references/e2e-ref-01-preflight-focus.md` - `skills/tools/macos-agent-ops/references/e2e-ref-02-finder-routine.md` - `skills/tools/macos-agent-ops/references/e2e-ref-03-matrix-routine.md` --- > Converted and distributed by [TomeVault](https://tomevault.io/claim/graysurf) — claim your Tome and manage your conversions.