# Waku Computer Use > Control local macOS, Windows, and Linux apps through Waku Computer Use. Prefer purpose-built connectors, APIs, or CLIs when available. - Skill: `egoist/waku-computer-use` (Agent Skill, multi-file: 8 files) - Install (CLI): `npx skillmds@latest add egoist/waku-computer-use` - Raw SKILL.md: https://api.skillmd.com/api/skills/egoist/waku-computer-use/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: egoist (https://skillmd.com/u/egoist) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/egoist/waku-computer-use --- # Waku Computer Use Use the `js` tool from `waku_js_repl` for computer interactions. It runs a persistent QuickJS kernel. Use the direct `cua` methods documented here; all methods are available immediately after bootstrap. Waku shows Cua's native virtual cursor automatically while actions run; no cursor setup is needed. ## Bootstrap Run once per fresh JavaScript session: ```js if (!globalThis.cua) { await setupComputerUseRuntime({ globals: globalThis }); } ``` After `js_reset`, bootstrap again before using `cua`. Module imports and Node subprocess APIs are unavailable. The native runtime is managed by Waku. Use `jsRepl.write(value)` for text or structured output and `await jsRepl.emitImage(image)` to show a returned image. Prefer top-level `var` for names reused across calls. Calls default to 30 seconds; set `timeout_ms` when an observation needs longer. ## API surface Call these methods directly. Arguments use the native names shown below; conditional constraints are validated by the runtime. ```ts type CuaResult = { content: Array<{ type: 'text'; text: string } | { type: 'image'; data: string; mimeType: string }>; structuredContent?: any; isError?: boolean; }; declare const cua: { readonly platform: "macos"; list_apps(args?: Record): Promise; list_windows(args?: { on_screen_only?: boolean; pid?: number }): Promise; get_window_state(args: { capture_mode?: "ax" | "vision"; include_accessibility_tree?: boolean; include_screenshot?: boolean; max_depth?: number; max_dimension?: number; max_elements?: number; pid: number; query?: string; screenshot_out_file?: string; session?: string; window_id: number }): Promise; verify_state(args: { expect: Array<{ element?: { enabled?: boolean | null; exists?: true; selected?: boolean | null; selector: { label_contains?: string; role?: string }; value_equals?: string | null } | null; window?: { bounds?: { height: number; tolerance_px?: number; width: number; x: number; y: number } | null; exists?: boolean | null } | null }>; include_screenshot?: boolean | null; pid: number; session?: string; stable_samples?: number; timeout_ms?: number; window_id: number }): Promise; launch_app(args?: { additional_arguments?: Array; bundle_id?: string; creates_new_application_instance?: boolean; name?: string; urls?: Array; webkit_inspector_port?: number }): Promise; kill_app(args: { pid: number }): Promise; set_window_frame(args: { height: number; pid: number; session?: string; width: number; window_id: number; x: number; y: number }): Promise; invoke_menu(args: { path: Array; pid: number; session?: string; window_id: number }): Promise; click(args?: { action?: string; button?: "left" | "right" | "middle"; count?: number; debug_image_out?: string; delivery_mode?: "background" | "foreground"; element_index?: number; element_token?: string; from_zoom?: boolean; modifier?: Array; pid?: number; scope?: "window" | "desktop"; session?: string; snapshot_id?: string; target?: { kind: "window"; pid: number; window_id: number; [key: string]: unknown } | { display_id: string; kind: "desktop"; [key: string]: unknown }; window_id?: number; x?: number; y?: number }): Promise; double_click(args: { delivery_mode?: "background" | "foreground"; element_index?: number; element_token?: string; pid: number; session?: string; snapshot_id?: string; window_id?: number; x?: number; y?: number }): Promise; right_click(args: { delivery_mode?: "background" | "foreground"; element_index?: number; element_token?: string; modifier?: Array; pid: number; session?: string; snapshot_id?: string; window_id?: number; x?: number; y?: number }): Promise; drag(args: { button?: "left" | "right" | "middle"; delivery_mode?: "background" | "foreground"; duration_ms?: number; from_x: number; from_y: number; from_zoom?: boolean; modifier?: Array; pid?: number; scope?: "window" | "desktop"; session?: string; steps?: number; target?: null; to_x: number; to_y: number; window_id?: number }): Promise; type_text(args: { delay_ms?: number; delivery_mode?: "background" | "foreground"; element_index?: number; element_token?: string; pid?: number; scope?: "window" | "desktop"; session?: string; snapshot_id?: string; target?: null; text: string; window_id?: number; x?: number; y?: number }): Promise; press_key(args: { delivery_mode?: "background" | "foreground"; element_index?: number; element_token?: string; key: string; modifiers?: Array; pid?: number; scope?: "window" | "desktop"; session?: string; snapshot_id?: string; target?: null; window_id?: number; x?: number; y?: number }): Promise; hotkey(args: { delivery_mode?: "background" | "foreground"; element_index?: number; element_token?: string; keys: Array; pid?: number; scope?: "window" | "desktop"; session?: string; snapshot_id?: string; target?: null; window_id?: number; x?: number; y?: number }): Promise; set_value(args: { element_index?: number; element_token?: string; pid: number; session?: string; snapshot_id?: string; value: string; window_id?: number }): Promise; scroll(args: { amount?: number; by?: "line" | "page"; delivery_mode?: "background" | "foreground"; direction: "up" | "down" | "left" | "right"; element_index?: number; element_token?: string; pid?: number; scope?: "window" | "desktop"; session?: string; snapshot_id?: string; target?: null; window_id?: number; x?: number; y?: number }): Promise; clipboard_read(args?: { include_text?: boolean; session?: string }): Promise; clipboard_write(args?: { file_path?: string; image_path?: string; session?: string; text?: string }): Promise; get_screen_size(args?: { session?: string }): Promise; get_desktop_state(args?: { screenshot_out_file?: string; session?: string }): Promise; get_cursor_position(args?: { session?: string }): Promise; move_cursor(args: { cursor_id?: string; scope?: "window" | "desktop"; session?: string; target?: null; x: number; y: number }): Promise; set_agent_cursor_enabled(args: { enabled: boolean; session: string }): Promise; set_agent_cursor_motion(args: { arc_flow?: number | null; arc_size?: number | null; dwell_after_click_ms?: number | null; end_handle?: number | null; glide_duration_ms?: number | null; idle_hide_ms?: number | null; session: string; spring?: number | null; start_handle?: number | null; turn_radius?: number | null }): Promise; set_agent_cursor_theme(args: { reduced_motion?: "auto" | "on" | "off"; session: string; theme_id: string }): Promise; get_agent_cursor_state(args: { session: string }): Promise; check_permissions(args?: { probe_direct_capture?: boolean; prompt?: boolean }): Promise; health_report(args?: { include?: Array; skip?: Array }): Promise; get_config(args?: Record): Promise; set_config(args?: { experimental_pip?: boolean; experimental_pip_geometry?: string; key?: string; max_image_dimension?: number; value?: unknown }): Promise; get_accessibility_tree(args?: Record): Promise; zoom(args: { pid?: number; window_id: number; x1: number; x2: number; y1: number; y2: number }): Promise; page(args: { action: "execute_javascript" | "get_text" | "query_dom" | "click_element" | "insert_text" | "type_keystrokes" | "enable_javascript_apple_events"; attributes?: Array; bundle_id?: string; cdp_port?: number; css_selector?: string; javascript?: string; pid?: number; selector?: string; target_url_contains?: string; text?: string; user_has_confirmed_enabling?: boolean; window_id?: number }): Promise; get_browser_state(args?: { continuation?: string; include_screenshot?: boolean; pid?: number; query?: string; scope_ref?: string; session?: string; snapshot_format?: "dom_refs_v1" | "semantic_v2"; tab_id?: string; target_id?: string; window_id?: number; [key: string]: unknown }): Promise; browser_prepare(args?: { allow_launch?: boolean; pid?: number; profile?: { mode: "isolated_new" | "isolated_named"; name?: string }; session?: string; strategy?: { kind: "existing_profile" }; window_id?: number; [key: string]: unknown }): Promise; browser_navigate(args: { session?: string; tab_id: string; target_id: string; url: string; [key: string]: unknown }): Promise; browser_click(args: { input_route?: "trusted" | "dom_event"; ref?: string; session?: string; tab_id: string; target_id: string; x?: number; y?: number; [key: string]: unknown }): Promise; browser_type(args: { mode?: "insert_text" | "keystrokes"; ref: string; replace?: boolean; session?: string; tab_id: string; target_id: string; text: string; [key: string]: unknown }): Promise; browser_dialog(args: { action: "inspect" | "accept" | "dismiss"; delivery_mode?: "background" | "foreground"; dialog_id?: string; prompt_text?: string; session?: string; tab_id: string; target_id: string; [key: string]: unknown }): Promise; browser_set_input_files(args: { files: Array; ref: string; session?: string; tab_id: string; target_id: string; [key: string]: unknown }): Promise; browser_download(args: { destination_root: string; ref: string; session: string; tab_id: string; target_id: string; [key: string]: unknown }): Promise; browser_pointer(args: { action: "hover" | "right_click" | "double_click" | "scroll" | "drag"; delta_x?: number; delta_y?: number; destination_ref?: string; input_route?: "trusted" | "dom_event"; ref?: string; session: string; tab_id: string; target_id: string; to_x?: number; to_y?: number; x?: number; y?: number; [key: string]: unknown }): Promise; start_recording(args: { output_dir: string; record_video?: boolean }): Promise; stop_recording(args?: Record): Promise; get_recording_state(args?: Record): Promise; replay_trajectory(args: { delay_ms?: number; dir: string; stop_on_error?: boolean }): Promise; install_ffmpeg(args?: { confirm?: boolean }): Promise; start_session(args?: { capture_scope?: "auto" | "window" | "desktop"; cursor_theme?: { reduced_motion?: "auto" | "on" | "off"; theme_id: string; [key: string]: unknown } | null; session?: string; [key: string]: unknown }): Promise; escalate_session(args: { detail?: string; reason: "ax_tree_pixel_mismatch" | "background_delivery_failed" | "foreground_ineffective" | "no_window_target" | "other"; session: string; [key: string]: unknown }): Promise; get_session(args?: { session?: string; [key: string]: unknown }): Promise; list_sessions(args?: { cursor?: string; limit?: number | null; [key: string]: unknown }): Promise; get_session_state(args?: { session?: string; [key: string]: unknown }): Promise; end_session(args?: { session?: string; [key: string]: unknown }): Promise; }; ``` ## Observe an app Use the app and window identities already known from the task or a previous observation. Otherwise, list apps directly and select the requested app: ```js var appList = await cua.list_apps(); jsRepl.write(appList.structuredContent.apps); ``` Select the intended app by its observed name, bundle ID, or launch path. If it is not running, use `cua.launch_app(...)` with the app's observed identity. List its windows and select the intended title: ```js // appInfo is the selected entry from appList.structuredContent.apps. var windows = await cua.list_windows({ pid: appInfo.pid }); jsRepl.write(windows.structuredContent.windows); ``` ```js // windowInfo is selected from windows.structuredContent.windows. var captureArgs = { pid: appInfo.pid, window_id: windowInfo.window_id }; var state = await cua.get_window_state(captureArgs); jsRepl.write(state.structuredContent); for (var image of state.content ?? []) { if (image.type === "image") await jsRepl.emitImage(image); } ``` Each method returns `content`, optional `structuredContent`, and optional `isError`. If `isError` is true, read the error before deciding what to do next. Keep native window IDs intact; do not convert them to 32-bit integers. ## Act and verify Use an `element_token` from the latest window state when available. For pixel actions, use coordinates from that exact window screenshot. Cua handles backing scale; do not resize the image or add the window's screen origin. ```js // button is the intended element from state.structuredContent.elements. var result = await cua.click({ pid: appInfo.pid, window_id: windowInfo.window_id, element_token: button.element_token, delivery_mode: "background", }); jsRepl.write(result); var after = await cua.get_window_state(captureArgs); jsRepl.write(after.structuredContent); ``` Typing and key presses use the same observed window: ```js await cua.type_text({ pid: appInfo.pid, window_id: windowInfo.window_id, text: "hello" }); await cua.press_key({ pid: appInfo.pid, window_id: windowInfo.window_id, key: "return" }); ``` Confirm the intended effect from a fresh state after acting. For asynchronous changes, poll observations with a bounded deadline without repeating the input. Tokens expire after a new snapshot, window, or session. Refresh stale tokens. A truncated or degraded tree may omit controls; use its screenshot and the returned capability information when choosing another route. Prefer background delivery. An unsupported route does not authorize an automatic foreground retry. Windows secure/elevated desktops and Linux compositor restrictions remain in effect. `cua.check_permissions()` reports capabilities; macOS permission prompts belong to Waku Settings > Computer Use. Stop, reset, and disconnect end the helper's work. An interrupted action may have completed, so inspect fresh state before retrying it. One implicit session is reused across calls. If using a named `session`, repeat that label on every call that accepts it.