code-yeongyu Cua control primitives
These tools target the active Cua sandbox (or localhost when in that mode).
Always take a cua_screenshot first when you need coordinates; never
guess pixel positions blindly.
Coordinates
Coordinates are in pixels from the top-left of the target display. The default sandbox display is 1024x768 (XGA) which matches Anthropic's recommended computer-use resolution. Cua handles Retina/HiDPI scaling internally on macOS.
cua_click
cua_click({ x: 320, y: 180 }) // left click
cua_click({ x: 320, y: 180, button: "right" }) // right click
cua_click({ x: 320, y: 180, clicks: 2 }) // double click
Supported buttons: left, right, middle.
cua_type
cua_type({ text: "hello world" })
Types one character at a time on the underlying surface. Newlines in the text are typed as Return.
cua_key
Press a single chord or a sequence of chords:
cua_key({ keys: "Return" })
cua_key({ keys: "ctrl+s" })
cua_key({ keys: ["cmd+space", "Return"] }) // Spotlight + Enter
Common chord names: Return, Escape, Tab, BackSpace, Delete,
Up, Down, Left, Right, Page_Up, Page_Down, Home, End,
F1 .. F12, ctrl+a, ctrl+c, ctrl+v, cmd+s (macOS), super+l
(Linux).
cua_scroll
cua_scroll({ x: 400, y: 300, dy: -5 }) // scroll down
cua_scroll({ x: 400, y: 300, dy: 5 }) // scroll up
cua_scroll({ x: 400, y: 300, dx: 3 }) // scroll right
Use dx / dy wheel deltas. Positive dx means right; negative dy
means down. scrollX / scrollY are still accepted aliases.
Screenshot-driven workflow
The canonical pattern:
1. cua_screenshot() // see the current state
2. <find target visually> // identify x, y of the element
3. cua_click({ x, y }) / cua_type
4. cua_screenshot() // verify the change
5. repeat
Take a fresh screenshot after every state-changing action. The model's internal memory of the screen is unreliable after typing or clicking.