Cua integration overview
You have access to the pi-cua-integration extension which exposes Cua
(trycua/cua) functionality through Pi tools. The extension supports three
modes; the active mode is selected at session start from .pi/cua.jsonc or
the default (local).
Modes
| Mode | Where actions happen | Sandbox | API key required |
|---|---|---|---|
local |
Local Docker / QEMU / Lume VM | yes | no |
localhost |
Your host machine (no sandbox) | no | no |
cloud |
cua.ai cloud sandbox | yes | CUA_API_KEY |
Local mode is the default. It runs everything in a sandbox started on your machine via Docker (XFCE container) for Linux, Lume for macOS guests, or QEMU for full VMs. No account is required.
Workflow
- For
localorcloudmodes: callcua_sandbox_startfirst to get a sandbox name, then call the control tools (cua_screenshot,cua_click, etc.) with thesandboxparameter (or rely on the default sandbox). - For
localhostmode: skipcua_sandbox_start. Control tools target the host directly. - Always end by stopping any sandbox you started, via
cua_sandbox_stop. The extension also auto-cleans on session shutdown.
Tools (10)
| Tool | Purpose |
|---|---|
cua_sandbox_start |
Start (or reconnect) a sandbox |
cua_sandbox_stop |
Destroy a sandbox |
cua_sandbox_list |
List active sandboxes |
cua_screenshot |
Capture PNG screenshot |
cua_click |
Click at (x, y) |
cua_type |
Type text |
cua_key |
Press a key chord (ctrl+s, Return, etc.) |
cua_scroll |
Scroll at coordinates |
Quick recipes
- Screenshot the host (localhost mode):
cua_screenshot() - Open a local Linux sandbox and inspect it:
cua_sandbox_start({ os: "linux", kind: "container" }) cua_screenshot() cua_sandbox_stop({ name: "<returned-name>" }) - Multi-step drive yourself (the main agent loop screenshots + clicks via
cua_screenshot/cua_click/cua_type; no separate ComputerAgent sub-agent is shipped — delegate that flow to thecua-skillglobal skill, which documents thecua do taskCLI for callers that want autonomous delegation).
Companion skills
cua-local-sandbox- Docker/QEMU/Lume runtime detailscua-localhost- Direct host control safety notescua-cloud-sandbox- Cloud sandbox configurationcua-control- Mouse and keyboard primitives reference
Configuration
Project config lives at .pi/cua.jsonc. Global config at ~/.pi/cua.json.
Schema is at schema/cua.schema.json inside the extension package.
The extension activates whenever Pi loads it. The active mode (default
local, sandboxed) is selected from the config; localhost and cloud
must be enabled explicitly.