Computer Use Bridge

Bridges desktop automation to agent tool calls. Use for: computer use, desktop control, click on screen, GUI automation.

Stijnman e02e6ac 1.1 KB Updated

File contents

Computer Use Bridge

When to Use

  • User says computer use or task matches this capability
  • User says desktop control or task matches this capability
  • User says click on screen or task matches this capability
  • User says GUI automation or task matches this capability

Workflow

  1. Confirm desktop environment and permissions.
  2. Plan actions: click, type, navigate (minimal steps).
  3. Execute via desktop-subagent-connector with screenshots.
  4. Verify outcome visually; retry once on mismatch.

Integrations

  • desktop-subagent-connector
  • hitl-approver

Error Handling

Failure Response
Permission denied Ask user to grant accessibility permissions.

Gotchas

  • Destructive GUI actions require hitl-approver.

Example

Input: User request matching triggers above. Output: Structured result per workflow with integrations invoked as needed.

Stijnman/grok-custom-skills/tree/main/.grok/skills/computer-use-bridge commit e02e6ac4c9

Frequently asked questions

npx skillmds@latest add stijnman/computer-use-bridge