Computer Use Bridge

Bridges desktop automation to agent tool calls. Use for: computer use, desktop control, click on screen, GUI automation.

Stijnman Updated 14 repo stars

File contents

Computer Use Bridge

When to Use

  • User says computer use or task matches this capability
  • User says desktop control or task matches this capability
  • User says click on screen or task matches this capability
  • User says GUI automation or task matches this capability

Workflow

  1. Confirm desktop environment and permissions.
  2. Plan actions: click, type, navigate (minimal steps).
  3. Execute via desktop-subagent-connector with screenshots.
  4. Verify outcome visually; retry once on mismatch.

Integrations

  • desktop-subagent-connector
  • hitl-approver

Error Handling

Failure Response
Permission denied Ask user to grant accessibility permissions.

Gotchas

  • Destructive GUI actions require hitl-approver.

Example

Input: User request matching triggers above. Output: Structured result per workflow with integrations invoked as needed.

Stijnman/grok-custom-skills commit e02e6ac4c9

Frequently asked questions

npx get-skill add stijnman/computer-use-bridge