# Desktop Automation Protocol

> How to use windows-computer-use-mcp effectively — tool selection, focus rules, safety, and mission design.

- Skill: `hybridlabor-api/desktop-automation-protocol` (Agent Skill)
- Install (CLI): `npx skillmds@latest add hybridlabor-api/desktop-automation-protocol`
- Raw SKILL.md: https://api.skillmd.com/api/skills/hybridlabor-api/desktop-automation-protocol/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: hybridlabor-api (https://skillmd.com/u/hybridlabor-api)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/hybridlabor-api/desktop-automation-protocol

---


# Desktop automation protocol (windows-computer-use-mcp)

## Tool selection guide

| Goal | Tool | Example |
|------|------|---------|
| Click a known button | `automation_elements(click, title="Save")` | fast, needs exact title |
| Click by intent | `automation_smart(click="the Save button")` | slower, finds across all windows |
| Complex multi-step | `automation_mission(run="save file as PDF")` | autonomous loop with retry |
| Record + replay | `automation_macro(record) → stop → replay` | repeatable sequences |
| Wait for event | `automation_watch(start, "window_appears", "Update")` | event-driven triggers |
| Explore desktop | `automation_smart(discover)` | get all running apps |
| Extract + chain | `automation_mission(workflow, actions, store_as, dollar-ref)` | cross-app data pipeline |

## Focus rules
- After automation starts, do not Alt+Tab or touch the mouse
- The target window must stay in foreground during click/type sequences
- If HITL approval appears, approve it and let the target window regain focus
- Prefer reading logs between paused steps or on a second monitor

## Safety
- Read docs/SAFETY.md before production use
- Pair with virtualization-mcp for sandbox/VM isolation when running untrusted automation
- Opt-in features (face, keylogger) are off by default — enable only when needed

## Mission design (automation_mission)
For `run` operation: provide a clear, concrete goal. The server decomposes via LLM sampling.
For `workflow` operation: provide explicit steps with tool, params, label, and optional store_as/dollar-ref for data chaining.
For self-healing: set app_path on steps so the mission can re-launch crashed apps.