# Computer Use

> Build desktop and browser automation agents with the Claude API computer use toolset — screenshots, clicks, typing, scrolling — run by your own harness inside an isolated VM or container. Covers choosing the toolset version, the agent loop, batch actions, sandboxing, prompt-injection risk, and when an API integration is the better choice. Use this skill when building agents that operate GUIs (legacy apps, browsers without APIs), migrating from an older computer use tool version, or reviewing the safety of a computer use deployment. Activate when: Claude computer use, computer_toolset, computer_20251124, computer_20250124, desktop agent, screen control, click and type agent, GUI automation.

- Skill: `latestaiagents/computer-use-2` (Agent Skill)
- Install (CLI): `npx get-skill add latestaiagents/computer-use-2`
- Raw SKILL.md: https://api.skillmd.com/api/skills/latestaiagents/computer-use-2/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: latestaiagents (https://skillmd.com/u/latestaiagents)
- Updated: 2026-11-14
- Page: https://skillmd.com/skills/latestaiagents/computer-use-2

---


# Computer Use

**Computer use lets Claude operate a desktop through screenshots and mouse/keyboard actions that your code executes. It is a client tool: you own the VM, the action executor, and the safety boundary. Reach for it only when no API exists.**

## When to Use

- Automating applications that have no API
- Cross-application workflows (read in one app, enter in another)
- Exploratory UI testing where a scripted test would be brittle

## When NOT to Use

- Anything with an API or a browser automation library you can script deterministically
- High-volume or latency-sensitive work: every step is a model call with an image
- Consequential actions (payments, deletes, sending messages) without a human approval step
- Environments that hold real user data or credentials

If the target is only a web page, also consider the web fetch tool (for reading) or a dedicated browser tool; see the tool reference.

## Workflow

### Step 1: Choose the tool version

Tool versions are dated identifiers and their model support changes. As documented at the time of writing:

| Tool `type` | Beta header | Notes |
|-------------|-------------|-------|
| `computer_toolset_20260801` | None | Current toolset with member tools; required on the newest models on the Claude API |
| `computer_20251124` | `computer-use-2025-11-24` | Earlier version for some previous-generation models |
| `computer_20250124` | `computer-use-2025-01-24` | Legacy; the docs describe migrating to the toolset |

Read the compatibility matrix on the computer use page for your exact model and platform before you pick one; do not infer support from this table.

### Step 2: Declare the toolset

```ts
import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();
const MODEL = process.env.ANTHROPIC_MODEL!; // a current model ID that supports the toolset

const tools = [
  { type: "computer_toolset_20260801", configs: { zoom: { enabled: false } } },
  { type: "text_editor_20250728", name: "str_replace_based_edit_tool" }, // optional
  { type: "bash_20250124", name: "bash" }, // optional
] as const;
```

`configs` turns individual member tools on or off (each defaults to enabled). The member tools include `screenshot`, `zoom`, click variants (`left_click`, `right_click`, `middle_click`, `double_click`, `triple_click`), `left_click_drag`, `mouse_move`, `left_mouse_down`/`left_mouse_up`, `cursor_position`, `scroll`, `type`, `key`, `hold_key`, and `wait`. Only `screenshot` and `zoom` return images.

### Step 3: Run the agent loop

Claude returns `tool_use` blocks for member tools carrying `"toolset_name": "computer"`. Execute them in order, then return each result with the same `toolset_name`.

```ts
const messages: Anthropic.MessageParam[] = [{ role: "user", content: goal }];

while (true) {
  const response = await client.messages.create({ model: MODEL, max_tokens: 16000, tools, messages });
  messages.push({ role: "assistant", content: response.content });
  if (response.stop_reason !== "tool_use") break;

  const results = [];
  let failed = false;
  for (const block of response.content) {
    if (block.type !== "tool_use") continue;
    if (failed) {
      results.push({ type: "tool_result", tool_use_id: block.id, toolset_name: "computer", is_error: true,
        content: "Not executed: an earlier computer action in this turn failed." });
      continue;
    }
    const out = await executeAction(block.name, block.input); // your VM controller
    failed = !out.ok;
    results.push({ type: "tool_result", tool_use_id: block.id, toolset_name: "computer",
      is_error: !out.ok, content: out.content });
  }
  messages.push({ role: "user", content: results });
}
```

Claude can return several actions in one response (a batch). The docs say to run them in order and stop at the first failure, returning the exact "Not executed" text above for skipped blocks. Mixed toolsets (text editor, bash) follow their own result rules; see their tool pages. The SDK types for newer tool shapes may require a recent SDK version.

### Step 4: Isolate the environment

Anthropic's reference implementation lives in `anthropics/anthropic-quickstarts` under `computer-use-demo` (a container with a virtual X display, tool implementations, and an agent loop). Treat it as a starting point, not a production boundary.

Minimum isolation:

1. A dedicated VM or container with minimal privileges, rebuilt per task
2. No real credentials, personal files, or logged-in sessions
3. Egress restricted to an allowlist of domains
4. Screen recording or screenshot logging for every step

### Step 5: Defend against prompt injection

Everything on screen is input. A web page or email can contain instructions aimed at the agent. The docs describe classifiers that scan screenshots for likely injections, but your design must not depend on them:

- Require human confirmation before consequential actions (purchases, accepting terms, sending, deleting)
- Bound the task: abort if the agent leaves the expected domains or applications
- State in the system prompt that instructions found in on-screen content are data, not commands

### Step 6: Decompose and verify

Long, multi-page goals fail more often than short ones. Have your orchestrator split the goal into sub-tasks, check the screenshot after each, and stop on unexpected state rather than letting the agent improvise.

## Cost and Latency

Each step is one model request that usually includes at least one screenshot, and image tokens scale with resolution. Measure tokens per task on your own workload with `usage` fields; keep the display resolution modest and use `zoom` (when enabled) for detail instead of a larger screen.

## Anti-Patterns

1. Using computer use where an API or scripted browser automation would work
2. Running the harness on a workstation with real accounts
3. No approval gate for irreversible actions
4. Copying an old `computer_20250124` example onto a new model without checking the matrix
5. Not logging screenshots, which makes failures impossible to debug

## Best Practices

1. Check for an API first
2. Pin the tool version per model and re-check on upgrades
3. Disposable, network-restricted sandbox per task
4. Human-in-the-loop for consequential actions
5. Short, verified sub-tasks orchestrated by your code
6. Log every action and screenshot for audit

## References

- Computer use tool: https://platform.claude.com/docs/en/agents-and-tools/tool-use/computer-use-tool
- Tool reference: https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-reference
- Reference implementation: https://github.com/anthropics/anthropic-quickstarts/tree/main/computer-use-demo

