# Computer Use

> Browser automation patterns via the browser_* tools (Playwright). Use when the user asks me to navigate a website, fill a form, scrape data, or test a UI flow.

- Skill: `kevin-liu-01/computer-use` (Agent Skill)
- Install (CLI): `npx skillmds@latest add kevin-liu-01/computer-use`
- Raw SKILL.md: https://api.skillmd.com/api/skills/kevin-liu-01/computer-use/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- License: MIT
- Author: kevin-liu-01 (https://skillmd.com/u/kevin-liu-01)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/kevin-liu-01/computer-use

---


# Computer Use (Browser)

Hermes ships with a `browser_*` toolset backed by Playwright. I can navigate, click, type, scroll, take snapshots, screenshot the page, and read the DOM.

## Workflow

1. **Observe before acting.** Always call `browser_snapshot` first. Don't click on guesses.
2. **Refs are sticky.** A snapshot returns refs you can pass to `browser_click(ref=...)`. Refs invalidate when the DOM mutates -- re-snapshot after navigation, click, type, scroll.
3. **One deliberate step, then verify.** After a click that should change state, snapshot again. After a navigate, wait briefly then snapshot. Don't chain four actions and hope.
4. **Stop after 4 failed attempts.** If repeated tries don't progress, surface the obstacle (login wall, captcha, missing element) instead of trying a fifth time.

## Common patterns

**Login that requires the user.** Stop and report. Don't fabricate credentials, don't bypass passkey prompts. The user manually completes the login, then I resume.

**Forms with autocomplete that change layout.** Type → wait 500ms → snapshot → click the now-correct ref. Never click a ref captured before typing.

**Lazy-loaded lists.** `browser_scroll` to bring an element into view. Use `scrollIntoView: true` rather than scrolling the page repeatedly.

**Multi-tab flows.** Use `browser_tabs(action="list")` to enumerate. Switch deliberately.

**Coordinate clicks.** `browser_mouse_click_xy` requires coordinates from a fresh viewport screenshot taken immediately before the click for the same tab. Any other tool call invalidates the screenshot.

## Anti-patterns

- Polling `browser_snapshot` every 500ms in a loop. The DOM hasn't changed; you're paying for nothing.
- Reusing a ref after a click without re-snapshotting.
- Single 30-second `browser_wait`. Prefer 2s + snapshot + retry; you proceed as soon as the page is ready.
- Trying to interact with iframe content. Hermes can't reach inside iframes. Skip or report.

## When I get stuck

Report:

1. Current URL.
2. What I was trying to reach.
3. The blocker observed (with a snapshot or screenshot).
4. The most likely next action (manual user step, different selector, different page entry).

Don't improvise more clicks; the user makes the call.

## Performance tooling

`browser_profile_start` and `browser_profile_stop` capture a CPU profile to `~/.cursor/browser-logs/`. When the user asks "why is this slow", I profile rather than guess.

## Console + network

`browser_console_messages` and `browser_network_requests` give me the page's JS errors and HTTP traffic. Useful for diagnosing "the button does nothing" -- usually it does something, just throws or returns 500.