# Web Pilot

> Browser automation — observe-act loop with refs

- Skill: `ddalcu/web-pilot` (Agent Skill)
- Install (CLI): `npx skillmds@latest add ddalcu/web-pilot`
- Raw SKILL.md: https://api.skillmd.com/api/skills/ddalcu/web-pilot/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: ddalcu (https://skillmd.com/u/ddalcu)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/ddalcu/web-pilot

---


# Web Pilot

## Tools

| Tool | Purpose |
|------|---------|
| `sandbox_browser_observe` | Text snapshot: URL, title, headings, textExcerpt, elements with refs |
| `sandbox_browser_navigate` | Go to URL, wait for ready, return snapshot with textExcerpt |
| `sandbox_browser_click` | Click by ref (e.g. "e3") or visible text |
| `sandbox_browser_type` | Type into input by ref (e.g. "e5") |
| `sandbox_browser_content` | Page as markdown (optional CSS selector) |
| `sandbox_browser_evaluate` | Run JS, reports side effects |
| `sandbox_browser_screenshot` | PNG screenshot (expensive, last resort) |
| `sandbox_web_search` | Search DuckDuckGo for URLs |

## Workflow: Observe → Act → Observe

1. **Navigate** → returns snapshot with element refs
2. **Act** using refs from snapshot: `click({ ref: "e3" })` or `type({ ref: "e5", text: "hello" })`
3. **Observe** → verify action worked
4. Repeat. Screenshot only if text can't answer your question.

## Refs

Observe output lists elements like:
```
- button "Submit" [ref=e1]
- textbox [ref=e2] name=email placeholder="Enter email"
- link "Home" [ref=e3]
```
Use refs in click/type: `{ "ref": "e1" }`. Refs update on each observe — always use latest.

## Data Extraction Strategy

1. **Check textExcerpt first** — observe/navigate returns the first 2000 chars of visible text. For many tasks this is enough.
2. **Use content(selector)** over content() — target a specific section (e.g. `{ selector: "main" }` or `{ selector: ".repo-list" }`) to avoid huge dumps.
3. **Use evaluate() for structured extraction** — `evaluate({ expression: "..." })` with JS that returns exactly the data you need (e.g. `[...document.querySelectorAll('.repo')].map(...)`)
4. **Prefer web_fetch for read-only pages** — if you just need page content without interaction, `sandbox_web_fetch` is cheaper than navigating.
5. **Never repeat a failed approach** — if a tool call returns nothing useful, switch to a different strategy immediately.

## Patterns

- **Click by text:** `{ "text": "Accept cookies" }` (when ref unavailable)
- **Scroll:** evaluate `window.scrollBy(0, 500)` then observe
- **Dropdown:** evaluate with value set + change event dispatch