# Focus > focus — show where the agent is looking - Skill: `haumer/focus` (Agent Skill, multi-file: 4 files) - Install (CLI): `npx skillmds@latest add haumer/focus` - Raw SKILL.md: https://api.skillmd.com/api/skills/haumer/focus/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - Author: Haumer (https://skillmd.com/u/haumer) - Updated: 2026-09-22 - Page: https://skillmd.com/skills/haumer/focus --- # focus — show where the agent is looking When a human watches an agent drive a browser, the frustrating part is not the speed, it is the opacity: things change and nobody can tell what the agent saw or intended. This skill adds a thin visual layer on top of browser-harness so every observation and action is announced, located and animated *before* it happens, and then happens for real at exactly that spot. Load it inside any browser-harness driver script: ```python import os exec(open(os.path.expanduser("~/.claude/skills/focus/focus.py")).read()) ``` `focus.py` re-injects `focus.js` into the page on every call, so navigations and SPA re-renders do not lose the overlay. ## The wrappers | Instead of | Use | What the viewer sees | |---|---|---| | `new_tab(url)` / `goto(url)` | `focus_new_tab(url, label)` / `focus_goto(url, label)` | narration bar stays alive across the navigation | | `js("...innerText")` | `focus_read(selector, label)` | spotlight on the region, scan line sweeping it top to bottom, returns the text | | looking at something | `focus_look(selector, label)` | spotlight + cursor glide + "LOOKING" chip | | `click(x, y)` | `focus_click(selector, label)` / `focus_click_at(x, y, label)` | cursor glides, presses, ripple, then the real click lands on the same pixel | | `type_text(text)` | `focus_type(selector, text, label)` | click into the field, then the characters appear one by one with a caret chip | | `press_key("Enter")` | `focus_press("Enter", label)` | narration beat, then the real key | | `scroll(...)` | `focus_scroll(dy, label)` | scrolls in visible steps | | narration | `focus_say(text, mood)` | bottom bar; mood `info`, `warn` (amber, for bugs), `ok` (green, for verified state) | | `wait(4)` after a click | `focus_settled()` / `focus_wait_for(selector)` | nothing; the script just continues the moment the page is ready | | clicking a link to see where it goes | `focus_peek(selector_or_url)` | a dashed PEEKING ring on the link; the page is read ahead without leaving | | screenshot + probing for selectors | `focus_map()` / `print(focus_brief(m))` | a MAPPING chip; returns every visible interactive element with a verified-unique selector | | one action per script | `focus_run([...steps], label)` | a REHEARSING chip while the plan replays in a hidden tab, then each step with its normal choreography | | `screenshot(path)` | `focus_shot(path)` | same, but the overlay is in the frame, so it doubles as evidence | | end of session | `focus_clear()` | overlay fades out | Targets are CSS selectors or `(x, y)` viewport coordinates for canvas UIs. Every wrapper returns the same thing the raw helper would, plus the target rect where useful. ## Always in motion, and honest about it The overlay never freezes, even while the driver is between scripts: the cursor drifts gently around its target and the ring breathes. After about two seconds without a call the agent visibly lets go: ring and spotlight fade, the cursor drifts to a rest spot beside the last target, and the chip reads "THINKING · deciding the next step · 12 s" with a running clock, so the viewer never mistakes an old highlight for a current intention. From then on it looks around: every two seconds or so the cursor glides to a nearby clickable thing and rests a faint dashed ring on it, the way eyes scan a page while deciding, until the next real action snaps the ring back to solid. During a rehearsal the chip counts the steps as they replay ("REHEARSING · 3/6 · click Pricing"). Three more things keep the scene truthful: - **Dialogs are never buried.** When a modal opens and the target is outside it, the spotlight releases; when the modal reaches into the narration bar's zone, the bar moves to the top of the page. - **Navigations keep the scene.** Cursor position, chip, bar text and the chat transcript are snapshotted in the tab and restored the moment the overlay is re-injected, so the first glide on a new page starts from where the cursor was, not from a corner. - **Gone targets release.** If the element under the ring is removed by a re-render, the ring fades instead of framing empty space. ## Survey — show the options before choosing When you are weighing candidates (which button is checkout, which link is the pricing page), do not just click one. Call ```python pick = focus_survey([("#a", "Buy now"), ("nav a[href*=pricing]", "Pricing link"), ("#cta", "Start trial")], "Which of these leads to the price?") ``` The cursor visits each candidate with a "CANDIDATE 2/3" chip and a soft spotlight, so the viewer sees the assessment, not only the verdict. In auto mode it returns `None` and you decide. In "Ask me" mode the chat offers the candidates as buttons and it returns the viewer's index, or `None` for "you decide". ## Peek ahead — read the next page before going there On an unknown journey most of the wall-clock goes into one-action-per-script round trips: click, wait, read, decide, next script. `focus_peek` collapses that. Give it the links you are considering and it fetches them from the tab's own origin (cookies included) in parallel, without navigating: ```python pages = focus_peek(["nav a[href*=pricing]", "nav a[href*=docs]", "https://example.com/faq"], "Which of these has the price list?") # each: {url, status, title, text (first 1500 chars), headings, links, forms, via} ``` The viewer sees a dashed **PEEKING** ring visit each link and a "peeked ahead: …" line in the chat, so reading ahead is as visible as everything else. For JS-rendered pages the fetched HTML is thin; `mode="auto"` then reads the page in a hidden background tab for about a second and closes it (`mode="tab"` forces that). With the peeks in hand, plan the next two or three actions and run them in one script, each followed by `focus_settled()`, so the viewer sees a continuous run instead of stop-and-go. ## Map and run — several steps per script `focus_map()` is the page in one call: `{url, title, headings, text, els}` where each element is `{i, tag, text, sel, rect, href?, type?, name?, value?, options?}` and `sel` has been checked unique in the document (shadow DOM elements carry `shadow: true` and a rect for `focus_click_at`). Read it with `print(focus_brief(m))`, then write the next few steps: ```python r = focus_run([ ("type", "#q", "colmar", "Searching for the destination"), {"kind": "press", "key": "Enter", "expect": "url:results"}, {"kind": "click", "target": "a[href*=details]", "label": "Opening the first result", "expect": "text:Price"}, ("read", ".price", "The price we came for"), ], "Search, open the first result, read its price") print(r["ok"], r["done"], focus_brief(r["map"], 30)) ``` Steps: `click`, `type`, `press`, `goto`, `wait`, `read`, `scroll`, `say`, as tuples or dicts; any step may carry `expect` (`url:…`, `text:…`, `js:…` or a selector) and `timeout`. The plan is first replayed in a hidden tab that shares the session (DOM-level clicks and typing, so it never needs focus). If step k fails there, only the k steps before it are performed visibly and the result has `stopped_at`, `why` and `ahead`, the map of what the failing step actually produced. `rehearse=False` skips the hidden run: use it for anything with side effects, since the hidden tab performs the steps for real. The controlled tab is brought to the front automatically when it is hidden (a hidden tab neither animates nor accepts input). ## Faster Pacing is meant for humans, but the gaps should be yours, not the tool's: - The choreography is tight by default (glide 0.45 s, click 0.24 s, read ≤ 3 s); `focus_speed(2)` halves it again, and the viewer can do the same from the bubble. - Never guess a wait. `focus_settled()` returns when the page DOM has been quiet for 0.3 s; `focus_wait_for("#result")` returns the instant the selector is visible. Both keep polling for Esc, messages and boxes. - Batch. A script is cheap; a decision is not. Map or peek first, then put every step you are already sure about into one `focus_run`. ## The chat bubble — the viewer talks back Bottom-right is a small chat bubble (unread badge, green/amber/red status dot). It opens a compact panel: the transcript of everything you said with `focus_say`, the viewer's messages, and the controls. Every wrapper passes through a gate that honours them, and every pause polls, so interrupts land within about a third of a second: | Control | Effect on the driver | |---|---| | **Pause / Resume** | the next action blocks until Resume. While paused the page is the viewer's: they can log in, fix a form, scroll around. | | **Step** | lets exactly one action through, then pauses again. | | **Stop** | raises `FocusStopped` at the next action; the overlay stays, red dot. | | **Esc** anywhere on the page | exit: the overlay disappears at once and the script ends with `FocusStopped` at its next action. The page is the viewer's. Only if they ask you to carry on, `focus_resume()` brings it back. In P mode Esc cancels the draft box instead, a second Esc leaves the mode. | | **1× / 2× / 0.5×** | changes pacing live (same as `focus_speed`). | | **Ask me** | manual mode: every click, keystroke, typing and navigation posts a "NEXT · CLICK …" card and waits for **Approve** or **Skip**. Skip makes the wrapper return `False` without acting. Also `focus_mode("manual")` or `FOCUS_MODE=manual`. | | **Message box** | raises `FocusMessage` with `.messages`. | ## P — the viewer points at things Pressing **P** anywhere on the page (outside a text field) pauses the agent and enters annotation mode: an amber hint appears, the cursor becomes a crosshair, and the viewer drags boxes over anything they want you to look at, typing a short note per box (Enter commits, Esc cancels). Pressing **P** again sends the boxes and resumes. On the driver side the boxes arrive as `FocusAnnotations`, raised at the next gate or pause, with: - `.boxes`: one dict per box with `n`, `note`, `rect` (viewport), `page` (document coordinates), `element` (tag, id, classes, a CSS selector, its text) for the element under the box centre, and `text`, the visible text inside the box; - `.screenshot`: a PNG taken the moment they arrived, boxes and notes drawn. The boxes stay on the page, dashed, until you answer with `focus_ack("…")`, which posts a green reply and removes them. **Protocol for interrupts.** A driver script is one `browser-harness` run, so `FocusMessage`, `FocusAnnotations` and `FocusStopped` end that run with a readable "FOCUS: …" line in the output. Read it (open the screenshot for boxes), answer with `focus_say` or `focus_ack`, and continue with a new script. Chat state, mode and speed live in the tab and survive navigations, so nothing is lost between scripts. After a pause or takeover, re-assert the page (URL plus a selector) before continuing: the viewer may have moved. No sub-agent is needed: the interrupt lands in your main loop, which is exactly where the next decision is made. ## Rules 1. **Announce, locate, act.** Never act on something the viewer has not seen highlighted. `focus_click` and `focus_type` do this for you. If you fall back to a raw helper, call `focus_look` first. 2. **Say why, not just what.** `focus_say("Checking the price before adding to cart — the listing said €12")` beats `focus_say("Clicking add to cart")`. The chip already names the mechanical action. 3. **Read visibly.** When you extract text for a decision, use `focus_read` on the region so the viewer sees which part of the page the decision came from. 4. **Never fake.** The overlay only decorates. Clicks are real CDP mouse events at the cursor tip; typing is real `Input.insertText`. Do not animate a click without performing it, and do not perform one you did not animate. 5. **Bugs go amber.** Anything unexpected gets `focus_say(..., "warn")` before you work around it, so the viewer never sees a silent retry. 6. **Respect the dock.** Never bypass the gate with raw helpers when a viewer is present. If a message arrives, answer it before doing anything else. 7. **Pace for humans.** Default pacing is about one action per second plus reading time. `focus_speed(2)` for a viewer who knows the flow, `focus_speed(0.6)` for a demo audience. `FOCUS_SPEED=0` turns pauses off for unattended runs while keeping the overlay for the screenshots. 8. **Clear at the end.** `focus_clear()` when the session is over, so the user's tab is left clean. ## Walkthrough integration The `walkthrough` skill (and `/wt`) should load `focus` and use these wrappers in place of its own narration-bar snippet. `focus_say` is the narration bar; `focus_click` / `focus_type` / `focus_read` are the step actions. Everything else in walkthrough (fresh persona, backend verification, bug log, run record) is unchanged. ## Demo ```bash browser-harness < ~/.claude/skills/focus/demo.py ``` Opens Wikipedia in a new tab, surveys three candidates, peeks at two links without leaving, reads the intro, types a search, opens a result, reads its first paragraph, takes evidence screenshots into `/tmp/focus-demo-*.png` and clears the overlay. Press P during the run to try annotation mode. ## Extending `focus.js` is plain DOM and CSS. Colours are two constants (`ACCENT` for the agent, `HUMAN` for the viewer's marks). New choreography goes in as another `window.__focus.` returning a Promise; mirror it with a `focus_` wrapper in `focus.py`. Keep animations under 700 ms and everything `pointer-events: none`, so the overlay can never eat a click meant for the page.