Browser Control

if command -v npx &>/dev/null; then
  _UK_DIR="${SKILL_DIR:-$(cd "$(dirname "$0")" && pwd)}"
  _UK_JSON=$(npx update-kit check --json --config "$_UK_DIR/../../update-kit.config.json" 2>/dev/null || echo '{}')
  _UK_KIND=$(echo "$_UK_JSON" | python3 -c "import sys,json; print(json.load(sys.stdin).get('kind',''))" 2>/dev/null || echo "")
  if [ "$_UK_KIND" = "upgrade_available" ]; then
    _UK_VER=$(echo "$_UK_JSON" | python3 -c "import sys,json; print(json.load(sys.stdin).get('latest',''))" 2>/dev/null)
    echo "Browser Control Skill update available: $_UK_VER — run: cd ~/.claude/skills/browser-control && npx update-kit apply"
  fi
fi

Browser Control

Control the user's real Chrome browser with full login session inheritance. Multi-channel web access — from lightweight search to full CDP browser automation — with three-layer safety, sub-agent parallel dispatch, and site experience memory.

When to Use

• Access any web page — public or behind corporate authentication (SSO, MFA, security gateways)
• Search, fetch, and extract information from the web
• Automate browser interactions: clicking, form filling, navigation, file upload
• Read content from dynamic pages (SPA, lazy-load, infinite scroll, virtual DOM)
• Write operations: compose messages, submit forms, upload files on web apps
• Rich interactions: React/Vue apps, rich text editors, file pickers, dropdowns
• Parallel research: dispatch sub-agents to investigate multiple targets simultaneously

Companion Sub-Skill: Web Search

For enhanced public web search, the companion web-search sub-skill (powered by omni-search-skill) provides multi-engine search, fetch, resolve, and crawl — stronger than built-in WebSearch/WebFetch.

On first use, check availability:

OMNI_SEARCH_DIR="${OMNI_SEARCH_DIR:-$(find ~ -maxdepth 4 -type d -name 'omni-search-skill' 2>/dev/null | head -1)}"
[ -n "$OMNI_SEARCH_DIR" ] && [ -f "$OMNI_SEARCH_DIR/scripts/omni_search.py" ] && echo "available" || echo "not installed"

If not installed, suggest once:

💡 For enhanced web search, consider installing omni-search-skill: git clone https://github.com/d-wwei/omni-search-skill.git && cd omni-search-skill && python3 -m pip install -r requirements.txt

Do not block on this — built-in multi-channel tools (WebSearch → WebFetch → Jina → curl → CDP) work without it. Only suggest once per session.

Browsing Philosophy

Browse like a human — goal-driven, adaptive, evidence-based.

Do not over-plan steps upfront. Enter with a clear goal, observe the page, decide the next action based on what you see, handle obstacles as they arise, and dig deeper when content is insufficient. Every decision traces back to: "Am I getting closer to what the user needs?"

The Four Phases

1. Understand — Clarify what the user wants. Define success: what information, what action, what result? This anchors all subsequent decisions.

2. Enter — Choose the most direct entry point for the task. Consider platform characteristics and success criteria. If login is needed or static methods are known to fail (e.g., Xiaohongshu, WeChat), go straight to CDP.

3. Verify — Every result is evidence, not just pass/fail. Compare against the success criteria from phase 1. Is the path progressing? Is the quality, relevance, and completeness pointing toward the goal? If direction is wrong, pivot immediately — don't retry the same approach hoping for different results. Search misses don't always mean "wrong query" — they might mean "target doesn't exist." API errors, missing DOM elements, unchanged results after retry: these all signal "reassess direction."

4. Complete — Check against the defined success criteria. Stop when done. Don't over-operate for the sake of "completeness."

Step-by-Step Reasoning (EOPA)

Before each browser action, follow this mental model:

E — EVALUATE: What happened after my last action? Did it succeed?

• If I navigated: did the URL change? Is the page loaded?
• If I clicked: did anything change on the page?
• If I typed: is the text visible in the field?

O — OBSERVE: What does the current page state tell me?

• Read page content or check the current URL
• What interactive elements are available?
• Am I on the right page for the user's goal?

Auto-snapshot rules — decide whether to re-read:

• Last action was read-only (get text, get URL, list elements) → page state unchanged, skip re-read
• Last action was click / navigate / submit / scroll-that-loads → page state likely changed, must re-read
• You already have the exact selector or element index → operate directly, no need to list all elements first
• Never execute the same read command twice consecutively

P — PLAN: What is the single next action needed?

• Do I need to scroll to reveal more content?
• Do I need to wait for something to load?
• Which specific element should I interact with?

A — ACT: Execute exactly ONE browser action, then return to E.

Multi-Channel Web Access

Choose the lightest tool that gets the job done. Escalate only when lighter tools fail.

Jina usage: Prefix URL without http(s):// — e.g., curl -s https://r.jina.ai/example.com. Great for articles, blogs, documentation. May extract wrong sections on dashboards or product pages.

Escalation principle: When multiple search attempts show no quality improvement, escalate to primary sources (official sites, original pages) via CDP.

Information integrity: Search engines are discovery tools, not proof. Primary sources (official sites, original documents, source code) outweigh secondary reports. Multiple media citing the same error creates circular confirmation — always trace to origin.

Programmatic vs GUI Interaction

Inside the browser, two approaches:

• Programmatic (construct URL, eval DOM): Fast and precise when it works, but sites may detect non-human behavior and trigger anti-scraping.
• GUI interaction (click buttons, fill inputs, scroll): Designed for humans — sites never restrict normal UI operations. Reliable fallback.

When programmatic access fails, GUI interaction is the dependable fallback. Site-generated links (DOM hrefs) carry full context; manually constructed URLs may miss implicit parameters.

Platform Detection

Before executing any command, detect the platform:

uname -s

• Darwin → macOS → AppleScript (basic) + CDP Proxy (advanced)
• Linux → CDP Proxy
• MINGW / CYGWIN / MSYS* / Windows_NT** → CDP Proxy (+ optional agent-browser)

Preflight Check

macOS AppleScript Preflight

# Check Chrome is running and JS access is enabled
osascript -e 'tell application "Google Chrome" to get name'
osascript -e 'tell application "Google Chrome" to execute active tab of front window javascript "document.title"'

If the second command fails: Chrome → View → Developer → Allow JavaScript from Apple Events (中文: 查看 → 开发者 → 允许 Apple 事件中的 JavaScript).

CDP Preflight (All Platforms)

bash "${SKILL_DIR}/scripts/check-deps.sh"

The script checks Node.js 22+, Chrome debug port, and starts the CDP Proxy if needed.

Chrome remote debugging setup: Open chrome://inspect/#remote-debugging in Chrome, check "Allow remote debugging for this browser instance", restart Chrome if needed.

Mode Routing

This skill operates in two modes. The /browse command handles routing automatically, but the logic is documented here for all agents.

Two Modes

Foreground (here) — macOS AppleScript, operates on user's visible page

• Targets front window → active tab — the page the user is looking at
• User sees every action in real time
• Zero dependencies beyond Chrome
• Single-page, supervised operations

Background (bg) — CDP Proxy, operates in background tabs

• Creates its own tabs, never touches user's tabs
• User continues browsing normally, zero disruption
• Supports sub-agent parallel dispatch
• Required for advanced write operations (keyboard, upload, hover, isTrusted)

Routing Decision

Platform is macOS?
├─ NO → background (CDP only option)
└─ YES
    ├─ User references current page? ("这个页面", "current tab", etc.)
    │   → foreground
    ├─ Multiple targets or parallel work? ("同时", "simultaneously", etc.)
    │   → background
    ├─ Implies don't-disturb? ("后台", "background", etc.)
    │   → background
    ├─ Needs keyboard/upload/isTrusted/hover?
    │   → background
    └─ DEFAULT → foreground (simpler, zero-dep)

Escalation Within Foreground Mode

If a foreground (AppleScript) operation fails:

1. Try React-compatible event dispatch (still AppleScript)
2. Still fails → auto-start CDP Proxy, switch to CDP for that operation
3. Simple reads can stay on AppleScript even after one operation escalated

Operation-Level Reference

Element Targeting Priority

macOS (AppleScript)

1. Element index — inject List Interactive Elements, then use window.__interactiveElements[index] (most reliable)
2. Text content match — most readable for buttons/links
3. CSS selector — for form inputs, specific elements
4. Coordinates — last resort, fragile across screen sizes

CDP (All Platforms)

1. CSS selector via /click — simple JS click, fast
2. CDP mouse event via /clickAt — trusted, bypasses anti-automation
3. JS eval — for complex targeting logic
4. Coordinates via CDP Input.dispatchMouseEvent — when all else fails

macOS AppleScript Quick Reference

Zero-dependency browser control via macOS native AppleScript. Full reference: modules/applescript-commands.md

# Get current URL
osascript -e 'tell application "Google Chrome" to return URL of active tab of front window'
# Get page text
osascript -e 'tell application "Google Chrome" to execute active tab of front window javascript "document.body.innerText.substring(0, 15000)"'
# Open URL in new tab
osascript -e 'tell application "Google Chrome" to tell front window to make new tab with properties {URL:"https://example.com"}'
# Click by CSS selector
osascript -e 'tell application "Google Chrome" to execute active tab of front window javascript "document.querySelector(\"#submit-btn\").click()"'
# Execute arbitrary JavaScript
osascript -e 'tell application "Google Chrome" to execute active tab of front window javascript "YOUR_JS_CODE_HERE"'

For JXA tab targeting and MutationObserver-based element waiting, see modules/advanced-applescript.md.

CDP Proxy Quick Reference

HTTP-to-CDP bridge proxy for the user's daily Chrome. Full reference: modules/cdp-proxy-api.md

# Create new background tab
curl -s "http://localhost:3456/new?url=https://example.com"
# Execute JS
curl -s -X POST "http://localhost:3456/eval?target=ID" -d 'document.title'
# JS click
curl -s -X POST "http://localhost:3456/click?target=ID" -d 'button.submit'
# Close tab
curl -s "http://localhost:3456/close?target=ID"
# Screenshot
curl -s "http://localhost:3456/screenshot?target=ID&file=/tmp/shot.png"

Setup: bash "${SKILL_DIR}/scripts/check-deps.sh" — proxy listens on localhost:3456.

Security Boundaries (Summary)

Three-layer safety system. Full reference with injectable scripts: modules/safety-system.md

Layer 1 — Domain Blacklist (read-only, no interactions):

• Banking: chase, wellsfargo, bankofamerica, citi, capitalone, usbank, pnc, tdbank, hsbc, any .bank
• Payments: paypal, venmo, stripe, squareup, wise, revolut, robinhood, coinbase, binance
• Identity/Auth: accounts.google.com, login.microsoftonline.com, login.live.com, icloud.com/account, *.okta.com, *.auth0.com, *.onelogin.com
• Cloud: console.aws.amazon.com, console.cloud.google.com, portal.azure.com
• Chrome: chrome://, chrome-extension://, about:

Layer 2 — Element Protection: Never interact with password fields (input[type=password], *[name*=password], *[autocomplete=*-password]). Never click payment buttons (pay, purchase, buy, checkout, place order, subscribe, 付款, 支付, 购买, 下单).

Layer 3 — Operation Confirmation: Confirm with user before: form submissions, submit/send/post/publish/create/delete/remove actions, file uploads to external services, actions on unvisited pages.

General: No untrusted JS in authenticated sessions. chrome:// shadow DOM is inaccessible. Cross-origin iframes cannot be accessed. Confirm before extracting sensitive financial/medical data.

Parallel Research: Sub-Agent Dispatch

Dispatch sub-agents for parallel execution when a task has multiple independent targets.

Full reference: modules/parallel-dispatch.md — load when you have multiple independent targets to research in parallel.

Site Experience Memory

Per-domain knowledge for known sites — load patterns, check experience, record new findings.

Full reference: modules/site-experience.md — load when operating on a site with known patterns.

Login Handling

The user's daily Chrome naturally carries login sessions. Most commonly used sites are already logged in.

Core question: Did I get the target content?

Open the page and try to get content first. Only if content is inaccessible and login would solve it:

"This page requires login to access [specific content]. Please log into [site name] in your Chrome, then tell me to continue."

After login, no restart needed — just refresh the page.

Information Verification

When official source is unavailable, authoritative media (original reporting, not syndicated) serves as secondary evidence — but inform the user: "Official source not found. The following is from [media name] reporting; transcription error possible."

Best Practices

1. Smart wait: Prefer Wait for Element over blind sleep. Fall back to sleep 2 only when no specific condition is available.
2. Scroll before interacting: Off-screen elements may fail to click.
3. Confirm actions succeeded: After navigate → check URL. After click → check content. After fill → verify value.
4. One action at a time: Execute one action, observe, then decide next.
5. Paginate long content: Use .substring(start, end) to read in chunks.
6. Prefer scrolling for lazy-load: Scroll the real page first rather than switching strategies.
7. Avoid redundant reads: Don't re-read after read-only actions. If you know the selector, operate directly.
8. Safety check on unfamiliar pages: Inject the safety check before interacting.
9. Verify each field after filling: Read back values. React/Vue may silently reject el.value.
10. Escalate progressively: AppleScript → CDP Proxy → CDP helper. Only escalate when simpler mode fails.
11. Screenshot before irreversible actions: Capture state before Submit/Send/Delete.
12. Coordinates expire: After scroll/reflow/dropdown, re-query positions before CDP click.
13. Tab hygiene: Never navigate in user's tabs. Create and close your own.
14. Proxy stays running: Don't stop the proxy — restarting requires Chrome re-authorization.

Recovery Strategy

If a browser action fails:

1. Observe: Read content or screenshot to understand current state
2. Re-index: Page may have changed — refresh element references
3. Alternative targeting: CSS selector failed → try text match. Text match failed → try different selector
4. Check for overlays: Dialogs, cookie banners, modals may be blocking — dismiss them first
5. Don't retry blindly: Never repeat the same failed action >2 times. Change approach.

Limitations

• Requires user login: Cannot log in on behalf of the user
• Long content: Scroll first for lazy-load, then paginate output with .substring()
• SPA navigation: Wait for content after clicking navigation elements
• Security: JS executes in the user's authenticated session — never run untrusted scripts
• macOS AppleScript: No screenshot API (use screencapture -l). Chrome must be in foreground.
• CDP: Requires Chrome with remote debugging enabled. Coordinates are viewport-relative.
• File upload: Only <input type="file">. Drag-and-drop zones without file input need DataTransfer API fallback.
• Cross-origin iframes: Many enterprise apps use strict CSP that blocks automation.
• Anti-scraping: Some sites detect automation. Escalate from programmatic to GUI interaction.

Module Index

References