Flow Next Drive

Codex note — Browser Use vs this skill: Codex desktop (v0.124+) bundles a Browser Use plugin (invoke $browser-use <task>) controlling its in-app browser. Scope is narrow: localhost, 127.0.0.1, ::1, file://, current in-app tab. No cookies, no auth, no extensions, no production sites, no Electron apps, no mobile sims. For those narrow cases, delegate: use $browser-use directly, or just describe the task in prose (Codex routes natural-language plugin calls). Use this skill (the prose triggers listed above — check the page, verify UI, test this app, etc.) for everything outside that scope — production sites, authenticated flows, cookies/saved sessions, Electron / native apps, iOS Simulator, proxies, headed browsers, video recording, visual diff. In Codex CLI (no desktop app, no in-app browser), always use this skill — Browser Use is not available there.

flow-next-drive — surface-aware UI automation

Drive any UI surface the way a real user would. Whatever driver the environment has, the work is the same shape: observe / navigate → snapshot → act on fresh refs → capture evidence → release. This skill is a router: it detects the surface, picks the highest available driver on a ladder, degrades gracefully when a richer driver is absent, and hands off to a per-rung reference for the command detail.

It orchestrates drivers — it does not reimplement them. The default rung (Vercel's agent-browser CLI) is the only driver assumed present; every other rung is detected and optional. A pass must succeed with whatever the environment actually has — most cloud VMs, Linux, and CI have no Computer Use, so it is never a hard dependency and never on a headless/no-display path.

Driver ladder + universal-flow structure adapted from Ray Fernando's running-bug-review-board skill (Apache-2.0) — see CHANGELOG.

Step 1 — Detect the surface, then branch

Classify the target into one of three buckets and take the matching path. The universal flow (Step 2) is shared; only the actuation and the per-surface reference differ.

How to decide:

• A bare URL, or a dev/staging/prod web app → A.
• A desktop app you can launch with --remote-debugging-port=<n> (or one already exposing one) → B. Electron and Windows WebView2 are Chromium; the web ladder drives them by CDP-attach. Do not route these to Computer Use.
• A desktop app with no CDP port — genuinely native (AppKit/SwiftUI), or a macOS WKWebView / Tauri-on-macOS app — → C. Per-platform caveat: Windows WebView2 is CDP-drivable (→ B); macOS WKWebView generally is not (→ C) — verify per platform.

When unsure whether a desktop app exposes CDP, probe for B first (try to launch/attach with a debug port). If no port is reachable, fall to C.

When .flow/features/ exists, Read .flow/features/README.md and the matching feature files first. They pre-resolve the route, preconditions, and gotchas. Select by **Surface:** plus sub-feature IDs (feature-entry-contract.md). Live detection above remains the fallback when the map is absent or does not cover this target.

Done when

• The target is classified A, B, or C before any driving starts, and the classification is stated. A pass that started acting before naming the surface has broken this.
• A desktop app was probed for a CDP port before being routed to C.
• When .flow/features/ existed, matching feature files were read before driving; live detection was the fallback otherwise.

Step 2 — The universal flow (all surfaces)

observe / list what's open
navigate to the target (URL, or focus the app window)
snapshot              → fresh element refs (after a DOM change; for ONE known target prefer semantic find)
act                   → click / fill / type / press / scroll toward the next step
verify                → expected text/state appeared AND console clean + no failed API/network requests
capture               → screenshot + console/errors at the moment of interest (and on failure)
release               → close the tab / end the session when fully done

verify is not DOM-only — every verify checks the console is clean and no API/network request failed, alongside the expected text or state. A pass declared on a green-looking DOM while a request returned 500 or the console threw an uncaught exception has broken this: that is exactly the silent breakage a real user hits, and the /flow-next:qa qa_verdict rests on this evidence. The tooling is already on the default rung (agent-browser console, agent-browser network requests --filter api; the DevTools-MCP rung has richer inspection). A failed request or console error under a green DOM is a finding, not noise.

Snapshot cost: a full interactive snapshot -i before every act is the dominant token cost of a long flow. Re-snapshot after a DOM change, but for a single known target prefer a semantic locator (find role|text|label … <action> — no snapshot needed), and use snapshot -c / -d <depth> when you only need to verify one region.

Refs (@e1, @e2, …) go stale after any navigation, click, or form submit. Element refs are refreshed by re-snapshotting after any navigation, click, or submit. A "ref not found" or pointer-events: none result reported as a bug before a re-snapshot has broken this — it is a stale snapshot until a fresh one says otherwise.

Done when

• Every act ran against refs from a snapshot taken after the last DOM change (or against a semantic locator that needs none).
• Every verify carries three checks — expected text/state, clean console, no failed API/network request.
• Evidence was captured at the moment of interest and on failure — screenshot plus console/network output — so a downstream /flow-next:qa verdict rests on artifacts rather than narration.
• The session or tab is released when the pass is done. A left-open session or daemon has broken this.

Step 3 — Web ladder (surfaces A and B)

Probe availability top-down and use the highest rung that passes; fail soft to the next; the terminal rung is manual. Never hard-depend on any rung above the default.

Surface B note: an Electron / WebView2 app is driven through this same web ladder, over its CDP debug port. Routing a Chromium-backed desktop app to the native rung has broken this. Attach to the app's remote-debugging port (agent-browser --cdp <port> / --auto-connect; chrome-devtools-mcp --browser-url=http://127.0.0.1:<port>). Launch the app with a dedicated debug port and a dedicated user-data-dir; treat the open debug port as a security exposure (any local app can drive that session).

agent-browser command detail lives in the rung reference, not here. The default-rung reference references/agent-browser.md is the entry point — setup/version check, the universal flow in agent-browser commands, the Chromium-desktop (Electron / WebView2) CDP driver, the --headed daemon-reuse gotcha, and an index into the per-topic references it folds: commands.md, advanced.md (CDP attach), auth.md, snapshot-refs.md, session-management.md, proxy.md, debugging.md.

Step 4 — Native rung (surface C): Cua Driver, then Computer Use

A genuinely native app (or a non-CDP webview) has no browser tab to attach to — the model has to drive the live machine. This rung is provider-agnostic; probe for the best available driver in this order, prefer the highest that passes, degrade to the next:

All share the universal flow (Step 2) — observe → act → verify → capture, described as goal + success state, not pixel coordinates; only the actuation differs. Detect, never assume (command -v, MCP list, uname -s); no native driver is ever a hard dependency. Attended vs headless splits the precedence: on a real display, prefer the background Cua Driver → Computer Use; on a headless/CI host (no screen) the Cua Sandbox is the only native option — the explicit ordering, the local-default/cloud-opt-in split, and provisioning/teardown live in references/cua.md.

→ Read references/cua.md for Cua Driver detection, the install/permission walkthrough (multi-host MCP wiring), the AX-tree driving loop, the macOS permission-split evidence mode, the Native-rung precedence list, licensing, and degradation. → Read references/computer-use.md for Computer Use availability detection, the enable/permission walkthrough, the driving loop, safety/hygiene, and the full graceful-degradation table.

Driver detection & graceful degradation (all surfaces)

Ask the user via plain text. Render the options below as a numbered list 1. … N., followed by a final option N+1. Other — type your own answer. Print the question, then the numbered list, then stop and wait for the user's next message before continuing. Parse the reply as: a bare number 1–N+1 → that option; the literal text of an option label → that option; free text after Other → custom answer.

1. Probe, don't assume. Detect each non-default rung before planning around it (command -v, MCP list, uname -s for the macOS-only paths). Treat anything above the default rung — incl. Cua Driver and Computer Use — as probably absent. On a Cursor host, probe cursor-ide-browser by exact server id at least once before concluding it is absent — catalog omission alone is not a negative, and there is no install step. If that probe fails in an attended session, ask once via plain-text numbered prompt: type @Browser in chat (no space), or open the Browser pane until it shows connected, and confirm Settings → Tools & MCP → Browser Automation is Browser Tab. On portable hosts without that tool, use a numbered prompt with a final Other — type your own answer option. After they confirm, re-probe once. Skip the ask when unattended / autonomous / $CI / FLOW_AUTONOMOUS=1 and degrade. A mid-run MCP server does not exist after this pass already drove the pane is the lease-drop flake, not a first-use miss — do not ask @Browser for that; see references/cursor-ide-browser.md.
2. Pick the highest rung that passes; fail soft to the next. The terminal rung is always manual / documented-limitation — the pass still completes.
3. No native driver is required or on a headless/CI path. Neither the local Cua Driver nor Computer Use runs without a real display; most VMs/Linux/CI lack both. (Headless/CI native driving is the opt-in Cua Sandbox surface — see references/cua.md.)
4. Graceful degradation on the native rung (C): (Determine attended vs headless first — $CI ⇒ headless, else the empirical cua-driver call get_screen_size display probe; NOT $DISPLAY on macOS. See references/cua.md § "Determining headless / CI".)
   • Attended (real display): prefer Cua Driver (background, provider-agnostic) when present → else Computer Use (screen-takeover) → else documented-limitation (document, don't fail).
   • Headless / CI (no display): the Cua Sandbox is the only native option (provision a hermetic VM, drive, tear down each run); local backend is the default, cua.ai cloud is explicit opt-in (bills + egress). No backend and no opted-in cloud → documented-limitation. See references/cua.md.
   • A Chromium-backed app (B) still drives via the web-ladder CDP attach (Step 3), or by driving its local dev-server URL in a browser. Note that shell-level integration (system tray, native menus, OS dialogs) can't be reached this way — surface that limitation.
   • A genuinely native app (C) with no native driver at all → document the limitation rather than fail.
   • On macOS, the Cua Driver's Accessibility-vs-Screen-Recording permission split means driving can work while screenshots don't — surface "AX-only evidence, no screenshot" rather than emit an empty one (references/cua.md).
5. agent-browser stays the only assumed-present driver. No MCP server, Cua Driver, or Computer Use is ever a hard install dependency; flowctl never imports any of them.

Done when

• Every rung above agent-browser that the plan relies on was probed first (command -v, MCP list, uname -s; for cursor-ide-browser, an id-probe — and on an attended Cursor miss, one @Browser ask plus one re-probe). A pass that planned around an unprobed rung has broken this.
• An absent driver degraded to the next rung or to a documented limitation, and the pass still reached a stated outcome rather than ending there.

Boundaries

• iOS / iPadOS app driving is out of scope — the request is declined and deferred to the community iOS simulator skills. A pass that spun up a simulator has broken this.
• This skill provides driver/actuation + the surface conditional. The full native-desktop QA workflow (scenario authoring, bug filing, verdict) is a downstream /flow-next:qa concern.
• Don't reinvent what a driver already does (Playwright, Computer Use) — orchestrate, don't replace.