Web Login Automation (anti-bot defenses)
Trigger conditions:
- A login page throws a CAPTCHA / 滑块验证 / 扫码验证 at password login.
- User replies with a scan confirmation ("扫了", "关注了") that belongs to a QR flow — possibly from a different session.
- Need to extract an API token / cookie from a logged-in console.
The ladder (in order — do not skip ahead)
- Recon: navigate to the login page, snapshot, enumerate login methods (密码 / 短信 / 微信登录). Note the post-login target URL (e.g.
/console/settings/token).
- Credentials: if a credentials file exists (e.g.
~/.<site>_credentials), inspect it with masked output only (length + 6-char prefix + 4-char suffix). Confirm the file actually contains what the automation needs — a common stall is the file holding login credentials (phone+password) while the pipeline needs an API token from a console settings page.
- Password route → CAPTCHA appears → run the CAPTCHA assessment ladder (below).
- QR human-in-the-loop (reliable fallback, near-100%): locate the QR
<img> (WeChat flows use mp.weixin.qq.com/cgi-bin/showqrcode?ticket=...), curl -sL it to a local file, send_message it with MEDIA:<path> + clear instructions + validity warning (~2 min), then poll location.href for the redirect (see Polling pattern).
CAPTCHA assessment ladder (before any drag attempt)
- Grep the SDK from page context:
fetch(scriptSrc).then(t => t.text()) then search for isTrusted, pointerId, getCoalescedEvents. 0 hits on isTrusted ⇒ synthetic events are not rejected on trust grounds alone.
- Probe before committing: mousedown + a few moves, then read
getComputedStyle(stripEl).transform / getBoundingClientRect(). If the piece visually moves, synthetic events do drive the UI — continue; if not, stop and go straight to QR.
- Patch environment before the captcha initializes:
Object.defineProperty(navigator,'webdriver',{get:()=>false,configurable:true}) plus fake plugins/languages. Re-apply after every page reload (no CDP persistence via browser tools).
- Synthetic drag — use
templates/slide-captcha-console.js (gap detection + human trajectory). Hard rules:
- Inject
movementX/movementY per event via Object.defineProperty (synthetic events default to 0; some SDKs accumulate deltas from them).
- Dispatch to the slider element, not
document — events dispatched on document propagate upward only and never reach element-level listeners. Element dispatch bubbles up and covers both.
- Fire pointer and mouse events (probe-proven pairing).
- Human trajectory: segmented variable speed, 2-3 micro-pauses, 4-5px overshoot then multi-step settle, y-arc ±1.5px, total 1.2-2s.
- Hard limit: 3-4 failed challenges → pivot to QR. Repeated failures poison the risk session and can flag the account. Do not keep hammering.
Gap detection (slide CAPTCHA)
- Draw bg image (with hole) and piece strip to canvas; compute grayscale gradients (Sobel-lite); normalized cross-correlation of piece-edge energy over bg, sliding only horizontally, restricted to the piece's alpha bounding box. Best offset = drag distance in display px.
- Watch display vs natural size (e.g. bg 296 natural shown at 300 CSS px → scale 300/296). Convert consistently.
- If images are cross-origin URLs and canvas taints, reload them via
new Image(); im.crossOrigin='anonymous' — works when the CDN sends Access-Control-Allow-Origin (aliyuncs does).
Headless constraints (know before involving the user)
- The Hermes browser runs
--headless=new — the user cannot see or touch the window. Never ask them to "drag the slider in the browser window".
- Chrome uses
--remote-debugging-pipe / port 0 — no external CDP websocket, so no trusted-input injection from terminal.
playwright-mcp is stdio-only (no HTTP port) — its browser_drag tool is unreachable from outside; verify with lsof -p <pid> -iTCP -sTCP:LISTEN.
- Multiple Chrome user-data-dirs may exist (agent browser vs playwright-mcp); launch flags like
--disable-blink-features=AutomationControlled may not apply to the one being driven — always verify navigator.webdriver in-page.
- Deep automation fingerprints (CDP side-channels, SwiftShader WebGL renderer) survive in-page patches — this is why the drag ladder has a hard attempt limit.
Taking over a stalled flow from another session
User confirmations for QR/scan flows routinely land in the wrong session ("扫了,关注了" arriving in the chat that didn't send the QR). Reconstruct, don't redo:
session_search discovery on keywords (扫码/二维码/关注), then scroll shape on the found session to read its tail — the last assistant message tells you what it was waiting for.
- Filesystem forensics:
ls -la the expected artifact paths (~/.<site>_credentials, state files, downloaded QR images) and read timestamps — they tell you whether the other session captured anything after the scan.
- Check for leftover polling processes (
ps aux | grep the flow keyword) before assuming the wait is still live.
- Verify the artifact is sufficient for your pipeline (see credentials pitfall in step 2 of the ladder), then take over.
Polling pattern for scan confirmation
Single browser_console eval with an internal loop beats many round-trips:
(async () => { const sleep = ms => new Promise(r => setTimeout(r, ms));
for (let i = 0; i < 12; i++) {
if (!location.pathname.includes('/login')) return JSON.stringify({done:true, url: location.href});
await sleep(2300);
}
return JSON.stringify({done:false, url: location.href});
})()
Credential hygiene
- Never echo full tokens/passwords into chat or files that get sent anywhere. Masked prefix/suffix inspection only.
- Temp files (e.g.
/tmp/.creds.json) must be chmod 600 and deleted immediately after use.
- Write creds via terminal with read-from-variable, not via write_file/heredoc (write paths can redact/corrupt key material).
Pitfalls (paid-for knowledge)
- Mixed content: an HTTPS page cannot
fetch('http://127.0.0.1:...') — the "serve credentials on localhost and fetch from the page" trick always fails with TypeError: Failed to fetch. Don't try; type creds via browser_type or use the QR route.
browser_navigate resets state: a fresh navigation drops un-patched patches and any half-completed captcha — re-apply patches and re-fill forms after any navigation.
- QR expiry ≈ 2 minutes: generate → send → poll in one continuous flow; if expired, regenerate a fresh one rather than telling the user the old one "should still work".
- wx-login interstitial: after the user scans, the page may land on a
wx-login?url=... interstitial before the console redirect — keep polling until the target path appears; one intermediate URL is not "logged in".
- Secrets in tool-call args stay in the local transcript — acceptable for the user's own machine, but prefer masked handling and never paste them into files synced off-device.
References
🗂️ 2026-09-09 curator merge: references/***SECRET***.md (from standalone login-captcha-strategy) — captcha strategy and abort thresholds, AutoDL/Aliyun battle-tested 2026-09-04: headless Chrome cannot pass the Aliyun slider even with isTrusted patched/webdriver hidden/movementX injected; after ≤2 probe attempts pivot immediately to the human path (WeChat QR / SMS / direct token input). Do not burn 5+ rounds.
references/aliyun-slider-captcha-notes.md — Aliyun captcha SDK findings from a real 4-attempt engagement (isTrusted audit, webdriver state, probe results, rejection forensics).
templates/slide-captcha-console.js — copy-paste browser_console script: CORS-safe gap detection + human-like drag with movementX injection.
1---2name: web-login-automation3description: Drive website login flows through the Hermes browser toolset when anti-bot defenses block password login — slider/point CAPTCHAs, WeChat-QR verification, SMS codes. Covers the assessment ladder, headless-browser constraints, synthetic-drag technique, and the human-in-the-loop QR handoff. Also covers taking over a login flow that another session started. Includes the battle-tested captcha strategy and abort thresholds (references/***SECRET***.md — headless cannot pass Aliyun slider; pivot to human path after ≤2 probes; 2026-09-09 curator merge of login-captcha-strategy).4metadata:5 source: https://skillmd.com/skills/openclaw-cn-dev/web-login-automation6---78# Web Login Automation (anti-bot defenses)910Trigger conditions:11- A login page throws a CAPTCHA / 滑块验证 / 扫码验证 at password login.12- User replies with a scan confirmation ("扫了", "关注了") that belongs to a QR flow — possibly from a *different* session.13- Need to extract an API token / cookie from a logged-in console.1415## The ladder (in order — do not skip ahead)16171. **Recon**: navigate to the login page, snapshot, enumerate login methods (密码 / 短信 / 微信登录). Note the post-login target URL (e.g. `/console/settings/token`).182. **Credentials**: if a credentials file exists (e.g. `~/.<site>_credentials`), inspect it with **masked output only** (length + 6-char prefix + 4-char suffix). Confirm the file actually contains what the automation needs — a common stall is the file holding *login* credentials (phone+password) while the pipeline needs an *API token* from a console settings page.193. **Password route** → CAPTCHA appears → run the CAPTCHA assessment ladder (below).204. **QR human-in-the-loop** (reliable fallback, near-100%): locate the QR `<img>` (WeChat flows use `mp.weixin.qq.com/cgi-bin/showqrcode?ticket=...`), `curl -sL` it to a local file, `send_message` it with `MEDIA:<path>` + clear instructions + validity warning (~2 min), then poll `location.href` for the redirect (see Polling pattern).2122## CAPTCHA assessment ladder (before any drag attempt)23241. **Grep the SDK** from page context: `fetch(scriptSrc).then(t => t.text())` then search for `isTrusted`, `pointerId`, `getCoalescedEvents`. 0 hits on `isTrusted` ⇒ synthetic events are *not* rejected on trust grounds alone.252. **Probe before committing**: mousedown + a few moves, then read `getComputedStyle(stripEl).transform` / `getBoundingClientRect()`. If the piece visually moves, synthetic events **do drive the UI** — continue; if not, stop and go straight to QR.263. **Patch environment before the captcha initializes**: `Object.defineProperty(navigator,'webdriver',{get:()=>false,configurable:true})` plus fake `plugins`/`languages`. Re-apply after every page reload (no CDP persistence via browser tools).274. **Synthetic drag** — use `templates/slide-captcha-console.js` (gap detection + human trajectory). Hard rules:28 - Inject `movementX/movementY` per event via `Object.defineProperty` (synthetic events default to 0; some SDKs accumulate deltas from them).29 - Dispatch to the **slider element**, not `document` — events dispatched on `document` propagate *upward only* and never reach element-level listeners. Element dispatch bubbles up and covers both.30 - Fire pointer **and** mouse events (probe-proven pairing).31 - Human trajectory: segmented variable speed, 2-3 micro-pauses, 4-5px overshoot then multi-step settle, y-arc ±1.5px, total 1.2-2s.325. **Hard limit: 3-4 failed challenges → pivot to QR.** Repeated failures poison the risk session and can flag the account. Do not keep hammering.3334## Gap detection (slide CAPTCHA)3536- Draw bg image (with hole) and piece strip to canvas; compute grayscale gradients (Sobel-lite); normalized cross-correlation of piece-edge energy over bg, sliding only horizontally, restricted to the piece's alpha bounding box. Best offset = drag distance in display px.37- Watch display vs natural size (e.g. bg 296 natural shown at 300 CSS px → scale 300/296). Convert consistently.38- If images are cross-origin URLs and canvas taints, reload them via `new Image(); im.crossOrigin='anonymous'` — works when the CDN sends `Access-Control-Allow-Origin` (aliyuncs does).3940## Headless constraints (know before involving the user)4142- The Hermes browser runs `--headless=new` — **the user cannot see or touch the window**. Never ask them to "drag the slider in the browser window".43- Chrome uses `--remote-debugging-pipe` / port 0 — no external CDP websocket, so no trusted-input injection from terminal.44- `playwright-mcp` is stdio-only (no HTTP port) — its `browser_drag` tool is unreachable from outside; verify with `lsof -p <pid> -iTCP -sTCP:LISTEN`.45- Multiple Chrome user-data-dirs may exist (agent browser vs playwright-mcp); launch flags like `--disable-blink-features=AutomationControlled` may not apply to the one being driven — always verify `navigator.webdriver` **in-page**.46- Deep automation fingerprints (CDP side-channels, SwiftShader WebGL renderer) survive in-page patches — this is why the drag ladder has a hard attempt limit.4748## Taking over a stalled flow from another session4950User confirmations for QR/scan flows routinely land in the *wrong* session ("扫了,关注了" arriving in the chat that didn't send the QR). Reconstruct, don't redo:511. `session_search` discovery on keywords (扫码/二维码/关注), then **scroll shape** on the found session to read its tail — the last assistant message tells you what it was waiting for.522. Filesystem forensics: `ls -la` the expected artifact paths (`~/.<site>_credentials`, state files, downloaded QR images) and read **timestamps** — they tell you whether the other session captured anything after the scan.533. Check for leftover polling processes (`ps aux | grep` the flow keyword) before assuming the wait is still live.544. Verify the artifact is *sufficient* for your pipeline (see credentials pitfall in step 2 of the ladder), then take over.5556## Polling pattern for scan confirmation5758Single `browser_console` eval with an internal loop beats many round-trips:5960```js61(async () => { const sleep = ms => new Promise(r => setTimeout(r, ms));62 for (let i = 0; i < 12; i++) {63 if (!location.pathname.includes('/login')) return JSON.stringify({done:true, url: location.href});64 await sleep(2300);65 }66 return JSON.stringify({done:false, url: location.href});67})()68```6970## Credential hygiene7172- Never echo full tokens/passwords into chat or files that get sent anywhere. Masked prefix/suffix inspection only.73- Temp files (e.g. `/tmp/.creds.json`) must be `chmod 600` and deleted immediately after use.74- Write creds via terminal with read-from-variable, not via write_file/heredoc (write paths can redact/corrupt key material).7576## Pitfalls (paid-for knowledge)7778- **Mixed content**: an HTTPS page **cannot** `fetch('http://127.0.0.1:...')` — the "serve credentials on localhost and fetch from the page" trick always fails with `TypeError: Failed to fetch`. Don't try; type creds via `browser_type` or use the QR route.79- **`browser_navigate` resets state**: a fresh navigation drops un-patched patches and any half-completed captcha — re-apply patches and re-fill forms after any navigation.80- **QR expiry ≈ 2 minutes**: generate → send → poll in one continuous flow; if expired, regenerate a fresh one rather than telling the user the old one "should still work".81- **wx-login interstitial**: after the user scans, the page may land on a `wx-login?url=...` interstitial *before* the console redirect — keep polling until the target path appears; one intermediate URL is not "logged in".82- **Secrets in tool-call args stay in the local transcript** — acceptable for the user's own machine, but prefer masked handling and never paste them into files synced off-device.8384## References8586> 🗂️ 2026-09-09 curator merge: `references/***SECRET***.md` (from standalone `login-captcha-strategy`) — captcha strategy and abort thresholds, AutoDL/Aliyun battle-tested 2026-09-04: headless Chrome cannot pass the Aliyun slider even with isTrusted patched/webdriver hidden/movementX injected; after ≤2 probe attempts pivot immediately to the human path (WeChat QR / SMS / direct token input). Do not burn 5+ rounds.8788- `references/aliyun-slider-captcha-notes.md` — Aliyun captcha SDK findings from a real 4-attempt engagement (isTrusted audit, webdriver state, probe results, rejection forensics).89- `templates/slide-captcha-console.js` — copy-paste `browser_console` script: CORS-safe gap detection + human-like drag with movementX injection.