# Browser Automation

> Browser Automation with Anti-Detection

- Skill: `lucadominguez/browser-automation` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add lucadominguez/browser-automation`
- Raw SKILL.md: https://api.skillmd.com/api/skills/lucadominguez/browser-automation/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: lucadominguez (https://skillmd.com/u/lucadominguez)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/lucadominguez/browser-automation

---

# Browser Automation with Anti-Detection

Use this skill when building any browser automation system that needs to evade bot detection. Covers the core patterns: stealth Playwright setup, fingerprinting, human-like behavior, React form workarounds, and multi-module architecture.

**Support files (core):**
- `references/playwright-ubuntu-patch.md` — Step-by-step for patching Playwright on unsupported distros
- `references/browser-use-setup.md` — browser-use LLM agent framework: WSL setup, API surface, OpenRouter integration, MCP server
- `references/reddit-post-submission.md` — Reddit CDP posting: right Chrome profile, title custom element, flair blocker
- `templates/bu_config.py` — Reusable browser-use config: OpenRouter LLM + headless Chromium with WSL args

**Related skills for extracted domains:**
- `proxy-management` — proxy pools, ProviderEmpire SID rotation, health tracking
- `sms-verification` — SMS provider evaluation, 5sim/SMSPool/sms-activate APIs
- `email-verification` — mail.tm REST API, IMAP polling, code extraction
- `facebook-automation` — FB Marketplace GraphQL, account creation, signup selectors
- `platform-automation` — Skool, Reddit anti-bot, curl_cffi TLS impersonation

## Core Stack

- **Playwright** (Python) — primary browser automation engine
- **playwright-stealth** — pre-built stealth patches (webdriver, chrome.runtime, plugins)
- **fake-useragent** — rotating user agents per profile
- Custom `context.add_init_script()` for additional fingerprint overrides
- **curl_cffi** — TLS impersonation for headless HTTP (see `platform-automation` skill)

## Proxy Architecture

**EXTRACTED to `proxy-management` skill.** Load with `skill_view(name='proxy-management')` for:
- Sticky residential IP assignment patterns
- Provider comparison (Bright Data, IPRoyal, Oxylabs, Proxy-Cheap)
- ProxyEmpire SID rotation
- Health tracking and failure handling
- Geo consistency rules

## Browser Fingerprinting

Each profile needs a unique, consistent fingerprint. The key dimensions:

```python
context_options = {
    "user_agent": unique_per_profile,
    "viewport": {"width": 1920, "height": 1080},  # vary slightly
    "timezone_id": "America/New_York",             # match proxy geo
    "locale": "en-US",
    "geolocation": {"latitude": 40.7, "longitude": -74.0},  # match proxy
    "permissions": ["geolocation"],
}
```

**Additional stealth via init_script** (runs before every page load):
```javascript
Object.defineProperty(navigator, 'webdriver', { get: () => false });
window.chrome = { runtime: {}, loadTimes: function() {}, csi: function() {}, app: {} };
Object.defineProperty(navigator, 'plugins', { get: () => [1, 2, 3, 4, 5] });
Object.defineProperty(navigator, 'languages', { get: () => ['en-US', 'en'] });
```

**Launch args for Chromium:**
```
--no-sandbox
--disable-blink-features=AutomationControlled
--disable-dev-shm-usage
--disable-features=IsolateOrigins,site-per-process
```

## Human-Like Behavior

The difference between getting banned and staying alive is behavior simulation:

- **Typing**: character-by-character with random delays (30-150ms per char), occasional typos + correction (10% chance)
- **Clicking**: move mouse to random position within element bounds, brief hover, then click
- **Scrolling**: random distance (200-800px), smooth behavior
- **Delays**: random.uniform(0.3, 2.0) between actions; longer delays (1-3s) before navigation
- **Variance**: no two profiles should behave identically — vary post timing ±30min, engagement volume, scroll distances

## Account Warming Protocol

New accounts need 3-4 weeks of human-like activity before aggressive posting:

```
Week 1: Browse only. Follow 10-20/day. Like posts. No original content.
Week 2: Reply to others. 1 post/day.
Week 3: 2 posts/day. Start using hashtags.
Week 4+: Full cadence.
```

## Playwright on Unsupported Distros

When Playwright's host platform check rejects your OS (e.g., new Ubuntu releases), the fix is patching the bundled coreBundle.js. Three locations may need patching:

1. **Python venv**: `venv/lib/python*/site-packages/playwright/driver/package/lib/coreBundle.js`
2. **Local npm**: `node_modules/playwright-core/lib/coreBundle.js`
3. **Global npm cache**: `/tmp/node_modules/playwright-core/lib/coreBundle.js`

The version check logic (~line 7749) maps Ubuntu major versions to supported host platforms. When your version falls through to the default case (which returns an unrecognized platform string like `ubuntu26.04-x64`), the browser download fails. Patch the condition to treat your version as the closest supported one:

```javascript
// BEFORE (fails on Ubuntu 26.04+):
if (major < 24)
    return { hostPlatform: "ubuntu22.04" + archSuffix, ... };
if (major < 26)
    return { hostPlatform: "ubuntu24.04" + archSuffix, ... };
return { hostPlatform: "ubuntu" + distroInfo.version + archSuffix, isOfficiallySupportedPlatform: false };

// AFTER (maps 24.x+ to ubuntu24.04 which has compatible binaries):
if (major < 24)
    return { hostPlatform: "ubuntu22.04" + archSuffix, ... };
return { hostPlatform: "ubuntu24.04" + archSuffix, isOfficiallySupportedPlatform: false };
```

Then set `PLAYWRIGHT_BROWSERS_PATH=./browsers` and install:
```bash
PLAYWRIGHT_BROWSERS_PATH=./browsers npx playwright install chromium
```

## Live-Platform Selector Discovery

When automating a site you haven't tested live, the CSS selectors in your code are GUESSES. The correct approach is a two-phase discovery:

**Phase 1: Headless reconnaissance** — Launch headless, navigate to the target page, dump all input/button attributes:
```python
page.goto("https://target.com/signup", wait_until="networkidle")
time.sleep(3)
# Get ALL attributes from every input
inputs = page.locator("input").all()
for i in inputs:
    attrs = page.evaluate('''(el) => {
        const r = {};
        for (const a of el.attributes) r[a.name] = a.value;
        return r;
    }''', i.element_handle())
    print(attrs)
```

**Phase 2: Flow discovery** — Modern SPAs often hide forms behind CTAs. Click buttons, wait for modals, re-check the DOM:
```python
# Click the CTA that opens the real form
page.locator("button:has-text('Get Started')").first.click()
time.sleep(3)
# Now check for inputs again — they may appear in a modal
```

For platform-specific verified selectors (Skool, Reddit, Facebook), load `skill_view(name='platform-automation')`.

## Email Verification

**EXTRACTED to `email-verification` skill.** Load `skill_view(name='email-verification')` for mail.tm REST API patterns, IMAP polling for Gmail/custom domains, and code/link extraction from verification emails.

## Multi-Module Orchestrator Architecture

For complex automation systems (account creation → community setup → content seeding → monitoring), use this module structure:

```
skool_automation/
├── config.py            # Pydantic models, YAML persistence, env vars
├── proxy_manager.py     # Pool with sticky assignment, health tracking, provider abstraction
├── browser_manager.py   # Playwright launch, fingerprinting, stealth, human-like I/O
├── [platform]_account.py  # Page-specific flows (signup, login, verification)
├── [platform]_manager.py  # Higher-level operations (create, configure, query)
├── content_seeder.py    # Posting, replying, liking with calendar + state tracking
├── profile_store.py     # SQLite CRUD for profiles, seeding state, stats
└── orchestrator.py      # Single entry point tying all modules together
```

Each module is a class instantiated once. The orchestrator composes them. Profiles flow through the system: proxy assigned → browser launched with fingerprint → account created → community built → content seeded. SQLite tracks progress so interrupted runs can resume.
```python
context.storage_state(path=f"profiles/{profile_id}/cookies.json")

# Restore on next launch:
context_options["storage_state"] = f"profiles/{profile_id}/cookies.json"
```

Use SQLite for structured profile data (email, proxy assignment, community state, seeding progress). JSON/YAML for configuration.

## React Form Submission (CRITICAL PITFALL)

React-based forms often silently ignore mouse clicks on submit buttons because React uses event delegation with synthetic events. `<span>` overlays inside `<button>` elements intercept clicks, and `element.click()` / `page.mouse.click()` produce events lacking `isTrusted: true`.

**The fix: Keyboard Tab + Enter**

```python
# Fill fields using keyboard navigation, not click+fill
page.locator("#first_name").first.click()
page.keyboard.type("James", delay=50)     # delay adds human-like typing
page.keyboard.press("Tab")
page.keyboard.type("Anderson", delay=50)
page.keyboard.press("Tab")
page.keyboard.type("email@domain.com", delay=50)
page.keyboard.press("Tab")
page.keyboard.type("password", delay=50)

# Tab to the submit button and press Enter
page.keyboard.press("Tab")
time.sleep(0.5)
page.keyboard.press("Enter")  # This triggers React's synthetic onSubmit
```

**DO NOT use:** `page.locator("button[type='submit']").click()`, `click(force=True)`, `page.evaluate("...click()")`, `page.mouse.click(x, y)`, or `form.submit()` — all fail silently on React.

## Email Verification Code Polling

**EXTRACTED to `email-verification` skill.**

## Facebook Automation

**EXTRACTED to `facebook-automation` skill.** Load `skill_view(name='facebook-automation')` for Marketplace GraphQL scraping (no browser), account creation with SMS verification, signup form selectors, and doc ID rotation.

## Platform-Specific Patterns

**EXTRACTED to `platform-automation` skill.** Load `skill_view(name='platform-automation')` for Skool.com signup flow, Reddit anti-bot strategy (CDP piercing, headless detection), curl_cffi TLS impersonation, and live selector discovery techniques.

## SMS Provider Evaluation & Selection

**EXTRACTED to `sms-verification` skill.** When choosing an SMS provider for automated account creation, load `skill_view(name='sms-verification')` for the full evaluation framework, 4-layer root cause analysis, provider catalog, and integration patterns.

## ProxyEmpire SID Rotation

**EXTRACTED to `proxy-management` skill.**

## Pitfalls

- **Try HTTP before browser**: Before reaching for Playwright or browser-use, test if the target site's login is a simple form POST. Use `curl_cffi` with TLS impersonation for sites that don't require JavaScript for their login flow. It's 100x lighter and avoids headless-detection entirely. Check with `re.findall(r'<form', html)` — if no forms exist in raw HTML, the login is JS-rendered and you need a real browser.
- **Headless detection by target sites**: Some sites (Reddit, Cloudflare-protected) detect and block headless Chromium even with stealth flags. Always confirm it's not an IP block first: `curl -sI https://target.com`. If curl gets 200/301 but headless Chromium gets blocked, the block is fingerprint-level. Try `--headless=new` flag, a residential proxy, or fall back to `curl_cffi` if no JS needed.
- **`networkidle` hangs on certain sites**: Reddit and other SPA-heavy sites maintain constant websocket/polling connections, so `wait_until="networkidle"` never resolves. Use `wait_until="domcontentloaded"` or `wait_until="commit"` with a fixed wait instead.
- **Consistent fingerprints matter more than perfect ones**: A profile that changes user agents or screen sizes between sessions triggers more flags than a mildly suspicious but consistent fingerprint.
- **Rate limiting**: Don't create accounts or communities back-to-back. Add 10-30 second cooldowns with jitter between operations.
- **Don't over-automate the platform itself**: Bot the funnel (YouTube, X) to drive real users. Inside the community, real members should drive engagement. Amplifier profiles are for launch seeding only — taper them off as real members join.