# Browser Automation

> Browser automation and web scraping using Playwright. Supports page screenshots, form filling, data scraping, and login flows. Use when the user needs to scrape web data, screenshot pages, or automate web interactions — especially when standard fetch tools cannot access dynamic content.

- Skill: `wuyue12-tech/browser-automation` (Agent Skill)
- Install (CLI): `npx skillmds@latest add wuyue12-tech/browser-automation`
- Raw SKILL.md: https://api.skillmd.com/api/skills/wuyue12-tech/browser-automation/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: wuyue12-tech (https://skillmd.com/u/wuyue12-tech)
- Updated: 2026-09-21
- Page: https://skillmd.com/skills/wuyue12-tech/browser-automation

---


# Browser Automation

Browser automation using Playwright for scraping, screenshots, and form interactions.

## When to Use

- User wants to scrape web data (e.g., Aeon article lists)
- User wants to screenshot a webpage
- User wants to automate form filling
- User wants to log into a website and perform actions
- Standard fetch tools cannot access dynamically loaded content

## Install Dependencies

On first use, install Playwright:

```bash
# Python approach
pip install playwright
playwright install chromium

# Or Node approach
npm install playwright
npx playwright install chromium
```

## Core Capabilities

### 1. Page Screenshot

```python
from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page(viewport={"width": 1280, "height": 800})
    page.goto("https://example.com")
    page.screenshot(path="screenshot.png", full_page=True)
    browser.close()
```

### 2. Data Scraping

```python
from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.goto("https://aeon.co/essays")

    # Wait for content to load
    page.wait_for_selector(".essay-card")

    # Extract data
    essays = page.query_selector_all(".essay-card")
    for essay in essays:
        title = essay.query_selector("h2").inner_text()
        link = essay.query_selector("a").get_attribute("href")
        print(f"{title}: {link}")

    browser.close()
```

### 3. Form Filling & Login

```python
page.goto("https://example.com/login")
page.fill("#username", "user@example.com")
page.fill("#password", "password123")
page.click("button[type=submit]")
page.wait_for_load_state("networkidle")
```

### 4. PDF Generation

```python
page.goto("https://example.com/article")
page.pdf(path="article.pdf", format="A4")
```

## Important Rules

1. **Try fetch tools first** — For static pages, prefer built-in fetch/web tools
2. **Only launch browser when necessary** — Dynamic content, interaction needed, login required
3. **Set reasonable timeouts** — Default 30 seconds, extend to 60 seconds for complex pages
4. **Use headless mode** — Do not display browser window by default
5. **Close immediately after scraping** — Avoid resource leaks

## Common Use Cases

### Scraping Aeon Article List

```python
from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.goto("https://aeon.co/essays")
    page.wait_for_selector("article")

    articles = page.query_selector_all("article")
    results = []
    for article in articles:
        title_el = article.query_selector("h2 a, h3 a")
        if title_el:
            results.append({
                "title": title_el.inner_text(),
                "url": title_el.get_attribute("href")
            })

    browser.close()
```

### Scrolling for Infinite-Load Pages

```python
# Scroll to load more content
for i in range(5):  # Scroll 5 times
    page.evaluate("window.scrollTo(0, document.body.scrollHeight)")
    page.wait_for_timeout(2000)  # Wait 2 seconds
```

## Troubleshooting

| Issue | Solution |
|-------|----------|
| Chromium download failed | Set mirror: `PLAYWRIGHT_DOWNLOAD_HOST=https://npmmirror.com/mirrors/playwright` |
| Timeout | Increase `timeout` parameter or check network |
| Element not found | Use `page.wait_for_selector()` to wait |
| Encoding issues | Specify `encoding="utf-8"` when reading |