Agent Browser

Browser Automation with agent-browser

Comprehensive browser automation skill using the agent-browser CLI for web testing, form filling, screenshots, data extraction, and interactive web automation.

Quick Start Workflow

agent-browser open <url>        # Navigate to page
agent-browser snapshot -i       # Get interactive elements with refs
agent-browser click @e1         # Click element by ref
agent-browser fill @e2 "text"   # Fill input by ref
agent-browser close             # Close browser

Core Commands

Navigation

agent-browser open <url>      # Navigate to URL (auto-prepends https://)
agent-browser back            # Go back in history
agent-browser forward         # Go forward in history
agent-browser reload          # Reload current page
agent-browser close           # Close browser session

Page Analysis

agent-browser snapshot            # Full accessibility tree
agent-browser snapshot -i         # Interactive elements only (recommended)
agent-browser snapshot -c         # Compact output
agent-browser snapshot -d 3       # Limit depth to 3 levels
agent-browser snapshot -s "#main" # Scope to CSS selector

The snapshot command returns elements with reference IDs like @e1, @e2 that you use for interactions.

Element Interactions

# Using refs from snapshot
agent-browser click @e1           # Click element
agent-browser dblclick @e1        # Double-click element
agent-browser fill @e2 "text"     # Clear and type text
agent-browser type @e2 "text"     # Type without clearing
agent-browser hover @e1           # Hover over element
agent-browser focus @e1           # Focus element

# Form controls
agent-browser check @e1           # Check checkbox
agent-browser uncheck @e1         # Uncheck checkbox
agent-browser select @e1 "value"  # Select dropdown option
agent-browser upload @e1 file.pdf # Upload files

# Scrolling
agent-browser scroll down 500     # Scroll page
agent-browser scrollintoview @e1  # Scroll element into view

Keyboard Input

agent-browser press Enter         # Press single key
agent-browser press Control+a     # Key combination
agent-browser keydown Shift       # Hold key down
agent-browser keyup Shift         # Release key

Data Extraction

agent-browser get text @e1        # Get element text
agent-browser get html @e1        # Get innerHTML
agent-browser get value @e1       # Get input value
agent-browser get attr @e1 href   # Get attribute
agent-browser get title           # Get page title
agent-browser get url             # Get current URL
agent-browser get count ".item"   # Count matching elements

State Checking

agent-browser is visible @e1      # Check if visible
agent-browser is enabled @e1      # Check if enabled
agent-browser is checked @e1      # Check if checked

Screenshots & Documentation

agent-browser screenshot          # Save to temporary directory
agent-browser screenshot path.png # Save to specific path
agent-browser screenshot --full   # Full page screenshot
agent-browser pdf output.pdf      # Save page as PDF

Waiting & Timing

agent-browser wait @e1                     # Wait for element to appear
agent-browser wait 2000                    # Wait milliseconds
agent-browser wait --text "Success"        # Wait for text to appear
agent-browser wait --url "**/dashboard"    # Wait for URL pattern
agent-browser wait --load networkidle      # Wait for network idle
agent-browser wait --fn "window.ready"     # Wait for JS condition

Semantic Finding (Alternative to refs)

agent-browser find role button click --name "Submit"
agent-browser find text "Sign In" click --exact
agent-browser find label "Email" fill "user@example.com"
agent-browser find placeholder "Search" type "query"
agent-browser find testid "submit-btn" click

Common Patterns

Form Automation

# 1. Navigate and analyze
agent-browser open https://example.com/form
agent-browser snapshot -i

# 2. Fill form using refs from snapshot
agent-browser fill @e1 "user@example.com"  # Email field
agent-browser fill @e2 "password123"       # Password field
agent-browser click @e3                    # Submit button

# 3. Wait for response
agent-browser wait --load networkidle
agent-browser snapshot -i  # Check result

Authentication & State Management

# Login once and save state
agent-browser open https://app.example.com/login
agent-browser snapshot -i
agent-browser fill @e1 "username"
agent-browser fill @e2 "password"
agent-browser click @e3
agent-browser wait --url "**/dashboard"
agent-browser state save auth.json

# Later sessions: reuse saved state
agent-browser state load auth.json
agent-browser open https://app.example.com/dashboard

Data Extraction Workflow

# Navigate to data source
agent-browser open https://example.com/data
agent-browser snapshot -i

# Extract information
agent-browser get text @e1 > title.txt
agent-browser get attr @e2 href > link.txt
agent-browser screenshot --full > page.png

# Navigate through pagination
agent-browser find text "Next" click
agent-browser wait --load networkidle
# Repeat extraction...

Testing & Validation

# Navigate to application
agent-browser open http://localhost:3000
agent-browser wait --load networkidle

# Test user flow
agent-browser find role button click --name "Get Started"
agent-browser wait --text "Welcome"
agent-browser screenshot test-step-1.png

# Validate elements exist
agent-browser is visible @e1 && echo "✓ Element visible"
agent-browser get text @e2 | grep -q "Expected" && echo "✓ Text correct"

Advanced Features

Sessions (Parallel Browsers)

agent-browser --session test1 open site-a.com
agent-browser --session test2 open site-b.com
agent-browser session list

Video Recording

agent-browser record start ./demo.webm    # Start recording
agent-browser click @e1                   # Perform actions
agent-browser record stop                 # Stop and save

Network Control

agent-browser network route <url> --abort      # Block requests
agent-browser network route <url> --body '{}'  # Mock responses
agent-browser network requests                 # View requests

Browser Configuration

agent-browser set viewport 1920 1080          # Set viewport size
agent-browser set device "iPhone 14"          # Emulate device
agent-browser set offline on                  # Toggle offline mode
agent-browser set credentials user pass       # HTTP basic auth

Debugging Options

agent-browser --headed open example.com   # Show browser window
agent-browser --json snapshot -i          # JSON output for parsing
agent-browser highlight @e1               # Highlight element
agent-browser console                     # View console messages
agent-browser errors                      # View page errors

Error Handling

# Handle certificate errors
agent-browser open https://localhost:8443 --ignore-https-errors

# Check if elements exist before interacting
if agent-browser is visible @e1; then
    agent-browser click @e1
else
    echo "Element not found, taking screenshot for debugging"
    agent-browser screenshot error-state.png
fi

Environment Configuration

Set these environment variables for persistent configuration:

export AGENT_BROWSER_SESSION="mysession"            # Default session name
export AGENT_BROWSER_EXECUTABLE_PATH="/path/chrome" # Custom browser path
export AGENT_BROWSER_PROVIDER="browserbase"         # Cloud browser provider

Integration with Pi Workflow

With Testing

# In test scripts
agent-browser open http://localhost:3000
agent-browser wait --load networkidle
agent-browser find testid "login-form" fill "user@test.com"
agent-browser screenshot test-login-form.png

With Documentation

# Generate screenshots for docs
agent-browser open https://myapp.com/dashboard
agent-browser screenshot --full docs/images/dashboard.png
agent-browser click @e1  # Navigate to feature
agent-browser screenshot docs/images/feature-view.png

With Data Collection

# Extract data for analysis
agent-browser open https://competitor.com
agent-browser get text ".pricing" > competitor-pricing.txt
agent-browser get count ".feature-list li" > feature-count.txt

Troubleshooting

Common Issues

1. Elements not found: Re-run snapshot -i after page changes
2. Timeouts: Use wait commands before interactions
3. Authentication: Save and reuse session state
4. Dynamic content: Wait for specific conditions before interacting

Debug Commands

agent-browser snapshot -i --json | jq '.elements[] | select(.text | contains("Submit"))'
agent-browser highlight @e1  # Visual confirmation
agent-browser console        # Check for JS errors

Related Skills

• Use with /skill:feature for testing new features
• Use with /skill:plan for documenting user flows and validating requirements

References

For detailed documentation on specific topics:

• Authentication Patterns
• Session Management
• Snapshot References
• Video Recording
• Proxy Support

Templates

Ready-to-use scripts for common workflows:

• Form Automation
• Authenticated Session
• Content Capture

Usage:

.pi/skills/agent-browser/templates/form-automation.sh https://example.com/form
.pi/skills/agent-browser/templates/authenticated-session.sh https://app.com/login
.pi/skills/agent-browser/templates/capture-workflow.sh https://site.com ./output