Cua

CUA — Computer Use Agent

Autonomous browser control powered by three cloud browser providers: Steel, Browser Use, and Browserbase. Each offers CDP-based headful browser sessions for Playwright/Puppeteer automation, LLM-driven visual browsing, and human-in-the-loop workflows.

Quick Start

# Environment setup
export STEEL_API_KEY="ste-..."
export BROWSERUSE_API_KEY="bu_..."
export BROWSERBASE_API_KEY="bb_live_..."
export BROWSERBASE_PROJECT_ID="446f4b4c-..."

# Daemon commands
/cua new                          # Create a Steel session (default provider)
/cua new --provider browserbase   # Create a Browserbase session
/cua new --provider browseruse    # Create a Browser Use session
/cua browse https://example.com   # Navigate in active session
/cua screenshot                   # Capture current page
/cua status                       # Show session details + live view URL
/cua release                      # End session
/cua list                         # List all active sessions

Architecture

┌──────────────────────────────────────────────────────────────────┐
│                      SolanaOS CUA Skill                          │
│                                                                   │
│  ┌──────────────┐  ┌───────────────┐  ┌───────────────────────┐ │
│  │  Steel        │  │  Browser Use   │  │  Browserbase          │ │
│  │  Provider     │  │  Provider      │  │  Provider             │ │
│  │              │  │               │  │                       │ │
│  │  • Sessions   │  │  • CDP URL     │  │  • Sessions           │ │
│  │  • Proxy      │  │  • Cloud       │  │  • Extensions         │ │
│  │  • CAPTCHA    │  │  • Profiles    │  │  • Stealth Mode       │ │
│  │  • Live View  │  │  • Proxy       │  │  • Proxies            │ │
│  │  • Recording  │  │               │  │  • Contexts           │ │
│  │  • Mobile     │  │               │  │  • Functions           │ │
│  │  • Regions    │  │               │  │  • Live View           │ │
│  │  • HLS Replay │  │               │  │  • Session Recording   │ │
│  └──────┬───────┘  └──────┬────────┘  └──────────┬────────────┘ │
│         │                  │                       │              │
│  ┌──────┴──────────────────┴───────────────────────┴───────────┐ │
│  │              CDP (Chrome DevTools Protocol)                  │ │
│  │                    WebSocket Connection                      │ │
│  └──────────────────────────┬──────────────────────────────────┘ │
│                              │                                    │
│  ┌──────────────────────────┴──────────────────────────────────┐ │
│  │           Playwright / Puppeteer / Selenium                  │ │
│  │                                                              │ │
│  │  • Page navigation      • Form automation                   │ │
│  │  • Element interaction   • Screenshot capture                │ │
│  │  • File download/upload  • PDF generation                    │ │
│  │  • Network interception  • Cookie management                 │ │
│  └──────────────────────────────────────────────────────────────┘ │
│                                                                   │
│  ┌──────────────────────────────────────────────────────────────┐ │
│  │              LLM Vision Agent (Optional)                      │ │
│  │                                                              │ │
│  │  Screenshot → Vision LLM → Action Plan → Playwright Exec    │ │
│  │                                                              │ │
│  │  Supported: Claude, GPT-4o, Gemini, Llama 3.2, Qwen 2.5 VL │ │
│  └──────────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────────┘

Provider Comparison

Steel Integration

Creating Sessions

// One-line Playwright connection (simplest)
browser := chromium.ConnectOverCDP("wss://connect.steel.dev?apiKey=YOUR_KEY")

// Or with full options
session := steel.Create(ctx, &steel.CreateOptions{
    UseProxy:     true,
    SolveCaptcha: true,
    Timeout:      600000, // 10 minutes
    Region:       "lax",
    DeviceConfig: &steel.DeviceConfig{Device: "mobile"},
})

Session Lifecycle

CREATE → LIVE → [automation] → RELEASE
                     ↓
              FAILED (crash/timeout)

• Sessions auto-release after timeout (default 5 min)
• CDP inactivity timeout: 10 min without commands
• Send heartbeat every 5 min to keep alive: page.evaluate(() => undefined)
• Always call Release() when done

Live View & Recording

// Embed live view (read-only)
iframe := session.DebugURL + "?interactive=false"

// Embed with control (human-in-the-loop)
iframe := session.DebugURL + "?interactive=true&showControls=true"

// Get HLS recording playlist
playlist := steel.HLSPlaylist(session.ID)
// Use with hls.js or native Safari HLS

Available Regions

Browserbase Integration

Creating Sessions

// Via Browserbase API
session := browserbase.CreateSession(ctx, cfg, "default")
// Returns CDP URL for Playwright connection

browser := chromium.ConnectOverCDP(session.CDPURL)
context := browser.Contexts()[0]
page := context.Pages()[0]

Features

• Stealth Mode: Automatic fingerprinting, advanced stealth (Scale plan)
• Proxies: Built-in residential proxy network
• Contexts: Persist cookies/storage across sessions
• Extensions: Load custom Chrome extensions
• Functions: Deploy automation as serverless API endpoints
• Search: Web search without browser session
• Fetch: HTTP fetch without browser session

Session Inspector

Every Browserbase session is recorded and can be inspected:

• Live browser state and interactions
• Network requests/responses
• Console output and errors
• Performance metrics

Browser Use Integration

Cloud Sessions

// Create cloud browser via API
session := browserUseProvider.CreateSession(ctx, cfg, "default")
// Returns CDP URL

// Or via CLI
browser-use cloud connect

Session Management

// List sessions
browser-use sessions

// Close specific session
browser-use --session mySession close

// Close all sessions
browser-use close --all

Daemon Commands

Autonomous CUA Agent

The CUA agent combines cloud browser sessions with vision LLMs for autonomous web navigation.

Agent Loop

1. CREATE SESSION   → Steel/Browserbase/BrowserUse
2. SCREENSHOT       → Capture page via CDP
3. VISION ANALYSIS  → Send to Claude/GPT-4o/Gemini
4. ACTION PLANNING  → LLM determines next step
5. PLAYWRIGHT EXEC  → Execute action via CDP
6. REPEAT           → Until objective complete or max steps
7. RELEASE          → Clean up session

Vision Prompt Template

You are a Computer Use Agent controlling a cloud browser via Playwright.

OBJECTIVE: {objective}

Current page screenshot is attached. Analyze the page and determine the next action.

Available actions:
- page.goto(url) — Navigate to URL
- page.click(selector) — Click element
- page.fill(selector, text) — Fill input
- page.press(selector, key) — Press key
- page.screenshot() — Capture page
- page.waitForSelector(selector) — Wait for element
- page.evaluate(js) — Run JavaScript
- DONE — Task is complete

Respond with JSON: {"action": "...", "args": {...}, "reasoning": "..."}

Environment Variables

API Reference

Steel API

POST   /v1/sessions                    Create session
GET    /v1/sessions                    List sessions
GET    /v1/sessions/:id                Get session details
POST   /v1/sessions/:id/release        Release session
POST   /v1/sessions/release            Release all sessions
GET    /v1/sessions/:id/screenshot     Capture screenshot
GET    /v1/sessions/:id/events         Get replay events
GET    /v1/sessions/:id/hls            Get HLS playlist
WSS    connect.steel.dev?apiKey=...&sessionId=...  CDP WebSocket

Browserbase API

POST   /v1/sessions                    Create session
GET    /v1/sessions/:id                Get session details
POST   /v1/sessions/:id                Close session (REQUEST_RELEASE)
GET    /v1/sessions/:id/debug          Get debug URLs

Browser Use API

POST   /api/v2/browsers                Create cloud browser
PATCH  /api/v2/browsers/:id            Stop browser (action: "stop")

Playwright Connection Patterns

Steel (one-liner)

import { chromium } from "playwright-core";
const browser = await chromium.connectOverCDP(
  `wss://connect.steel.dev?apiKey=${process.env.STEEL_API_KEY}`
);

Steel (managed session)

import Steel from "steel-sdk";
import { chromium } from "playwright";

const client = new Steel({ steelAPIKey: process.env.STEEL_API_KEY });
const session = await client.sessions.create({
  useProxy: true,
  solveCaptcha: true,
});

const browser = await chromium.connectOverCDP(
  `wss://connect.steel.dev?apiKey=${process.env.STEEL_API_KEY}&sessionId=${session.id}`
);

const page = browser.contexts()[0].pages()[0];
await page.goto("https://example.com");

await browser.close();
await client.sessions.release(session.id);

Browserbase

import { chromium } from "playwright-core";
import { Browserbase } from "@browserbasehq/sdk";

const bb = new Browserbase({ apiKey: process.env.BROWSERBASE_API_KEY });
const session = await bb.sessions.create();

const browser = await chromium.connectOverCDP(session.connectUrl);
const page = browser.contexts()[0].pages()[0];
await page.goto("https://example.com");

Browser Use (CDP)

import { chromium } from "playwright-core";
// After creating a session via API, connect with CDP URL
const browser = await chromium.connectOverCDP(session.cdpUrl);