# Bx Playwright AI

> Use this skill when connecting AI agents to a browser with bx-playwright: accessibility snapshots in ai mode with element refs (page.snapshot, ref=e12 selectors), playwright().aiTools() for bx-ai agents, aiToolDefinitions() and models.AiBrowser for other frameworks, the Playwright MCP server (bxplaywright mcp), recording BoxLang code with bxplaywright codegen, and API self-description with help().

- Skill: `ortus-boxlang/bx-playwright-ai` (Agent Skill)
- Install (CLI): `npx get-skill add ortus-boxlang/bx-playwright-ai`
- Raw SKILL.md: https://api.skillmd.com/api/skills/ortus-boxlang/bx-playwright-ai/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Web & Frontend
- Author: ortus-boxlang (https://skillmd.com/u/ortus-boxlang)
- Updated: 2026-11-15
- Page: https://skillmd.com/skills/ortus-boxlang/bx-playwright-ai

---


# bx-playwright: AI Agents

> BoxLang is the AI-native software productivity platform for building, modernizing and running applications, with developers and AI agents working together.

## Snapshots With Element Refs

```js
println( page.snapshot( options = { mode : "ai" } ) )
// - heading "Sign in" [level=1] [ref=e2]
// - textbox "Email" [ref=e5]
// - button "Sign in" [ref=e7]
page.fill( "ref=e5", "luis@ortus.com" ).click( "ref=e7" )
```

`page.snapshot()` without options gives the plain accessibility YAML (also used by `toMatchAriaSnapshot()`). `page.snapshot( "@form" )` limits it to an element.

## Tools for bx-ai Agents

```js
agent = aiAgent(
	name         : "Browser",
	instructions : "Use the browser tools to complete the task. Act on element refs from the snapshots.",
	tools        : playwright( { baseURL : "http://localhost:8080" } ).aiTools()
)
agent.run( "Log in as luis@ortus.com with password secret and tell me how many todos I have" )
```

| Tool | Arguments |
|---|---|
| `browser_visit` | `url` |
| `browser_snapshot` | |
| `browser_click` | `target` (ref like `e12` or button/link text) |
| `browser_fill` | `target`, `value` |
| `browser_select` | `target`, `value` |
| `browser_press` | `key` |
| `browser_back` | |
| `browser_text` | `target` (optional) |
| `browser_screenshot` | `path` |
| `browser_close` | |

Each action returns `URL`, `Title` and the ai snapshot. Errors come back as text (`Error [Playwright.Timeout]: ... Fix: ...`) so the agent can recover. The tools share one page. `aiTools()` throws `Playwright.NotInstalled` without bx-ai.

Other frameworks: `playwright().aiToolDefinitions()` returns `[ { name, description, arguments : { name : description }, handler } ]`. Direct use:

```js
browser = new models.AiBrowser@bxplaywright( playwright() )
state   = browser.visit( "https://boxlang.io" )
state   = browser.click( "Docs" )
browser.close()
```

## MCP and Codegen

- `bxplaywright mcp` starts Playwright's MCP server for MCP clients (Claude, Cursor, ...).
- A bx-ai agent can launch it as a child process over stdio (needs the `bx-ai` module, see its `bx-ai-mcp` skill). The agent gets the tools the server lists, and `closeMCPServers()` stops the process:

```js
agent = aiAgent(
    name      : "BrowserAgent",
    model     : aiModel( provider: "openai" ),
    mcpServers: [ { command : [ "bxplaywright", "mcp" ], prefix : "browser" } ]
)
agent.run( "Open https://boxlang.io and tell me the page title" )
agent.closeMCPServers()
```

  Use `aiTools()` (above) when the agent should call bx-playwright directly in the same JVM, and the MCP route when you want Playwright's own MCP tool set or a separate process.
- `bxplaywright codegen http://localhost:8080 --output=flow.bxs` records a session and writes bx-playwright code (`page.byRole( "button", { name : "Save" } ).click()` style). Untranslatable lines are kept as `// TODO translate:` comments.

## Self Description

- `page.help()`, `page.help( "fill" )`, `playwright().help()`: methods, arguments and descriptions from the docblocks.
- `bxplaywright help --json`, `bxplaywright doctor --json`, `bxplaywright devices --json`.
- Every error has a `type` and a `detail` with the fix.

