bx-playwright: AI Agents
BoxLang is the AI-native software productivity platform for building, modernizing and running applications, with developers and AI agents working together.
Snapshots With Element Refs
println( page.snapshot( options = { mode : "ai" } ) )
// - heading "Sign in" [level=1] [ref=e2]
// - textbox "Email" [ref=e5]
// - button "Sign in" [ref=e7]
page.fill( "ref=e5", "luis@ortus.com" ).click( "ref=e7" )
page.snapshot() without options gives the plain accessibility YAML (also used by toMatchAriaSnapshot()). page.snapshot( "@form" ) limits it to an element.
Tools for bx-ai Agents
agent = aiAgent(
name : "Browser",
instructions : "Use the browser tools to complete the task. Act on element refs from the snapshots.",
tools : playwright( { baseURL : "http://localhost:8080" } ).aiTools()
)
agent.run( "Log in as luis@ortus.com with password secret and tell me how many todos I have" )
| Tool | Arguments |
|---|---|
browser_visit |
url |
browser_snapshot |
|
browser_click |
target (ref like e12 or button/link text) |
browser_fill |
target, value |
browser_select |
target, value |
browser_press |
key |
browser_back |
|
browser_text |
target (optional) |
browser_screenshot |
path |
browser_close |
Each action returns URL, Title and the ai snapshot. Errors come back as text (Error [Playwright.Timeout]: ... Fix: ...) so the agent can recover. The tools share one page. aiTools() throws Playwright.NotInstalled without bx-ai.
Other frameworks: playwright().aiToolDefinitions() returns [ { name, description, arguments : { name : description }, handler } ]. Direct use:
browser = new models.AiBrowser@bxplaywright( playwright() )
state = browser.visit( "https://boxlang.io" )
state = browser.click( "Docs" )
browser.close()
MCP and Codegen
bxplaywright mcpstarts Playwright's MCP server for MCP clients (Claude, Cursor, ...).- A bx-ai agent can launch it as a child process over stdio (needs the
bx-aimodule, see itsbx-ai-mcpskill). The agent gets the tools the server lists, andcloseMCPServers()stops the process:
agent = aiAgent(
name : "BrowserAgent",
model : aiModel( provider: "openai" ),
mcpServers: [ { command : [ "bxplaywright", "mcp" ], prefix : "browser" } ]
)
agent.run( "Open https://boxlang.io and tell me the page title" )
agent.closeMCPServers()
Use aiTools() (above) when the agent should call bx-playwright directly in the same JVM, and the MCP route when you want Playwright's own MCP tool set or a separate process.
bxplaywright codegen http://localhost:8080 --output=flow.bxsrecords a session and writes bx-playwright code (page.byRole( "button", { name : "Save" } ).click()style). Untranslatable lines are kept as// TODO translate:comments.
Self Description
page.help(),page.help( "fill" ),playwright().help(): methods, arguments and descriptions from the docblocks.bxplaywright help --json,bxplaywright doctor --json,bxplaywright devices --json.- Every error has a
typeand adetailwith the fix.