Pmx Canvas Testing

PMX Canvas Testing

Use this skill when changing the PMX Canvas product checkout and you need a consistent verification path. The commands and paths below refer to that source checkout, not to a workspace that merely has the published package and skills installed.

For an isolated consumer workspace, do not expect the package to contain this checkout's tests/, source-only scripts, or Playwright configuration. Validate package transports and host integration through their public CLI, MCP, HTTP, and SDK surfaces; use the published-consumer-e2e skill when a packed-install outside-in check is required.

When To Use

• Any code change that should be validated before handoff
• Adding or updating tests
• Debugging a regression or flaky behavior
• Updating CI or coverage commands
• Deciding the minimum acceptable verification for a task

Default Verification Ladder

Pick the narrowest command that proves the change, then escalate if the change crosses layers.

bun run test                # Fast Bun suite for server/state/API coverage
bun run test:coverage       # Same Bun suite with coverage output
bun run test:web-canvas     # Browser smoke against a real running app
bun run test:all            # Bun suite + browser smoke

Direct bun test is side-effect-guarded by tests/preload.ts (wired via bunfig.toml [test].preload): it defaults PMX_CANVAS_DISABLE_BROWSER_OPEN=1 so the suite's open-as-site tests cannot launch the developer's real browser, whichever way the tests are invoked. The preload covers bun test only — when you boot a server yourself (bun run src/cli/index.ts), pass --no-open and set PMX_CANVAS_DISABLE_BROWSER_OPEN=1 explicitly.

Never pipe a gating command through tail, head, or grep. The pipeline reports the exit code of the LAST command, so bun run test | tail -5 exits 0 on a red suite. Redirect to a file and check $?, then read the file.

Which Command To Run

• Server/state/API-only changes: run bun run test
• Test-only changes: run bun run test and bun run test:coverage if coverage matters
• Client/UI/browser interaction changes: run bun run test:web-canvas
• Cross-stack or non-trivial changes: run bun run test:all
• Before changing browser-visible behavior under src/client/: rebuild with bun run build Manual browser validation also requires a fresh client bundle. bun run test:web-canvas already does this for you.

Coverage Notes

• bun run test:coverage covers the Bun unit suite under tests/unit/
• Coverage output is written to coverage/lcov.info and also printed as a text summary
• CI currently uses that same unit-test coverage command, then runs browser smoke separately
• Do not describe test:coverage as full-stack coverage; Playwright coverage is not wired in here

Current Project Test Surface

• In the product source checkout, Bun tests live under tests/unit/.
• In the product source checkout, Playwright browser smoke lives under tests/e2e/.
• Product CI runs those checkout suites: Bun coverage plus the browser smoke flow.
• An isolated installed consumer instead tests the package's public transports and host behavior; it does not run or depend on the product checkout's source suites.

WebView Automation Caveat

• Some Linux/CI environments expose Bun.WebView but still cannot start a usable automation session within the timeout window
• When testing WebView automation, treat a cleanly reported unsupported/timeout runtime boundary as distinct from a product regression

Prefer extending the existing suites before inventing a one-off script.

Test Authoring Rules

• Keep unit tests isolated. Reset singleton server state between tests.
• Test public behavior first: HTTP endpoints, persisted state, visible UI outcomes
• Use browser tests for interactions the user actually performs: node creation, pins, snapshots, loading the workbench, and other sync-sensitive flows
• Avoid brittle selectors. Prefer stable text, roles, titles, or deliberate component hooks
• If a change spans server and client, add at least one server-side assertion and one browser or API-level proof
• MCP harnesses using the official SDK must wind down BOTH halves: call client.close() AND transport.close() on the StdioClientTransport. Closing only the client leaves the spawned stdio server process alive after the test exits (0.4.0 cycle finding)
• Before a version-test cycle, run pmx-canvas skills sync --check in the consumer workspace — exit 1 means the installed skill copies are stale against the package; run pmx-canvas skills sync --yes to refresh them (whole trees, whatever agent layout owns them) before trusting skill-guided results

Tests That Cannot Fail

A test that passes against the broken code is worse than no test — it certifies the bug. Every regression test must be shown to discriminate: run it against the unfixed code (or hard-code the broken value) and watch it go red BEFORE you trust the green.

Four ways a test silently stops discriminating in this repo:

• Polled assertions pass on the first sample. expect.poll(fn).toBeGreaterThanOrEqual(180) succeeds the instant one sample qualifies, so it never observes the failure window. Wait for the settle condition, then assert once.
• Committed fixtures embed generated output. src/server/demo-state.json carries each primitive's generated markup; count and coverage assertions are satisfied by stale markup just as well as fresh. Guard generated fixtures by rebuilding from the generator's INPUT and byte-comparing against the current renderer.
• Absolute assertions about globally-wired side effects are order-dependent. Anything flowing through a single-slot listener (architecture rule 8 in CLAUDE.md) is wired or not depending on whether an earlier test file booted a server — so the test passes alone and fails in the suite, or vice versa. Assert DIFFERENTIALLY: run the action disarmed and armed, require the deltas to match. Always run the full suite before trusting a new test; a single-file run hides this.
• A new node type passes every test and renders nothing. Server, API, and client-unit assertions all pass while isCanvasNodeType drops the type during layout apply. Only a DOM census on a live board catches it.

Layout And Embedded Content Checks

• For seeded or generated boards, add API-level geometry assertions: expected node/edge counts, group counts, valid edge endpoints, no visible node overlaps, and group children contained with header/padding space.
• For grouped layouts, test non-group node overlap separately from group containment. Group frames are allowed to contain children; children should not overlap each other or collide with headers.
• For edge-heavy layouts, assert endpoints exist and long cross-board edges are intentional. If a user says an edge “comes from nowhere,” add a regression check for missing endpoints or excessive edge distance in that board.
• For graph, json-render, mcp-app, webpage, and image nodes, API geometry is not enough. Verify the rendered browser frame when changing sizing: iframe/body scrollHeight and scrollWidth should fit the available frame unless scrolling is the intended behavior.
• When checking embedded frame fit manually, start from a clean seeded state, rebuild stale bundles, and inspect the actual iframe document in a browser. Server dimensions can look correct while the embedded content is still clipped.
• A node's stored content is not render evidence. An html/primitive node's stored HTML holds every conditional branch — hidden banners, error states, empty states — so a content-level read (search hit, pinned context, node get, any text summary) quotes strings the user never saw. The 0.4.7 report filed an "AX bridge unavailable" banner as a live product gap on exactly this basis; the served surface had the bridge injected ahead of the check and the banner was hidden in the document. Before filing a rendering bug, fetch the served surface or inspect the DOM.
• User-facing creation flows must end with the new nodes visible: after the create (plus the skill-mandated focus/fit), a screenshot must show them without any manual pan. A panned: true API result alone is not proof — verify the frame is on-screen and unobscured.

Failure Handling

• Never wave away a failure without checking whether your change caused it
• If the failure is truly pre-existing, say that explicitly and include the failing command
• If a command cannot run in the environment, say what blocked it
• If browser tests fail after a client change, confirm the bundle was rebuilt and the server started from the updated code

Handoff Standard

Before marking work done, report:

• Which verification command(s) you ran
• Whether they passed
• Any meaningful gaps, skipped checks, or known pre-existing failures

For non-trivial changes, the default expectation is bun run test:all unless there is a clear reason to scope verification more narrowly.

Presence, sessions, and the redesigned chrome (rail-chrome-v2)

• Agent/human presence is in-memory and TTL-swept. An e2e reset must detach any attached session (POST /api/canvas/ax/presence { attached: false } per attached presence), clear the scope fence, and mark its own writes with x-pmx-workbench: 1 — otherwise the harness reads as an external writer and a leftover fence refuses the next test's clear. Unattached writers from earlier tests can still be live (90 s): assert about YOUR writers, not exact totals.
• Assert SSE frames as received (readSseEvent in tests/unit/agent-presence-api.test.ts), not the object handed to the emitter — the envelope overwrites sessionId and timestamp.
• Two-tab behaviour (human cursors, the edit lock) needs two browser contexts; name them with /workbench?name=… so assertions can target a cursor by its tag.
• The Browser pane reports visibility: hidden and never fires rAF: drags, drop pills, edge previews and presence animations only work under Playwright. Use the pane for static checks.
• Selection is shift-click on the node BODY (the titlebar drags); groups have no ports and their drag handle is the edge row; the selection bar and command bar both float bottom-center (the selection bar lifts during a session).