Tracy Gui Verify

Test the Tracy GUI headlessly

Two routes run the profiler GUI on a display-less machine. Pick one per test goal:

Route A — Web GUI (emscripten, headless browser)

Build the emscripten profiler GUI into an instrumented temporary workspace, serve it, and drive it with the browser tool. The harness exports debug_snapshot/debug_clear, exposing the app-side mouse/ID state — the ground truth for targeting and verification, not screenshots.

Constants marked M are workstation-dependent — re-verify them. Everything else is stable; do not re-derive it.

Setup

sh <this-skill-dir>/setup.sh            # workspace=/tmp/tracy-protocol
# or: TRACY_WEB_WS=/elsewhere TRACY_WEB_REPO=<repo> sh setup.sh

<this-skill-dir> is the project-scoped .omp/skills/tracy-gui-verify/ under the active project root — not a fixed home directory. If multiple copies exist (one per project that has used this skill), use the copy inside the repo being tested; TRACY_WEB_REPO controls which repo is copied into the workspace.

The script does: fresh source copy (excludes .git and build dirs) → applies harness.patch (adds debug_snapshot/debug_clear exports + event/frame recording to the emscripten backend, in the COPY only) → configures → builds Release with the emsdk at $EMSDK_DIR (default ~/emsdk) and the CPM cache at ~/.cache/cpm.

Pitfall: if the workspace is deleted while a tracy-web server is running, stop the hub process first — a stale server keeps the deleted cwd and silently returns empty responses while the port still accepts connections.

Pitfall: a tracy-web that died in an earlier session leaves the daemon in failed state: stop only reports the old failure, and start keeps returning "Daemon tracy-web has unacknowledged completion notifications" while the port stays free. Drain the daemon with hub op=logs name=tracy-web (the log also shows the cause), then relaunch with hub op=restart name=tracy-web (retained spec). If the port is NOT free, a stale process owns it — often one serving an earlier (now-stale) build directory: the new httpd.py dies on the bind and your requests silently hit the old directory. Check ss -tlnp | grep 8000 and kill the owner. After any start/restart, verify the server is actually serving: curl -s -o /dev/null -w "%{http_code}" http://127.0.0.1:8000/index.html must return 200 — the daemon message is not proof of a server.

After editing the GUI in the repo: copy the changed files into $WS/src/ (same relative path), re-run cmake --build $WS/build -j$(nproc), reload the page. If a change touches profiler/src/BackendEmscripten.cpp or profiler/CMakeLists.txt, the harness patch may no longer apply — re-apply it manually (keep the Dbg* harness section, the DbgPushEvent/DbgPushFrame calls in the mouse callbacks + NewFrame, and the two debug_* exports in -sEXPORTED_FUNCTIONS).

Serve + open + ready

Server (long-running → hub):

hub op=start name=tracy-web application=python3 args=["$WS/build/httpd.py"]
    cwd=$WS/build ready={port:8000} persist=true

httpd.py sends the COOP/COEP headers required for pthreads/SharedArrayBuffer — use it, not a bare http.server.

Browser:

browser open url=http://127.0.0.1:8000/index.html viewport={width:1600,height:900}

Ready signal (in a run cell; run code executes in Node scope — page JS goes through tab.evaluate):

let title = null;
for (let i = 0; i < 360; i++) {
  title = await tab.evaluate(() => document.title);
  if (/ - Tracy Profiler/.test(title)) break;
  await new Promise(r => setTimeout(r, 250));
}

Title is Tracy Profiler X.Y.Z until the preloaded embed.tracy (DarkRL capture) finishes loading, then <trace> (embed.tracy) - Tracy Profiler X.Y.Z (profiler/src/main.cpp SetWindowTitleCallback). Wait ~1.5 s after, then confirm the app loop lives: snap().frames.at(-1).tick > 0.

Harness API (ground truth)

const snap  = () => tab.evaluate(() => JSON.parse(Module.ccall('debug_snapshot','string',[],[])));
const clear = () => tab.evaluate(() => Module.ccall('debug_clear','void',[],[]));

snap() → { dpr, innerW, innerH, bufW, bufH, rectL, rectT, rectW, rectH, title, cursor, frames:[…64], events:[…64] } (newest last).

• frames[]: one sample per rAF tick, including idle ticks. tick = main-loop count (always advances), f = ImGui::GetFrameCount() (advances only when a frame actually renders — the idle gate, §Idle), mx/my = app mouse position, md[3] = mouse button states, q = pending input events, wa = tracy::s_wasActive at tick start, hi/ai = ImGui HoveredId/ActiveId.
• events[]: raw callback data as the app received it. type 0=move 1=down 2=up 3=enter 4=leave 5=wheel; tx/ty = Emscripten int targetX/Y; ax/ay = position passed to ImGui (-1 for non-move); t = DOMHighResTimeStamp.

Coordinate mapping

appX = floor( trunc(cssX - rectL) * dpr )     // device px, what ImGui hit-tests
cssX = appX / dpr                              // inverse for targeting
screenshot_px = cssX * dpr

Chain: DOM clientX → Emscripten targetX = int(clientX - (rect.left|0)) (src/lib/libhtml5.js, EmscriptenMouseEvent.targetX is an int) → Tracy AddMousePosEvent(targetX * dpr, …) (BackendEmscripten.cpp) → ImGui ImFloor (imgui.cpp AddMousePosEvent). Canvas fills the viewport, so rectL/rectT are 0 (re-read anyway).

M: dpr on this workstation's headless browser is 1.25 (canvas buffer 2000×1125 for a 1600×900 viewport; screenshots are captured at device px). Never assume 1. App positions come in multiples of dpr; verify targeting within ±1 px via frames[].mx/my.

Click protocol

Click = move, press, refresh through the hold, release, verify — all against rendered frames. The idle gate (§Idle) renders only 3 frames per wake, and a hold with no mouse events at all refreshes nothing, so a fixed-wait down/up from a cold idle can be swallowed (both events received, zero frames rendered) or released after the wake budget expires. Issuing mouse moves while the button is down keeps the budget alive: a same-coord re-move is delivered (verified: it reaches the canvas handler, wakes the app, and a same-coord hold click straddles cleanly) and is the safest refresh — zero hover drift; a ±dx px wiggle works too, but must stay inside the hit area:

async function click(cssX, cssY, holdMs = 400) {
  await page.mouse.move(cssX, cssY);          // 1. ALWAYS move first
  await new Promise(r => setTimeout(r, 150)); // 2. >= 1 rendered frame
  const pre = await snap();
  const pf = pre.frames.at(-1);
  await clear();
  await page.mouse.down();                    // 3. press
  let df = null, hiDuring = null;
  const t0 = Date.now();
  while (Date.now() - t0 < holdMs) {
    await new Promise(r => setTimeout(r, 70));
    await page.mouse.move(cssX, cssY);        // 4. same-coord re-move refreshes the wake budget
    const f = (await snap()).frames.at(-1);
    if (f.md[0] === 1) { df ??= f.f; hiDuring = f.hi; }
  }
  await page.mouse.up();                      // 5. release
  let uf = null;
  for (let i = 0; i < 40; i++) {
    await new Promise(r => setTimeout(r, 40));
    const f = (await snap()).frames.at(-1);
    if (f.md[0] === 0) { uf = f.f; break; }
  }
  await new Promise(r => setTimeout(r, 400));
  const s = await snap();
  const d = s.dpr, ex = Math.floor(Math.trunc(cssX - s.rectL) * d), ey = Math.floor(Math.trunc(cssY - s.rectT) * d);
  return { hoverID: pf.hi, hiDuring, appPos: [pf.mx, pf.my], expected: [ex, ey],
           straddle: { df, uf },
           ok: df !== null && uf !== null && uf > df && hiDuring === pf.hi &&
               Math.abs(pf.mx - ex) <= 1 && Math.abs(pf.my - ey) <= 1 };
}

Rules:

1. Move before every click. The emscripten mousemove handler is the only thing that sets the app mouse position; mousedown/mouseup only queue button state. Fresh page position is invalid (-FLT_MAX) — a click without a prior move activates nothing.
2. Press and release must each render in separate frames with the item hovered (ImGui's PressedOnClickRelease — doc table at imgui_widgets.cpp:489-497; input trickling, ConfigInputTrickleEventQueue, defers a same-frame release). The same-coord refresh is what guarantees rendered frames across the hold from a cold idle.
3. Verify every click: ok: true (position landed, press and release each rendered, hover held — hiDuring === hoverID), plus a screenshot diff as supplementary visual confirmation. Non-ImGui targets (timeline zone bars) have hi = 0 by design — for those verify by screenshot, not hover.
4. Double-click trap: two same-position clicks <300 ms apart = double-click (MouseDoubleClickTime = 0.3 s) → zoom/thread-info actions fire. Wait ≥350 ms between repeated clicks at one spot.
5. Drag: move → down → (move + wait 70 ms)×n → up. Wheel: page.mouse.wheel({ deltaY: ±500 }) after positioning. Keys: page.keyboard.type(..., { delay: 50 }) after focusing the input by click; page.keyboard.press('Enter').
6. Danger: toolbar device x 10–32 (css 8–26) = red power button (leftmost toolbar icon, hover ID 3313137574), closes the trace view (back to "Get started"). Never click it unless intended.

Idle and power saving

main.cpp DrawContents keeps an activeFrames = 3 budget refreshed on any input event (tracy::s_wasActive), a connected client, view animation, or a queued input; exhausted → sleep 16 ms with no ImGui::NewFrame and no GL draw (canvas keeps the last frame). Expect ~97 % of ticks to skip rendering while idle.

• Clicks from a cold idle land only via the click protocol's hold refresh (§Click protocol) — a fixed-wait down/up can be swallowed by the idle gate (both events received, zero frames rendered). Before interacting, assert tick/f is advancing; a stalled main loop (initial trace parse, hidden tab) renders nothing and drops everything. During the initial load the cursor style is wait.
• focusLostLimit (reduce render rate on focus loss) is desktop-only; it does not exist in the emscripten path.
• canvas.style.cursor is not a hover oracle: this ImGui build sets the Hand cursor only in TextLink, not on buttons.

Targeting

Screenshots never resolve targeting. Full-page shots arrive downscaled JPEG (1024 px wide for a 2000-px device buffer; the factor varies), and page.screenshot({ clip }) returns a PNG at device scale — a W×H CSS clip returns W·dpr × H·dpr px (CSS = clip offset + px/dpr). Icon buttons are 15–25 device px wide, so visual estimates carry ±20 px error, enough to hit the neighboring button or a gap. Target from the app's own hit-testing — sweep the row and click the run's center:

async function sweepRow(dpr, cssY, devFrom, devTo, step = 4) {
  const ranges = []; let cur = null;
  for (let dev = devFrom; dev <= devTo; dev += step) {
    await page.mouse.move(dev / dpr, cssY);
    await new Promise(r => setTimeout(r, 50));
    const s = await snap();
    const hi = s.frames[s.frames.length - 1].hi;
    if (cur && hi === cur.id) cur.end = dev;
    else { if (cur) ranges.push(cur); cur = { id: hi, start: dev, end: dev }; }
  }
  ranges.push(cur);
  return ranges.filter(r => r.id !== 0 && r.end - r.start >= 4);
}

• Same non-zero hi across consecutive steps = one widget; hi = 0 = gap. Click the run's center: click((r.start + r.end) / 2 / dpr, cssY).
• Layout is state-dependent: docked panels (Statistics, Flame, …) resize the main window and shift the toolbar — re-sweep after any state change.
• Close popups before sweeping: while a popup is open, hovers outside the popup can read hi = 0 even over live widgets (verified with ZoomPopup over the toolbar). Clicking a % option in ZoomPopup does not close the popup — click an empty area first.
• Hover IDs are stable across sessions (label hashes) — the map below holds for the default state (fresh load, no panels, 1600×900, dpr 1.25).
• Zoomed region shots (for correlating IDs with icons): page.screenshot({clip}) returns a Buffer — write it with Node fs, then read the file:

const fs = require('fs');
const buf = await page.screenshot({ clip: { x: 0, y: 0, width: 720, height: 28 } });
fs.writeFileSync('/tmp/toolbar-zoom.png', buf);

Toolbar map — default state (device px; css = device/1.25; row y css 15)

Button names and functions are documented in the user manual's Control menu section (manual/tracy.md, generated into the profiler build directory by the build, e.g. build/profiler/manual/tracy.md). The web build omits Connection (live capture only) and Tracy Assist (desktop only).

User scale (DPI zoom)

The 🔍+ button (hover ID 2335673581) — Display scale in the user manual — opens a ZoomPopup with 50–300 % steps. In the web build this scales the UI layout only: fonts, Style::ScaleAllSizes, and window sizes (main.cpp SetupDPIScale, scale = devicePixelRatio × userScale). The canvas buffer and the mouse→app mapping stay at devicePixelRatio (BackendEmscripten.cpp) — hit regions become physically larger in the same coordinate space, targeting precision is unchanged, and the standard click protocol works as-is.

Use it when small buttons are hard to hit or details are hard to read in screenshots. At 200 % every toolbar hit region is roughly 2× (verified: power 16→40 px, gear 25→50 px, Messages 105→190 px wide; row height 24→46 device px). The layout reflows completely at any scale — re-sweep after changing it and after setting it back. userScale is session-only unless Options → "Save UI scale" is on; a fresh page load starts at 100 %.

Limitations

• A connected live-streaming client cannot be tested in the web build (the browser GUI cannot open a TCP server). The embedded-trace GUI is the live-app proxy: real trace data, full GUI — for live-client behavior use Route B.
• embed.tracy (10 MB) is downloaded from share.nereid.pl at configure time — network needed once per fresh workspace; the file then lives in $WS/build.

References

Sources for the behavior the rules above encode — consult them when re-verifying a constant or when a rule fails:

• profiler/src/BackendEmscripten.cpp — emscripten input handlers, canvas/dpr sizing, cursor mapping.
• profiler/src/main.cpp:540-574 — idle gate (activeFrames budget).
• profiler/src/profiler/TracyMouse.cpp — per-frame click state cache.
• profiler/CMakeLists.txt:334-353 — emscripten link options.
• ImGui 1.92.9b (vendored copy in the CPM cache, ~/.cache/cpm/imgui/): imgui_widgets.cpp:483-545 (click semantics table), imgui.cpp:11278-11403 (event trickling), imgui.cpp:1736 (trickle default on), imgui.cpp:1992-2016 (AddMousePosEvent floor/dedup).
• Emscripten 5.0.7: src/lib/libhtml5.js (fillMouseEventData, targetX = e.clientX - (rect.left|0) into HEAP32), system/include/emscripten/html5.h (EmscriptenMouseEvent.targetX is int; EmscriptenWheelEvent embeds a mouse sub-struct).
• Upstream: imgui issue #4921 (new input event API — events spread over multiple frames), PR #2525 (same-frame click/release), issue #1992 (rapid down/up "soft clicks" dropped).

Route B — Native GUI (X11, headless)

Run the native (GLFW) GUI on a virtual X server for what Route A cannot test: a live TCP connection to a client, the native file dialogs (save/open flows), and X11 input semantics. Stack: Xvfb + openbox + xdotool + ImageMagick import. Constants marked M were verified on this workstation (1280x800 screen, openbox, profiler window client area at (1,20)); re-verify after any screen/WM change, and always confirm against a screenshot before clicking.

Build

cmake -S profiler -B /tmp/prof-x11 -DCMAKE_BUILD_TYPE=Release -DLEGACY=ON -DGTK_FILESELECTOR=ON
cmake --build /tmp/prof-x11 --target tracy-profiler -j$(nproc)

• LEGACY=ON selects the X11 (GLFW) backend; the Linux default is Wayland.
• GTK_FILESELECTOR=ON routes NFD through GTK. Without it the default Linux NFD uses the XDG desktop portal and fails headless with the notification "File selector is not available".
• Verify after building: ldd /tmp/prof-x11/tracy-profiler | grep -c gtk ≥ 1.

Services

hub op=start name=xvfb application=Xvfb args=[":99","-screen","0","1280x800x24"]

Ready when /tmp/.X11-unix/X99 exists — check the socket. (If you add -nolisten tcp to the Xvfb args, a TCP port readiness check cannot work; the UNIX socket is then the only listener.) Then:

DISPLAY=:99 nohup openbox >/dev/null 2>&1 &

openbox is required. Without a window manager the window lands at an arbitrary position, xdotool windowactivate fails ("WM claims not to support _NET_ACTIVE_WINDOW"), and keyboard focus has no owner. Under openbox the profiler window is placed deterministically (M: (1,20), 1278x775 on a 1280x800 screen).

Run

DISPLAY=:99 LIBGL_ALWAYS_SOFTWARE=1 nohup /tmp/prof-x11/tracy-profiler >gui.log 2>&1 &

• LIBGL_ALWAYS_SOFTWARE=1 — no GPU; mesa swrast renders.
• Window: DISPLAY=:99 xdotool search --name "Tracy Profiler" → getwindowgeometry for the origin; windowactivate --sync to focus.
• Live test client: tracy-monitor -p $(pgrep -n sleep) style — attach to a long-lived process; the in-process client listens on 8086 and advertises via UDP (discovery works headless; the client line appears under "Discovered clients").

Interaction rules

1. Verify every click with a screenshot diff — import -window root before and after, then PIL ImageChops.difference(a, b).getbbox(): no (or irrelevant) bbox = the click missed. Expect at least one swallowed click; re-issue it and re-verify.
2. Move before click; sleep ≥0.3 s between two clicks at the same spot (ImGui double-click threshold is 0.3 s).
3. Keyboard (xdotool key) is reliable once openbox owns focus.
4. First launch only: an achievements prompt appears bottom-right — click "No" (M: 725,585). The choice persists in the Tracy config; later launches skip it.
5. A stale "Client not ready / another server" notification means the client was replaced underneath the GUI — click Reconnect (M: 738,574); there is no automatic retry.

Coordinates (M: screen px, 1280x800, window at (1,20))

Directly clicked and verified: Connect, Save trace…, GTK Save, Replace, and the save-confirmation button. Derived as the verified window-relative offset translated to the openbox window origin (confirm on first use): wrench, Reconnect, achievements "No".

The Get-started panel, connection popup, and the two confirmation panels keep fixed offsets inside the profiler window; the GTK dialogs are separate top-level windows sized to the screen. Derive any missing coordinate from a screenshot rather than guessing.

Save flow (end-to-end)

1. Start the client, click Connect (the GUI never auto-connects).
2. The connection popup opens automatically on connect; click Save trace….
3. GTK dialog: the Name entry is focused and the last-used directory is preselected; xdotool type --delay 30 <name>; click Save (1195,773).
4. Existing target file → a separate overwrite-confirmation window appears; click Replace (790,482).
5. Tracy's own confirmation panel appears (shows the final Path:); click its Save trace (757,540).
6. Verify the file (below).

File and connection checks

• Live connection: ss -tnp | grep 8086 shows ESTAB between tracy-profiler and the client process; the window title becomes <program> @ <date> - Tracy Profiler X.Y.Z.
• Saved .tracy: first 4 bytes = 74 72 FD 50; tail -c +11 file | zstd -t → premature end means the frame decodes fully (expected — the cut falls past the final frame into trailing metadata); unsupported format means a bad offset or a non-trace file.
• No file.tmp may remain after a completed save; a save killed mid-write leaves the previous file intact (the GUI writes through SafeFileWrite, tmp + rename-on-commit).
• Video evidence while interacting: ffmpeg -f x11grab -video_size 1280x800 -i :99 -t N -c:v libx264 out.mp4 (background it; it records the whole interaction).

Limitations

• No debug harness — the screenshot diff is the only oracle; targeting is coordinate-from-screenshot, slower and coarser than Route A's hit-test sweeps.
• The verified coordinates are tied to the M constants (1280x800 screen, openbox default theme, 100 % UI scale); re-derive from a screenshot if any of them changes.
• The GTK dialogs are separate top-level windows whose layout follows the system GTK theme.

Teardown

pkill -f prof-x11; pkill -f tracy-monitor
pkill -f "sleep 600"          # the test target from §Run — adjust to what you started
pkill openbox
# hub op=stop name=xvfb

References

• profiler/src/BackendGlfw.cpp — X11/GLFW input handlers (mouse/keyboard → ImGui).
• profiler/CMakeLists.txt:5-7 — NO_FILESELECTOR, GTK_FILESELECTOR, LEGACY (X11 vs default Wayland) options.
• cmake/vendor.cmake:222-242 — NFD packaging: NFD_PORTAL on by default on Linux, the GTK_FILESELECTOR switch, the nfd-xdg-foreign-v2.patch.
• server/TracySafeFileWrite.hpp — the GUI save's tmp + rename-on-commit (the invariants in §File and connection checks).
• NFD (btzy/nativefiledialog-extended, pinned in cmake/vendor.cmake) — the GTK dialog behavior (Name entry focus, last-directory memory, overwrite confirmation).