Zupulse Desktop Debug
All paths below are relative to the repository root unless absolute.
Host facts (launch flags, userdata, native dialogs, engines, OMR fixtures): apps/desktop-shell/AGENTS.md.
Locators and deterministic journeys: apps/desktop-shell/e2e/desktop.spec.ts.
Product behavior: docs/features/contracts/ (not this skill).
When not to use
- Browser or iPad. This skill launches Electron only. For the Browser demo, use
zupulse-web-debug.
- Unit tests, schema tests, or
pnpm desktop:test:e2e journeys that already assert the same outcome.
- Reading code or Feature Contracts without driving the running app.
If a playbook's steps become stable assertions, add them to e2e/desktop.spec.ts and leave a one-line pointer here.
Launch
pnpm desktop:build- Choose a workdir under
tmp-run/<slug>/ (e.g. tmp-run/pdf-omr-happy-path/). Put driver logs, userdata/, shots/, exports, and cmd/res files there. Do not use tasks/ — that directory is for initiative execution state, not debug runs. tmp-run/ is gitignored.
- Seed
userdata/ before first launch (apps/desktop-shell/AGENTS.md). Default locale is en-US so e2e role names match.
- Start the driver (OMR jobs run longer than a single shell call):
W="$(pwd)/tmp-run/<slug>"
mkdir -p "$W"
cd apps/desktop-shell
nohup node ../../.agents/skills/zupulse-desktop-debug/scripts/driver.mjs "$W" > "$W/driver.log" 2>&1 &
for i in $(seq 1 40); do [ -f "$W/driver.ready" ] && break; sleep 1; done
[ -f "$W/driver.ready" ] || { echo "driver failed to start"; cat "$W/driver.log"; exit 1; }
- Send commands from
apps/desktop-shell:
node ../../.agents/skills/zupulse-desktop-debug/scripts/cmd.mjs "$W" '{"action":"shot","name":"01-home"}'
- Cleanup (always):
kill "$(cat "$W/driver.pid")" then pkill -f "user-data-dir=$W/userdata". Never leave Electron running.
driver.mjs <workdir> [repoRoot] — repoRoot defaults to the repository containing the skill. Ready signal is $W/driver.ready (not driver.log; file-redirected stdout may buffer). If $W/driver.window-closed appears, restart the driver; userdata persists.
Commands
JSON actions via cmd.mjs. Optional __timeout ms on any command (default 240s wait for the result file). Click/fill/select fail when the locator matches 0 or 2+ nodes. Failed commands write shots/fail-<id>.png when the window is still open.
| Action |
Fields |
Notes |
shot / shotFull |
name |
PNG under $W/shots/ |
nav |
hash (/pdf-omr or #/pdf-omr) |
Waits until location.hash matches |
clickRole |
role, name, exact? |
Unique match required |
clickTestId |
testId |
Unique match required |
clickText |
text, exact? |
Unique match required |
selectOption |
combobox, value |
Unique combobox |
fill |
value, plus testId or role+name, exact? |
Unique match required |
press |
key, optional role+name, exact? |
Page-level if no role |
mockOpen |
path or paths, or canceled: true |
One-shot; next dialog restores the original |
mockSave |
path, or canceled: true |
One-shot |
unmock |
|
Restore both dialogs now |
menuClick |
id |
Native app menu (export-diagnostics) |
bodyText |
|
First 6000 chars; fastest UI assert |
waitText / waitGone |
text, exact?, timeoutMs? |
At least one match |
wait |
ms |
Last resort |
eval |
js |
Renderer only. window.confirm is auto-dismissed — override first: "window.confirm = () => true" |
hash |
|
Full page.url() |
mockOpen before the select click; mockSave before export. Re-mock before every later dialog.
Playbook JSON is literal except <repo> (repository root) and <workdir> (the driver workdir).
Playbooks
| When |
File |
| Real-engine PDF OMR happy path (stages → preview → export MXL) |
references/playbooks/pdf-omr-happy-path.md |
| Known-bad PDF / artifact diagnosis |
references/playbooks/pdf-omr-engine-failure.md |
| Library import, export, delete |
references/playbooks/library-import-export.md |
| Viewer playback, tempo, loop |
references/playbooks/viewer-playback.md |
| Native Help diagnostic export |
references/playbooks/desktop-diagnostics.md |
1---2name: zupulse-desktop-debug3description: Build, launch, and drive the Zupulse desktop-shell Electron app (apps/desktop-shell) for manual UI verification, bug reproduction, and UX audits — including PDF OMR (PDF 识谱) flows with real local engines. Use when a task requires opening the desktop app, selecting files through native dialogs, running PDF recognition, taking UI screenshots, or reproducing Desktop-only behavior. Do not use for Browser/iPad, unit tests, or journeys already covered by pnpm desktop:test:e2e unless investigating a failure those tests do not cover.4---56# Zupulse Desktop Debug78All paths below are relative to the repository root unless absolute.910Host facts (launch flags, userdata, native dialogs, engines, OMR fixtures): `apps/desktop-shell/AGENTS.md`.11Locators and deterministic journeys: `apps/desktop-shell/e2e/desktop.spec.ts`.12Product behavior: `docs/features/contracts/` (not this skill).1314## When not to use1516- Browser or iPad. This skill launches Electron only. For the Browser demo, use `zupulse-web-debug`.17- Unit tests, schema tests, or `pnpm desktop:test:e2e` journeys that already assert the same outcome.18- Reading code or Feature Contracts without driving the running app.1920If a playbook's steps become stable assertions, add them to `e2e/desktop.spec.ts` and leave a one-line pointer here.2122## Launch23241. `pnpm desktop:build`252. Choose a workdir under `tmp-run/<slug>/` (e.g. `tmp-run/pdf-omr-happy-path/`). Put driver logs, `userdata/`, `shots/`, exports, and cmd/res files there. Do **not** use `tasks/` — that directory is for initiative execution state, not debug runs. `tmp-run/` is gitignored.263. Seed `userdata/` **before** first launch (`apps/desktop-shell/AGENTS.md`). Default locale is `en-US` so e2e role names match.274. Start the driver (OMR jobs run longer than a single shell call):2829```bash30W="$(pwd)/tmp-run/<slug>"31mkdir -p "$W"32cd apps/desktop-shell33nohup node ../../.agents/skills/zupulse-desktop-debug/scripts/driver.mjs "$W" > "$W/driver.log" 2>&1 &34for i in $(seq 1 40); do [ -f "$W/driver.ready" ] && break; sleep 1; done35[ -f "$W/driver.ready" ] || { echo "driver failed to start"; cat "$W/driver.log"; exit 1; }36```37385. Send commands from `apps/desktop-shell`:3940```bash41node ../../.agents/skills/zupulse-desktop-debug/scripts/cmd.mjs "$W" '{"action":"shot","name":"01-home"}'42```43446. Cleanup (always): `kill "$(cat "$W/driver.pid")"` then `pkill -f "user-data-dir=$W/userdata"`. Never leave Electron running.4546`driver.mjs <workdir> [repoRoot]` — repoRoot defaults to the repository containing the skill. Ready signal is `$W/driver.ready` (not `driver.log`; file-redirected stdout may buffer). If `$W/driver.window-closed` appears, restart the driver; userdata persists.4748## Commands4950JSON actions via `cmd.mjs`. Optional `__timeout` ms on any command (default 240s wait for the result file). Click/fill/select fail when the locator matches 0 or 2+ nodes. Failed commands write `shots/fail-<id>.png` when the window is still open.5152| Action | Fields | Notes |53| ----------------------- | ----------------------------------------------------- | --------------------------------------------------------------------------------------------------- |54| `shot` / `shotFull` | `name` | PNG under `$W/shots/` |55| `nav` | `hash` (`/pdf-omr` or `#/pdf-omr`) | Waits until `location.hash` matches |56| `clickRole` | `role`, `name`, `exact?` | Unique match required |57| `clickTestId` | `testId` | Unique match required |58| `clickText` | `text`, `exact?` | Unique match required |59| `selectOption` | `combobox`, `value` | Unique combobox |60| `fill` | `value`, plus `testId` **or** `role`+`name`, `exact?` | Unique match required |61| `press` | `key`, optional `role`+`name`, `exact?` | Page-level if no role |62| `mockOpen` | `path` or `paths`, or `canceled: true` | **One-shot**; next dialog restores the original |63| `mockSave` | `path`, or `canceled: true` | One-shot |64| `unmock` | | Restore both dialogs now |65| `menuClick` | `id` | Native app menu (`export-diagnostics`) |66| `bodyText` | | First 6000 chars; fastest UI assert |67| `waitText` / `waitGone` | `text`, `exact?`, `timeoutMs?` | At least one match |68| `wait` | `ms` | Last resort |69| `eval` | `js` | Renderer only. `window.confirm` is auto-dismissed — override first: `"window.confirm = () => true"` |70| `hash` | | Full `page.url()` |7172`mockOpen` **before** the select click; `mockSave` **before** export. Re-mock before every later dialog.7374Playbook JSON is literal except `<repo>` (repository root) and `<workdir>` (the driver workdir).7576## Playbooks7778| When | File |79| -------------------------------------------------------------- | ------------------------------------------------ |80| Real-engine PDF OMR happy path (stages → preview → export MXL) | `references/playbooks/pdf-omr-happy-path.md` |81| Known-bad PDF / artifact diagnosis | `references/playbooks/pdf-omr-engine-failure.md` |82| Library import, export, delete | `references/playbooks/library-import-export.md` |83| Viewer playback, tempo, loop | `references/playbooks/viewer-playback.md` |84| Native Help diagnostic export | `references/playbooks/desktop-diagnostics.md` |
hikerpig