Shell Desktop MCP

Shell Desktop MCP

Overview

Use this skill to drive the Electron shell via MCP with deterministic, testable workflows. Always do preflight first, then run one focused operation batch per objective.

Path Resolution

Skill-local paths (for example scripts/... and references/...) are relative to this directory: /codex/skills/shell-desktop-mcp.

Standard Workflow

1. Confirm backend availability.
2. Capture baseline shell/sim state.
3. Run the requested simulation, window, input, or asset operations.
4. For renderer/WebGPU investigations, run diagnostics tools in the same batch.
5. Re-check state and summarize deltas.
6. Capture reproducible evidence (status + screenshot/log payload) when relevant.

Preflight (Required)

Run this sequence first:

1. Call mcp__idle-engine-shell-desktop__health.
2. If health is not OK, start with bounded wait:

bash /codex/skills/shell-desktop-mcp/scripts/preflight-and-wait.sh gateway-backend 18 10

• Uses 18 attempts with 10-second interval (about 3 minutes max).
• Starts shell-desktop in detached mode (nohup) and writes PID/log paths.
• Requires stable health before success (two consecutive smoke checks + short guard window).
• Fails fast if the launcher process exits before health becomes OK.

3. Re-run mcp__idle-engine-shell-desktop__health and verify ok: true.
4. Run MCP smoke test:

pnpm shell:desktop:mcp:smoke

5. Capture baseline:

• mcp__idle-engine-shell-desktop__sim_status
• mcp__idle-engine-shell-desktop__window_info
• If diagnosing renderer/WebGPU: renderer.status (wrapper: mcp__idle-engine-shell-desktop__renderer_status) + probe.webgpuHealth (wrapper: mcp__idle-engine-shell-desktop__probe_webgpuHealth)

If startup details are needed, use docs/shell-desktop-mcp.md in the repo.

Operation Patterns

Simulation Control

• Use mcp__idle-engine-shell-desktop__sim_start, mcp__idle-engine-shell-desktop__sim_pause, mcp__idle-engine-shell-desktop__sim_resume, mcp__idle-engine-shell-desktop__sim_stop for lifecycle transitions.
• Use mcp__idle-engine-shell-desktop__sim_step only while paused for deterministic advancement.
• Use mcp__idle-engine-shell-desktop__sim_enqueue to inject runtime commands in deterministic order.
• sim_enqueue requires object commands (not strings). Minimal valid shape per command is { type: string }.
• Example:

{
  "commands": [
    {
      "type": "TEST_COMMAND",
      "payload": { "source": "mcp" },
      "priority": 1
    }
  ]
}

• Capture mcp__idle-engine-shell-desktop__sim_status before and after lifecycle mutations.

Window Diagnostics

• Use mcp__idle-engine-shell-desktop__window_info to inspect bounds/url/devtools state.
• Use mcp__idle-engine-shell-desktop__window_resize before screenshot collection.
• Use mcp__idle-engine-shell-desktop__window_devtools only when renderer debugging is needed.
• Always verify window_devtools effects with a follow-up window_info.
• In some headless environments, window_devtools may return success while devToolsOpen remains false; treat this as an environment limitation unless other tool calls fail.
• Use mcp__idle-engine-shell-desktop__window_screenshot to collect bounded PNG evidence.

Input Injection

• Use mcp__idle-engine-shell-desktop__input_controlEvent with explicit intent, phase, and optional value/metadata.
• Example:

{
  "intent": "collect",
  "phase": "start",
  "value": 1,
  "metadata": { "source": "mcp-skill-test" }
}

• After input injection, step or resume sim and capture resulting status and/or screenshot.

Full Surface Validation (Periodic)

Use this when validating MCP server coverage end-to-end (for example when updating this skill):

1. Run preflight + smoke.
2. Capture baseline: health, sim_status, window_info, renderer_status, probe_webgpuHealth.
3. Exercise window tools: window_resize, window_screenshot, window_devtools, window_info.
4. Exercise input/sim tools: input_controlEvent, sim_pause, sim_step, sim_enqueue, sim_resume, sim_stop, sim_start, sim_status.
5. Exercise asset tools: asset_list root, asset_list recursive, asset_read on a discovered file.
6. Exercise diagnostics tools: logs_tail (unfiltered + filtered) and logs_since with cursor polling.
7. Restore runtime state and window size (1200x800 unless scenario requires otherwise), then capture final baseline snapshot.

Report tool-by-tool status and any semantic gaps (for example "success response did not change observable state").

Diagnostics and Triage

• Use renderer.status (wrapper: mcp__idle-engine-shell-desktop__renderer_status) to capture #output text, optional error banner text, and renderer state snapshots.
• Use logs.tail (wrapper: mcp__idle-engine-shell-desktop__logs_tail) for bounded recent diagnostics with optional filters: source, subsystem, severity, limit.
• Use logs.since (wrapper: mcp__idle-engine-shell-desktop__logs_since) for incremental polling from a cursor id when tracking live recovery/regressions.
• Use probe.webgpuHealth (wrapper: mcp__idle-engine-shell-desktop__probe_webgpuHealth) for machine-readable WebGPU status (ok, lost, recovered) plus last loss reason/timestamp.
• Prefer dotted tool names (renderer.status, logs.since, probe.webgpuHealth) in runbooks; expect underscore mapping in Codex tool wrappers.

Asset Inspection

• Use mcp__idle-engine-shell-desktop__asset_list first.
• Use mcp__idle-engine-shell-desktop__asset_read only on discovered paths.
• Keep reads bounded with maxBytes.

Determinism and Evidence

For reproducible debugging, prefer:

1. sim_pause
2. sim_step with explicit step count
3. window_screenshot (bounded)
4. sim_status
5. Diagnostics snapshot (renderer.status + filtered logs.tail + probe.webgpuHealth) for renderer/WebGPU scenarios

Always report:

• exact nextStep
• step count used
• window size at screenshot time
• screenshot byte count
• diagnostics filters used (if any), nextCursor, and entry count when using log tools
• WebGPU probe status and last loss reason (if present)

Known Warnings

Do not treat these as root-cause failures by default when MCP calls succeed:

• DBus socket warnings in headless Linux
• DevTools autofill protocol warnings
• file descriptor comparison capability warning

Details: references/known-benign-warnings.md

Failure Handling

When preflight or smoke test fails:

1. Stop with a clear timeout/process-exited reason.
2. Include current health response.
3. Include launcher command used and elapsed attempts.
4. Do not proceed to scenario actions until preflight is healthy.

References

• Startup/runbook: docs/shell-desktop-mcp.md
• Tool sequencing examples: references/tool-workflows.md
• Benign warning interpretation: references/known-benign-warnings.md
• Preflight helper: scripts/preflight-and-wait.sh