E2e Exe Dev

e2e on exe.dev

For each unexpected failure, preserve the test evidence, then follow the companion e2e-triage at a safe checkpoint and include its findings and next actions in the final E2E report. Research each distinct failure once, including when another E2E skill delegates here; do not delay active prompts, change acceptance results or submit public issues/comments without authorization. Install e2e-triage alongside this skill. If its relative link is unavailable, resolve it by skill name in the agent's installed catalog; if absent, report that triage was unavailable and continue the authorized test/report without installing anything implicitly.

For an installation on an existing Mac, locally or over SSH, use e2e-macos. This skill's target workflow is Linux; its installer is shared with the Mac driver and its LaunchAgent helper.

For the real public interactive installer, use the companion e2e-wizard and this driver's --interactive mode. The workflow below remains the headless test; its setup repairs and acceptance results do not qualify the wizard.

Installs NanoClaw on a disposable exe.dev VM without a human at the keyboard and proves the whole loop: message in over the cli channel → host → container → Claude → reply out. It reuses the setup wizard's own step processes (pnpm exec tsx setup/index.ts --step …) in the wizard's order — no parallel install logic to drift — and replaces each prompt with an env var or a pre-seeded state. The assertion is the wizard's first-chat probe (setup/lib/agent-ping.ts): pnpm run chat ping must return a reply.

Ships in nanocoai/nanoclaw-oss-dev-tools as skills/e2e-exe-dev, in the portable Agent Skills format. Install it into any agent with npx skills add nanocoai/nanoclaw-oss-dev-tools, or in Claude Code with /plugin install nanoclaw-e2e@nanoclaw-oss-dev-tools (there it is invoked as /nanoclaw-e2e:e2e-exe-dev). In Codex, invoke $e2e-exe-dev. Resolve the scripts relative to this SKILL.md, then run them from the root of the NanoClaw checkout you want tested. The installed skill and the checkout being tested are separate directories. Install e2e-wizard alongside this skill; its shared read-only helper discovers the exact provider picker.

Three scripts under scripts/:

When to use

• A branch or PR touches setup, the container image, OneCLI wiring, the service, or the router/delivery path and unit tests can't prove it.
• Someone reports an install failure you want to reproduce on a clean machine.
• You want a base VM with Docker, OneCLI and the built image already in place so later runs skip the 3–10 minute build.

Not for: wiring real channels (Slack, Telegram, …). The e2e agent talks over the always-on cli channel (src/channels/cli.ts) so no platform credentials are involved. Use /manage-channels on the VM afterwards if you want more.

Prerequisites

• An exe.dev account with ssh exe.dev working from your machine. exe.dev has two SSH destinations: ssh exe.dev <cmd> is the lobby (VM lifecycle only — no shell, no scp) and ssh <ssh_dest> is the VM (full shell). Pin the key for both so non-interactive runs never stall on key selection:

  Host exe.dev *.exe.xyz
    IdentitiesOnly yes
    IdentityFile ~/.ssh/id_ed25519
  

  exe.dev publishes its own agent skill (using-exe-dev, in skill/SKILL.md of github.com/boldsoftware/exe.dev) and docs at https://exe.dev/docs.md (index) / https://exe.dev/docs/all.md (one page). ssh exe.dev help <command> is the authoritative flag reference; this skill only adds the NanoClaw side.

• For the headless Claude path, an Anthropic API key or OAuth token in a local file, default ~/.nanoclaw-e2e/anthropic_key. It goes to the VM over stdin into a 0600 file; the installer seeds it into the OneCLI vault exactly as setup/auth.ts does (onecli secrets create --type anthropic --host-pattern api.anthropic.com).

• A Debian/Ubuntu image on the VM (exe.dev's default). Both bootstrap helpers are apt-based: setup/install-node.sh (NodeSource) and setup/install-docker.sh (get.docker.com + usermod -aG docker). Docker runs inside exe.dev VMs.

Choose the provider and authentication first

Resolve the exact NanoClaw commit and inspect its provider picker before reading a credential or provisioning anything. The discovery helper belongs to the sibling e2e-wizard skill and reads Git objects, so uncommitted files cannot change the choices:

COMMIT="$(git rev-parse --verify HEAD^{commit})"
PROVIDER_HELPER=/absolute/path/to/installed/e2e-wizard/scripts/provider-options.py
python3 "$PROVIDER_HELPER" --root "$PWD" --revision "$COMMIT"

Show the reported providers to the operator and ask which provider to test. Then inspect that provider and show its exact auth_prompt and auth_methods:

python3 "$PROVIDER_HELPER" --root "$PWD" --revision "$COMMIT" --provider claude

For an installable provider, first fetch the one nc:copy from-branch: payload named by its offered skill from the owning remote, then pass that fetched ref as --payload-ref. Record both nanoclaw_commit and auth_source_commit. Never guess a payload remote or silently choose a provider or auth method.

The unattended headless installer currently supports Claude api and oauth. Its existing mode is only for an explicitly selected reused gateway whose vault already has a usable Anthropic credential; it is not a public-picker choice. The public wizard can also automate credential-file methods exposed by other offered providers. Browser, subscription, or device methods require a separately authorized live human handoff; these drivers stop before allocation because they cannot complete that handoff unattended. skip cannot produce an E2E pass. Read a credential only after the operator chooses its matching method.

Workflow

1. Run it from the checkout root, on the branch you want tested. Set E2E_SKILL_DIR to the actual directory containing this SKILL.md:

   E2E_SKILL_DIR=/absolute/path/to/installed/e2e-exe-dev
   cd /absolute/path/to/nanoclaw
   bash "$E2E_SKILL_DIR/scripts/exe-run.sh" \
     --provider claude --auth-method api --credential-file /path/to/anthropic-key \
     --result-file /path/to/results/exe-headless.json
   bash "$E2E_SKILL_DIR/scripts/exe-run.sh" --ref origin/main --name nc-main \
     --provider claude --auth-method oauth --credential-file /path/to/anthropic-oauth-token \
     --result-file /path/to/results/exe-main.json
   

   Keep --result-file on every live run so the report and sanitized evidence exist before the post-run retention choice. For an already authorized disposable run, add --rm (the result parent directory must exist). --rm is rejected without --result-file; <result-file>.artifacts must be a new local path so teardown cannot precede validated evidence persistence.

   --ref resolves in the local checkout to an exact commit before creating a VM. Fetch locally first if you want an updated origin/main. The VM fetches that SHA from --repo (default: local origin), checks it out detached, and verifies HEAD matches. The commit must be fetchable from that repository; local-only commits and uncommitted edits are not uploaded. The installer comes from the installed skill, so compatible older NanoClaw refs can be tested too.

2. Read the result. The last thing printed on success is

   === NANOCLAW E2E: RESULT ===
   STATUS: pass
   COMMIT: <full tested SHA>
   SERVICE_TYPE: systemd-user | systemd-system | nohup
   PING: ok
   REPLY: <first 200 chars of the agent's reply>
   RESULT: logs/e2e/result.json
   === END ===
   

   Exit codes: 0 pass · 1 a step failed (its === NANOCLAW SETUP: … === block is printed right above the failure; raw output in ~/nanoclaw/logs/e2e/<step>.log on the VM) · 2 no reply / auth error · 3 the host never opened data/cli.sock or the socket was unreachable. The driver uses 64 for invalid arguments, 65 for checkout/ref errors, 66 for an unreadable key, 69 for unconfirmed VM creation/reachability, 70 for unconfirmed snapshot creation, and 74 for local result writing/export failure; other command failures can return their own nonzero exit code.

   After installer preflight, logs/e2e/result.json records pass/failure, exit code, tested commit, tracked changes present at start, phase, service type, ping classification and timestamps. An in-progress run has status running. It replaces a previous run's result and contains no credentials or reply text. With --result-file, after argument parsing and destination validation the driver replaces any old local report with running before checking the checkout, credentials or VM. It exports a matching completed installer result and a sanitized evidence directory before snapshot/removal. The evidence binds the run ID, NanoClaw SHA, provider, auth method, provider auth-source SHA, dev-tools SHA when available, and exact harness digest. It includes checksummed setup/runtime logs and a service/container/socket state snapshot. Otherwise the driver records the driver failure, exit code, phase and requested ref/commit; commit is null because no tested revision was confirmed. An interrupted run may remain running. A failed export keeps the VM. Once exported, the installer result describes the test; snapshot/removal errors are reported by the driver's exit code.

   On the VM, a checkout failure on a base leaves a running record with phase checkout and the requested commit, replacing any old pass. Failures before installer preflight on a fresh VM may have no remote result file; the local driver report still records the failure.

3. Bake a base VM once the run is green, then clone it for every later run:

   bash "$E2E_SKILL_DIR/scripts/exe-run.sh" --snapshot my-nanoclaw-e2e-base \
     --provider claude --auth-method api --credential-file /path/to/anthropic-key \
     --result-file /path/to/results/base.json
   bash "$E2E_SKILL_DIR/scripts/exe-run.sh" --base my-nanoclaw-e2e-base --ref my-branch \
     --provider claude --auth-method api --credential-file /path/to/anthropic-key \
     --result-file /path/to/results/cached.json
   

   VM names are global across exe.dev (they become <name>.exe.xyz), so pick a distinctive snapshot name. Default run names include a random suffix. A taken name or invalid create/copy response stops the run; an existing VM is never adopted as a replacement. A failed snapshot keeps the tested VM, even with --rm.

   Every step is idempotent against the cloned state: onecli --reuse when .env already has ONECLI_URL, auth --check short-circuits on an existing vault secret, container rebuilds only what changed, init-cli-agent reuses the group and wiring. Cached checkouts are pointed at the requested repository and exact SHA; tracked edits in a base checkout cause a failure instead of being overwritten.

4. Poke at it. ssh <ssh_dest> (printed at the end; ssh exe.dev ls --json lists it), then in ~/nanoclaw: pnpm run chat hi, bin/ncl groups list, tail -f logs/nanoclaw.log.

5. Choose retention after evidence is safe. Recommend deleting a disposable, run-owned VM after a pass and retaining an unexpected failure until triage is complete. Offer the operator that choice after reporting the result. With --rm, the driver revalidates its run marker, confirms no harness controller is active, calls ssh exe.dev rm <name> --json, verifies the name is absent from ssh exe.dev ls --json, and writes teardown-receipt.json into the validated artifact directory. Any failed export, checksum, redaction, local persistence, ownership, controller, or absence check blocks or leaves teardown unconfirmed. For manual cleanup, repeat the same checks and record the receipt; inventory lookup alone never proves ownership.

Running the installer somewhere else

e2e-install.sh has no exe.dev dependency. On a CI runner or any fresh box:

git clone https://github.com/nanocoai/nanoclaw-oss-dev-tools.git
E2E_SKILL_DIR="$(pwd)/nanoclaw-oss-dev-tools/skills/e2e-exe-dev"
git clone https://github.com/nanocoai/nanoclaw.git
cd nanoclaw
NANOCLAW_E2E_PROVIDER=claude NANOCLAW_E2E_AUTH_METHOD=api \
NANOCLAW_E2E_AUTH_SOURCE_COMMIT="$(git rev-parse HEAD)" \
NANOCLAW_E2E_KEY_FILE=/path/to/key bash "$E2E_SKILL_DIR/scripts/e2e-install.sh"

The target needs git and python3 before starting the installer. The exe.dev driver installs these if missing; provision them yourself on other hosts.

Env it honors: NANOCLAW_E2E_ROOT, NANOCLAW_E2E_KEY_FILE, NANOCLAW_ONECLI_API_HOST + NANOCLAW_ONECLI_API_TOKEN (remote gateway instead of a local install — the same vars setup/auto.ts reads), NANOCLAW_DISPLAY_NAME, NANOCLAW_E2E_TZ (default UTC), NANOCLAW_E2E_KEEP_AGENT (default 1), NANOCLAW_E2E_FORCE_AUTH (1 replaces an existing vault secret with the key file — token rotation on a --base VM whose snapshot still holds the old one).

The driver forwards the remote gateway settings, display name, timezone, keep-agent and force-auth settings over stdin into a private environment file, which is removed after loading. NANOCLAW_E2E_KEY_FILE (or --key-file) selects the local key to upload; NANOCLAW_E2E_ROOT applies to standalone installer runs. The driver always uses ~/nanoclaw on its VM.

Compatibility evidence

On 2026-09-12 a fresh exe.dev VM failed at the onecli step against NanoClaw fd767d381fe5a1fea51373e4e8cb411da79880ec with Port 5432 is already in use and nothing listening: the image had started shipping /exe.dev/bin/sh first on PATH (see the gotcha below). With the installer's system-shell preflight, a new fresh VM passed end to end against d96dde93db7a766531ca288190ca202ad473547e: the note line reported /exe.dev/bin removed from PATH, every step from environment to verify succeeded, SERVICE_TYPE: nohup, PING: ok, exit 0, about 3 minutes 20 seconds including OS bootstrap, and the VM was removed with --rm. NanoClaw itself pins /bin/sh in those steps from nanocoai/nanoclaw#3776, so later refs no longer depend on this preflight.

Fresh and cached VM installations passed on 2026-09-10 against NanoClaw 74224f62a6c08418acccc727114ab02f92e403bf. Both produced real model replies, SERVICE: running, and successful final verification using the nohup fallback. The cached run reused OneCLI, the vault credential, Docker build layers and the existing agent wiring.

The fresh install passed with skill version 0.2.3, but its subsequent snapshot returned driver exit 70: exe.dev created the copy using a response shape that the driver did not recognize. Version 0.2.4 accepts that shape only when its source and destination names match the request. The corrected driver's live cached-copy run completed with exit 0; offline regressions also cover snapshot confirmation and mismatched copy responses.

Older NanoClaw refs, including 2c754a2234390fcc597273cef6344d99e8ac03d0, could fail verification after a successful ping when systemd's user bus was unavailable. The live-tested ref fixes this by consulting the nohup PID file when no service is found. Check the selected ref before assuming compatibility.

What the installer calls, and why each call is shaped that way

Everything below is a wizard step or a script that already ships; the installer adds sequencing and assertions only.

Step output is parsed from the === NANOCLAW SETUP: <STEP> === … STATUS: … === END === block every step prints (setup/status.ts).

Gotchas (each one cost a wrong assumption)

• sh on PATH may not be the system shell. exe.dev images since 2026-09-09 put /exe.dev/bin first on PATH with their own sh, whose builtin lsof always exits 0. NanoClaw's setup pipes downloaded installers into sh (setup/onecli.ts, setup/install-docker.sh), so the OneCLI installer's port probe reported every port as busy — Port 5432 is already in use (probably a local PostgreSQL) with nothing listening, and the same for POSTGRES_PORT=5433. The installer now drops the directory holding a foreign sh from PATH before the first step (it prints a note: line) and stops when no system shell exists at /bin/sh or /usr/bin/sh. The product-side fix pins /bin/sh in those steps.
• The wizard is not headless. pnpm run setup:auto prompts for start mode, existing-install action and OneCLI reuse with no env bypass, and nanoclaw.sh reads its root warning from /dev/tty. Only the step runner (setup/index.ts --step) is prompt-free — that's why the installer is built on it and never touches nanoclaw.sh/setup:auto.
• No user systemd ⇒ nothing starts. setupNohupFallback writes start-nanoclaw.sh and reports SERVICE_LOADED: false; it does not run it. The installer starts it. exe.dev VMs are exactly this shape: systemd is PID 1 but systemctl --user fails ("Failed to connect to bus"), so service falls back to nohup. verify.ts used to check nanoclaw.pid only when there was no systemd at all and reported SERVICE: not_found here. The live-tested SHA above fixes this by checking the PID file whenever no service is found; older NanoClaw refs may still have the limitation.
• A non-interactive SSH shell has no ~/.local/bin on PATH — pnpm, node, onecli all live there. Prefix ad-hoc commands on the VM with export PATH="$HOME/.local/bin:$PATH"; the installer does this itself.
• Treat the token as exposed once it has been in any log. The installer redacts --value in its own output, but the OAuth token is long-lived: rotate it (revoke, claude setup-token) after a run whose logs left the machine, and re-seed base VMs with NANOCLAW_E2E_FORCE_AUTH=1.
• Keep the agent, or verify fails. The wizard deletes its ping_test group after the ping (scripts/delete-cli-agent.ts) before running verify, which then relies on a channel deferring its wiring. With no channel configured that's registeredGroups: 0 → failed. The installer keeps e2e-agent (set NANOCLAW_E2E_KEEP_AGENT=0 to delete it after the ping; expect verify to fail then).
• The vault is not optional. Spawn refuses without a gateway contribution (src/gateway-providers/onecli.ts: "refusing to spawn container without credentials"), and onecli is the only provider in trunk. There is no API-key-in-env shortcut.
• Docker is the only driver. src/drivers/index.ts ships docker alone; NANOCLAW_RUNTIME_DRIVER set to anything else throws at startup.
• First container boot is 30–60 s after the ping is sent; chat.ts waits up to 120 s and the installer wraps it in timeout 150.
• GITHUB_TOKEN in the environment counts as a configured channel in verify.ts (has() reads process.env). The installer unsets it so the report describes the install, not the shell.
• Egress the VM must have: deb.nodesource.com, get.docker.com, onecli.sh, github.com (OneCLI CLI release tarball), Docker Hub (base image), api.anthropic.com. exe.dev VMs have a full network stack; locked- down runners do not.
• The key is on argv for one process. auth --create --value is how setup/auth.ts accepts it (it then execFileSyncs onecli, no shell), so it is visible to ps on the VM for that step. Fine for a disposable VM; use NANOCLAW_ONECLI_API_HOST with a pre-seeded remote vault if not.
• Don't synthesize the VM hostname. new --json returns vm_name and ssh_dest; the current cp --json response uses name, source and ssh_host. The driver checks the requested destination name and, for that copy format, the source name too. It also accepts the older vm_name response shape. Modern destinations look like <name>.exe.xyz, legacy ones like vm+<name>@vm.exe.xyz; the driver passes the returned destination to SSH. It never falls back to ls --json to infer creation success.
• First contact blocks on the host-key prompt in a non-interactive shell with nothing visible. Every VM connection in the driver carries -o StrictHostKeyChecking=accept-new.
• The lobby is not a shell. scp/sftp/commands against exe.dev fail; files go to the VM destination, and the driver uses ssh <vm> 'cat > file' < local (exe.dev's documented scp-less path).
• exe.dev --setup-script is capped at 10 KiB and runs at first boot, before you can hand it a secret safely. The driver therefore SSHes in after boot instead — no size limit, secrets over stdin.

Troubleshooting

Teardown

ssh exe.dev rm <name> when done — the VM's persistent disk holds the vault secret. A standalone installer writes setup state and logs into its target checkout; the driver runs setup on the VM. Uninstall the plugin with /plugin uninstall nanoclaw-e2e@nanoclaw-oss-dev-tools.