Gcloud Vm

Google Cloud VMs

You have gcloud and can create/start/stop VMs yourself. Any time a task needs a real OS the Mac host and Docker cannot give you, spin one up. Project pullfrog, zone us-central1-a — gcloud's default zone is us-west1-a, so always pass --zone us-central1-a explicitly.

For Frizz specifically, the case that matters is Windows. tmux is gone, so Windows is a supported platform on paper; the only way to keep that true is to actually run there. See the Windows section below — the bring-up is fiddly and the recipe here is the one that works.

Auth: use the service-account key, not gcloud auth login

The user credential's refresh token gets revoked periodically by org session-control policy, so interactive login is not durable. A service-account key is exempt and works non-interactively:

export CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE=~/.config/pullfrog/vertex-service-account.json
gcloud compute instances list --project=pullfrog

It has Owner on pullfrog, so list/describe/start/stop/create/delete all work. Only fall back to ! gcloud auth login if this errors Reauthentication failed.

The standing instances, and when to make your own

They belong to the nub project and may be stopped, wedged, or configured for someone else's key — do not assume you can reach them. (Measured 2026-08-03: nub-win timed out during banner exchange and nub-win2 refused the nub-vm key with "Too many authentication failures".) When that happens, create your own throwaway box rather than fighting theirs, and delete it when done.

gcloud compute instances create frizz-win-tmp \
  --zone us-central1-a --project pullfrog \
  --machine-type e2-standard-4 \
  --image-family windows-2022 --image-project windows-cloud \
  --boot-disk-size 50GB

gcloud compute instances delete frizz-win-tmp --zone us-central1-a --quiet   # a created VM bills its disk even when STOPPED

SSH — user nub, key ~/.ssh/nub-vm, and the IP is DYNAMIC

Re-read the IP on every reconnect, not once per session. A box can be stopped out from under you mid-run and come back on a different address, so a sudden timeout on a previously-working box usually means "it moved", not "it wedged".

IP=$(gcloud compute instances describe "$NAME" --zone us-central1-a --project=pullfrog \
      --format='value(networkInterfaces[0].accessConfigs[0].natIP)')
ssh -i ~/.ssh/nub-vm -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \
    -o IdentitiesOnly=yes -o ConnectTimeout=15 nub@"$IP" 'echo ok'

-o IdentitiesOnly=yes matters: without it ssh offers every key in your agent and the server closes the connection with "Too many authentication failures" before it ever tries the right one.

Always reachability-guard a dispatch (ConnectTimeout, timeout) and retry with backoff after a fresh start — sshd is not up the instant STATUS flips RUNNING.

Windows bring-up — the part that is NOT documented anywhere else

A fresh windows-2022 image will NOT accept SSH, and the failure is silent. Four things must be true, and the official enable-windows-ssh=TRUE metadata alone gets you none of them. Do all of it in ONE create so you only pay for one reset:

1. The image ships no OpenSSH Server. The guest agent logs Could not determine if openssh version is compatible: could not find version and gives up. Install it from a startup script.
2. The guest agent only creates the nub account when it sees ssh-keys metadata, and after the first pass it says No new keys found, skipping account setup and never writes authorized_keys. So pass the metadata AND write the key yourself.
3. nub is an Administrator, and Windows OpenSSH ignores ~/.ssh/authorized_keys for admins. The key must go in C:\ProgramData\ssh\administrators_authorized_keys, with inheritance stripped.
4. A startup script only runs at boot, so add-metadata must be followed by gcloud compute instances reset.

# win-ssh-setup.ps1 — ASCII ONLY (see the codepage gotcha below)
Add-WindowsCapability -Online -Name OpenSSH.Server~~~~0.0.1.0 -ErrorAction SilentlyContinue
Set-Service -Name sshd -StartupType Automatic
Start-Service sshd -ErrorAction SilentlyContinue
$key = '<contents of ~/.ssh/nub-vm.pub>'
New-Item -ItemType Directory -Force -Path "C:\Users\nub\.ssh" | Out-Null
Set-Content -Path "C:\Users\nub\.ssh\authorized_keys" -Value $key -Encoding ascii
$adm = "C:\ProgramData\ssh\administrators_authorized_keys"
Set-Content -Path $adm -Value $key -Encoding ascii
icacls.exe $adm /inheritance:r /grant "Administrators:F" /grant "SYSTEM:F" | Out-Null
Restart-Service sshd -ErrorAction SilentlyContinue

gcloud compute instances create frizz-win-tmp --zone us-central1-a --project pullfrog \
  --machine-type e2-standard-4 --image-family windows-2022 --image-project windows-cloud \
  --boot-disk-size 50GB \
  --metadata enable-windows-ssh=TRUE,ssh-keys="nub:$(cat ~/.ssh/nub-vm.pub)" \
  --metadata-from-file windows-startup-script-ps1=win-ssh-setup.ps1
gcloud compute instances reset frizz-win-tmp --zone us-central1-a --project pullfrog

Then poll for $env:PROCESSOR_ARCHITECTURE to come back AMD64. Read the progression: Operation timed out → the network path or sshd is not up; Connection refused → the path is open and sshd is not listening; Disconnected from … port 22 → sshd is up and your key is being rejected (go fix administrators_authorized_keys).

Windows gotchas that each cost a cycle

• The home directory is C:\Users\nub.<HOSTNAME>, not C:\Users\nub. Always use $env:USERPROFILE; a hardcoded C:\Users\nub\… silently misses.
• nub.exe needs the MSVC runtime. A bare Server 2022 has none, and the only symptom is exit code -1073741515 (0xC0000135, DLL_NOT_FOUND) with no message. Install vc_redist.x64.exe first.
• This repo needs pnpm, not npm. npm install fails EUNSUPPORTEDPROTOCOL Unsupported URL Type "workspace:". Install pnpm@10 globally, then pnpm install --no-frozen-lockfile.
• Write every remote PowerShell script as ASCII + CRLF. PowerShell 5.1 reads a BOM-less script in the ANSI codepage, so one UTF-8 character anywhere — an em-dash in a comment is the usual culprit — fails with The string is missing the terminator, and the error points at the LAST line of the file rather than the offending one.
• -File with a path containing the profile directory can fail to resolve; use -Command "& \"$env:USERPROFILE\script.ps1\"" so the guest expands the path.
• PowerShell's Tee-Object/Out-File write UTF-16LE by default. A log you scp back will look like it has NUL bytes between every character; iconv -f UTF-16LE -t UTF-8, or pass -Encoding utf8.
• IsOutputRedirected is always True over SSH, so any is_terminal() branch takes the non-TTY path and first-run console behavior cannot be exercised. That needs a ConPTY harness or RDP.
• A long run will outlive a foreground command. Redirect to a file on the GUEST, run the SSH command in the background, and poll the guest-side file — an SSH pipe left open for ten minutes gets client_loop: send disconnect: Broken pipe, and the harness reports the pipeline's exit code, so a dead run arrives labelled "exit 0".

Other gotchas

• A RUNNING instance can be a DEAD instance — read the serial console FIRST.

  gcloud compute instances get-serial-port-output "$NAME" --zone us-central1-a | tail -40
  

  This diagnosed an 11-day SSH-dead box instantly (an OOM had wedged sshd). It beats guessing "network problem" every time, and on Windows it is the only way to see what the guest agent is doing.
• Judge results by behavioral/differential evidence — EPERM vs success, a before/after delta, a control that must still fail — not wall-clock, since a shared VM may be contended.
• Delete your box. gcloud compute instances delete <name> --zone us-central1-a --quiet. Verify with instances list that only the pre-existing ones remain.

Worked example: what this caught

Driving a real windows-2022 box on 2026-08-03 found two Windows bugs that every local gate had passed: a Codex cwd validator gated on startsWith("/") (rejecting every C:\ path — 41 tests), and the broker resolving claude to npm's #!/bin/sh shim instead of the native claude.exe, which broke every Claude dispatch on Windows. The second one also killed the *obvious* fix: preferring .cmd via PATHEXT returns a path Node refuses to spawn without a shell (EINVAL, CVE-2024-27980). Only running it on the real OS surfaced that — reasoning about it produced a confident wrong answer.