Slurm Cloud Agent

Slurm Cloud Agent

Run a Cursor My Machines worker on this Slurm cluster from a login-node tmux session. The worker registers with Cursor and waits for cloud-agent tasks. When a task later runs shell commands, the existing ~/.cursor/skills/slurm/SKILL.md skill creates or reuses the cursor-agent Slurm allocation on demand. This skill manages everything by running ~/.cursor/skills/slurm-cloud-agent/scripts/slurm-cloud-agent.sh.

Quick start

1. Start a worker for rocm-systems:

~/.cursor/skills/slurm-cloud-agent/scripts/slurm-cloud-agent.sh start rocm-systems

2. Start a second worker for quasar against the same allocation:

~/.cursor/skills/slurm-cloud-agent/scripts/slurm-cloud-agent.sh start quasar

3. Inspect:

~/.cursor/skills/slurm-cloud-agent/scripts/slurm-cloud-agent.sh status

4. Stop a worker without touching the shared allocation:

~/.cursor/skills/slurm-cloud-agent/scripts/slurm-cloud-agent.sh stop rocm-systems

What it does

• Probes for salloc, srun, squeue, sinfo, tmux, git. Fails fast if any are missing.
• Installs the Cursor agent CLI from https://cursor.com/install if it is not already on PATH.
• Honors CURSOR_API_KEY / --auth-token-file / existing ~/.cursor/... credentials. Falls back to interactive agent login only if none are present.
• Clones or fast-forwards the repo profile under $CURSOR_WORKER_DIR_ROOT/<repo>.
• Spawns a per-repo cursor-worker-<repo> tmux session whose body is a worker watchdog: run agent worker start --name slurm-<repo> --worker-dir <dir> on the login node, restart on crash or disconnect.
• Uses lazy allocation: this skill never creates cursor-agent-alloc or calls salloc directly. The sibling slurm skill creates cursor-agent on the first shell command a cloud-agent task runs.
• Trigger the worker from chat surfaces with worker=slurm-<repo> (e.g. @Cursor worker=slurm-rocm-systems fix the flaky test).

Behavior rules

• Do not pass --pool to agent worker start. This skill targets My Machines, not Self-Hosted Pool.
• Always pass --worker-dir explicitly so the worker is bound to the right repo checkout.
• start <repo> replaces an existing cursor-worker-<repo> tmux session so script updates take effect immediately.
• stop <repo> only stops the worker tmux session. It does not release Slurm allocations; allocation lifecycle belongs to the slurm skill.
• This skill never creates cursor-agent-alloc. The slurm skill creates it lazily on the first shell command the worker runs, using the slurm skill's own GRES, CPU, memory, and time defaults.
• Never run workload commands directly from this skill. Runtime-sensitive work inside cloud-agent tasks must continue to follow the slurm skill and use srun --jobid="$JOBID" ....

Common overrides

Adding a new repo profile

Drop a file scripts/repos/<repo>.sh that sets at minimum:

REPO_NAME="myrepo"
REPO_URL="https://github.com/org/myrepo.git"
REPO_BRANCH="${REPO_BRANCH:-main}"
# Optional:
# post_clone_hook() { (cd "$1" && git submodule update --init); }

Then run ./scripts/slurm-cloud-agent.sh start <repo>.

Additional resources

• REFERENCE.md
• EXAMPLES.md
• ~/.cursor/skills/slurm/SKILL.md — sibling skill that owns the cursor-agent salloc convention.
• Cursor My Machines docs