Flux job management
Flux is a resource manager / scheduler used on HPC clusters. Use it to run anything heavy — builds, test suites, model serving, data processing — as a job on compute nodes. Do not run heavy work directly on the login node.
Queue names, bank/account, and node counts are cluster-specific. This skill
covers the generic commands only; get the per-machine values (queue, bank,
typical -N/-n) from the project's machine-specific setup notes before
submitting. In general, you want to use --exclusive when you request nodes.
Interactive allocations
flux alloc -B <bank> --exclusive -N1 -q pdebug -t 1h # open a new shell with the allocated nodes
flux alloc -B <bank> --exclusive -N4 -t 8h
Submitting jobs
flux submit ./script.sh # queue a job; prints a job ID and returns immediately
flux run ./script.sh # run interactively and block until it finishes
flux batch ./batch.sh # submit a batch script (directives via '# flux:' lines)
flux submit -N2 -n8 ./script.sh # request 2 nodes, 8 tasks
flux submit --queue=<queue> --name=<name> ./script.sh # target a queue, name the job
A batch script declares its resources with # flux: directive lines, e.g.:
#!/bin/sh
# flux: -N4 -n16
flux run -n16 ./my_step.sh
Checking whether a job is running
flux jobs # your active jobs (pending + running)
flux jobs -a # include completed / inactive jobs
flux jobs --filter=running # only running jobs (also: pending, inactive)
flux job last # job ID of your most recent submission
Reading output and exit code
flux submit --watch ./script.sh # stream output live as it runs
flux job attach $(flux job last) # attach to / print output of the last job
flux submit --output=job-{{id}}.out ./script.sh # write stdout to a file named per job ID
flux jobs --no-header -o '{status}:{returncode}' <jobid> # status + exit code of one job
A non-zero returncode means the job failed — inspect its output before assuming
the step succeeded. Do not report a job as "passed" until you have confirmed both
that it is inactive and that its return code is 0.
Cancelling jobs
flux cancel <jobid> # cancel a single job
flux cancel --all # cancel all of your jobs
flux cancel --states=RUN # cancel jobs in a given state
Inspecting resources
flux resource list # nodes available to you and their state
flux uptime # is the Flux instance up, and for how long
flux overlay status # health of the Flux overlay network
Notes for agents
- Prefer
flux submit(non-blocking) for long work, then poll withflux jobs; useflux runonly for quick interactive checks. - Always capture the job ID from
flux submit(or useflux job last) so you can check status and output later. - Never launch a full model-serving stack or a long test run on the login node — submit it as a Flux job.
Full command reference: https://flux-framework.org/cheat-sheet/