SLURM Job Script Generator
Goal
Generate a correct, copy-pasteable SLURM job script (.sbatch) for running a simulation, and surface common configuration mistakes (bad walltime format, conflicting memory flags, oversubscription hints).
Requirements
- Python 3.8+
- No external dependencies (Python standard library only)
- Works on Linux, macOS, and Windows (script generation only)
Inputs to Gather
| Input |
Description |
Example |
| Job name |
Short identifier for the job |
phasefield-strong-scaling |
| Walltime |
SLURM time limit |
00:30:00 |
| Partition |
Cluster partition/queue (if required) |
compute |
| Account |
Project/account (if required) |
matsim |
| Nodes |
Number of nodes to allocate |
2 |
| MPI tasks |
Total tasks, or tasks per node |
128 or 64 per node |
| Threads |
CPUs per task (OpenMP threads) |
2 |
| Memory |
--mem or --mem-per-cpu (cluster policy dependent) |
32G |
| GPUs |
GPUs per node (optional) |
4 |
| Working directory |
Where the run should execute |
$SLURM_SUBMIT_DIR |
| Modules |
Environment modules to load (optional) |
gcc/12, openmpi/4.1 |
| Run command |
The command to launch under SLURM |
./simulate --config cfg.json |
Decision Guidance
MPI vs MPI+OpenMP layout
Does the code use OpenMP / threading?
├── NO → Use MPI-only: cpus-per-task=1
└── YES → Use hybrid: set cpus-per-task = threads per MPI rank
and export OMP_NUM_THREADS = cpus-per-task
Rule of thumb: if you see diminishing strong-scaling efficiency at high MPI ranks, try fewer ranks with more threads per rank (and measure).
Memory flag selection
- Use either
--mem (per node) or --mem-per-cpu (per CPU), not both.
- Follow your cluster’s documentation; some sites enforce one style.
- SLURM
--mem units are integer MB by default, or an integer with suffix K/M/G/T (and --mem=0 commonly means “all memory on node”).
Script Outputs (JSON Fields)
| Script |
Key Outputs |
scripts/slurm_script_generator.py |
results.script, results.directives, results.derived, results.warnings |
Workflow
- Gather cluster constraints (partition/account, GPU policy, memory policy).
- Choose a process layout (MPI-only vs hybrid MPI+OpenMP).
- Generate the script with
slurm_script_generator.py.
- Inspect warnings (conflicts, suspicious layouts).
- Save the generated script as
job.sbatch.
- Submit with
sbatch job.sbatch and monitor with squeue.
CLI Examples
# Preview a job script (prints to stdout)
python3 skills/hpc-deployment/slurm-job-script-generator/scripts/slurm_script_generator.py \
--job-name phasefield \
--time 00:10:00 \
--partition compute \
--nodes 1 \
--ntasks-per-node 8 \
--cpus-per-task 2 \
--mem 16G \
--module gcc/12 \
--module openmpi/4.1 \
-- \
./simulate --config config.json
# Write to a file and also emit structured JSON
python3 skills/hpc-deployment/slurm-job-script-generator/scripts/slurm_script_generator.py \
--job-name phasefield \
--time 00:10:00 \
--nodes 1 \
--ntasks 16 \
--cpus-per-task 1 \
--out job.sbatch \
--json \
-- \
/bin/echo hello
Conversational Workflow Example
User: I need an sbatch script for my MPI simulation. I want 2 nodes, 64 ranks per node, 2 OpenMP threads per rank, and 2 hours.
Agent workflow:
- Confirm partition/account and whether GPUs are needed.
- Generate a hybrid job script:
python3 scripts/slurm_script_generator.py --job-name run --time 02:00:00 --nodes 2 --ntasks-per-node 64 --cpus-per-task 2 -- -- ./simulate
- Explain the mapping:
- Total ranks = 128
- Threads per rank = 2 (
OMP_NUM_THREADS=2)
- If the user provides node core counts, sanity-check oversubscription using
--cores-per-node.
Error Handling
| Error |
Cause |
Resolution |
time must be HH:MM:SS or D-HH:MM:SS |
Bad walltime format |
Use 00:30:00 or 1-00:00:00 |
nodes must be positive |
Non-positive nodes |
Provide --nodes >= 1 |
Provide either --mem or --mem-per-cpu, not both |
Conflicting memory directives |
Choose one memory style |
Provide a run command after -- |
Missing launch command |
Add -- ./simulate ... |
Limitations
- Does not query cluster hardware or site policies; it can only validate internal consistency.
- SLURM installations vary (GPU directives, QoS rules, partitions). Adjust directives for your site.
References
references/slurm_directives.md - Common #SBATCH directives and mapping tips
Version History
- v1.0.0 (2026-02-25): Initial SLURM job script generator
1---2name: slurm-job-script-generator3description: Generate SLURM `sbatch` job scripts and sanity-check HPC resource requests (nodes, tasks, CPUs, memory, GPUs) for simulation runs. Use when preparing submission scripts, deciding MPI vs MPI+OpenMP layouts, standardizing `#SBATCH` directives, or debugging job launch configuration (`sbatch`/`srun`).4---5
6# SLURM Job Script Generator
7
8## Goal
9
10Generate a correct, copy-pasteable SLURM job script (`.sbatch`) for running a simulation, and surface common configuration mistakes (bad walltime format, conflicting memory flags, oversubscription hints).
11
12## Requirements
13
14- Python 3.8+
15- No external dependencies (Python standard library only)
16- Works on Linux, macOS, and Windows (script generation only)
17
18## Inputs to Gather
19
20| Input | Description | Example |
21|-------|-------------|---------|
22| Job name | Short identifier for the job | `phasefield-strong-scaling` |
23| Walltime | SLURM time limit | `00:30:00` |
24| Partition | Cluster partition/queue (if required) | `compute` |
25| Account | Project/account (if required) | `matsim` |
26| Nodes | Number of nodes to allocate | `2` |
27| MPI tasks | Total tasks, or tasks per node | `128` or `64` per node |
28| Threads | CPUs per task (OpenMP threads) | `2` |
29| Memory | `--mem` or `--mem-per-cpu` (cluster policy dependent) | `32G` |
30| GPUs | GPUs per node (optional) | `4` |
31| Working directory | Where the run should execute | `$SLURM_SUBMIT_DIR` |
32| Modules | Environment modules to load (optional) | `gcc/12`, `openmpi/4.1` |
33| Run command | The command to launch under SLURM | `./simulate --config cfg.json` |
34
35## Decision Guidance
36
37### MPI vs MPI+OpenMP layout
38
39```
40Does the code use OpenMP / threading?
41├── NO → Use MPI-only: cpus-per-task=1
42└── YES → Use hybrid: set cpus-per-task = threads per MPI rank
43 and export OMP_NUM_THREADS = cpus-per-task
44```
45
46**Rule of thumb:** if you see diminishing strong-scaling efficiency at high MPI ranks, try fewer ranks with more threads per rank (and measure).
47
48### Memory flag selection
49
50- Use **either** `--mem` (per node) **or** `--mem-per-cpu` (per CPU), not both.
51- Follow your cluster’s documentation; some sites enforce one style.
52- SLURM `--mem` units are integer MB by default, or an integer with suffix `K/M/G/T` (and `--mem=0` commonly means “all memory on node”).
53
54## Script Outputs (JSON Fields)
55
56| Script | Key Outputs |
57|--------|-------------|
58| `scripts/slurm_script_generator.py` | `results.script`, `results.directives`, `results.derived`, `results.warnings` |
59
60## Workflow
61
621. Gather cluster constraints (partition/account, GPU policy, memory policy).
632. Choose a process layout (MPI-only vs hybrid MPI+OpenMP).
643. Generate the script with `slurm_script_generator.py`.
654. Inspect warnings (conflicts, suspicious layouts).
665. Save the generated script as `job.sbatch`.
676. Submit with `sbatch job.sbatch` and monitor with `squeue`.
68
69## CLI Examples
70
71```bash
72# Preview a job script (prints to stdout)
73python3 skills/hpc-deployment/slurm-job-script-generator/scripts/slurm_script_generator.py \
74 --job-name phasefield \
75 --time 00:10:00 \
76 --partition compute \
77 --nodes 1 \
78 --ntasks-per-node 8 \
79 --cpus-per-task 2 \
80 --mem 16G \
81 --module gcc/12 \
82 --module openmpi/4.1 \
83 -- \
84 ./simulate --config config.json
85
86# Write to a file and also emit structured JSON
87python3 skills/hpc-deployment/slurm-job-script-generator/scripts/slurm_script_generator.py \
88 --job-name phasefield \
89 --time 00:10:00 \
90 --nodes 1 \
91 --ntasks 16 \
92 --cpus-per-task 1 \
93 --out job.sbatch \
94 --json \
95 -- \
96 /bin/echo hello
97```
98
99## Conversational Workflow Example
100
101**User**: I need an `sbatch` script for my MPI simulation. I want 2 nodes, 64 ranks per node, 2 OpenMP threads per rank, and 2 hours.
102
103**Agent workflow**:
1041. Confirm partition/account and whether GPUs are needed.
1052. Generate a hybrid job script:
106 ```bash
107 python3 scripts/slurm_script_generator.py --job-name run --time 02:00:00 --nodes 2 --ntasks-per-node 64 --cpus-per-task 2 -- -- ./simulate
108 ```
1093. Explain the mapping:
110 - Total ranks = 128
111 - Threads per rank = 2 (`OMP_NUM_THREADS=2`)
1124. If the user provides node core counts, sanity-check oversubscription using `--cores-per-node`.
113
114## Error Handling
115
116| Error | Cause | Resolution |
117|-------|-------|------------|
118| `time must be HH:MM:SS or D-HH:MM:SS` | Bad walltime format | Use `00:30:00` or `1-00:00:00` |
119| `nodes must be positive` | Non-positive nodes | Provide `--nodes >= 1` |
120| `Provide either --mem or --mem-per-cpu, not both` | Conflicting memory directives | Choose one memory style |
121| `Provide a run command after --` | Missing launch command | Add `-- ./simulate ...` |
122
123## Limitations
124
125- Does not query cluster hardware or site policies; it can only validate internal consistency.
126- SLURM installations vary (GPU directives, QoS rules, partitions). Adjust directives for your site.
127
128## References
129
130- `references/slurm_directives.md` - Common `#SBATCH` directives and mapping tips
131
132## Version History
133
134- **v1.0.0** (2026-02-25): Initial SLURM job script generator
tools-only