Simulation Orchestrator
Goal
Provide tools to manage multi-simulation campaigns: generate parameter sweeps, track job execution status, and aggregate results from completed runs.
Requirements
- Python 3.10+
- No external dependencies (uses Python standard library only)
- Works on Linux, macOS, and Windows
Inputs to Gather
Before running orchestration scripts, collect from the user:
| Input |
Description |
Example |
| Base config |
Template simulation configuration |
base_config.json |
| Parameter ranges |
Parameters to sweep with bounds |
dt:[1e-4,1e-2],kappa:[0.1,1.0] |
| Sweep method |
How to sample parameter space |
grid, lhs, linspace |
| Output directory |
Where to store campaign files |
./campaign_001 |
| Simulation command |
Command to run each simulation |
python sim.py --config {config} |
Decision Guidance
Choosing a Sweep Method
Need every combination (full factorial)?
├── YES → Use grid (warning: exponential growth with parameters)
└── NO → Is space-filling coverage needed?
├── YES → Use lhs (Latin Hypercube Sampling)
└── NO → Use linspace for uniform sampling per parameter
| Method |
Best For |
Sample Count |
grid |
Low dimensions (1-3), need exact corners |
n^d (exponential) |
linspace |
1D sweeps, uniform spacing |
n per parameter |
lhs |
High dimensions, space-filling |
user-specified budget |
Campaign Size Guidelines
| Parameters |
Grid Points Each |
Total Runs |
Recommendation |
| 1 |
10 |
10 |
Grid is fine |
| 2 |
10 |
100 |
Grid acceptable |
| 3 |
10 |
1,000 |
Consider LHS |
| 4+ |
10 |
10,000+ |
Use LHS or DOE |
Script Outputs (JSON Fields)
| Script |
Output Fields |
scripts/sweep_generator.py |
configs, parameter_space, sweep_method, total_runs |
scripts/campaign_manager.py |
campaign_id, status, jobs, progress |
scripts/job_tracker.py |
job_id, status, start_time, end_time, exit_code |
scripts/result_aggregator.py |
summary, statistics, best_run, failed_runs |
Workflow
Step 1: Generate Parameter Sweep
Create configurations for all parameter combinations:
python3 scripts/sweep_generator.py \
--base-config base_config.json \
--params "dt:1e-4:1e-2:5,kappa:0.1:1.0:3" \
--method linspace \
--output-dir ./campaign_001 \
--json
Step 2: Initialize Campaign
Create campaign tracking structure:
python3 scripts/campaign_manager.py \
--action init \
--config-dir ./campaign_001 \
--command "python sim.py --config {config}" \
--json
Step 3: Track Job Status
Monitor running jobs:
python3 scripts/job_tracker.py \
--campaign-dir ./campaign_001 \
--update \
--json
Step 4: Aggregate Results
Combine results from completed runs:
python3 scripts/result_aggregator.py \
--campaign-dir ./campaign_001 \
--metric objective_value \
--json
CLI Examples
# Generate 5x3=15 runs varying dt (5 values) and kappa (3 values)
python3 scripts/sweep_generator.py \
--base-config sim.json \
--params "dt:1e-4:1e-2:5,kappa:0.1:1.0:3" \
--method linspace \
--output-dir ./sweep_001 \
--json
# Generate LHS samples for 4 parameters with budget of 20 runs
python3 scripts/sweep_generator.py \
--base-config sim.json \
--params "dt:1e-4:1e-2,kappa:0.1:1.0,M:1e-6:1e-4,W:0.5:2.0" \
--method lhs \
--samples 20 \
--output-dir ./lhs_001 \
--json
# Check campaign status
python3 scripts/campaign_manager.py \
--action status \
--config-dir ./sweep_001 \
--json
# Get summary statistics from completed runs
python3 scripts/result_aggregator.py \
--campaign-dir ./sweep_001 \
--metric final_energy \
--json
Conversational Workflow Example
User: I want to run a parameter sweep on dt and kappa for my phase-field simulation. I want to try 5 values of dt between 1e-4 and 1e-2, and 4 values of kappa between 0.1 and 1.0.
Agent workflow:
- Calculate total runs: 5 x 4 = 20 runs
- Generate sweep configurations:
python3 scripts/sweep_generator.py \
--base-config simulation.json \
--params "dt:1e-4:1e-2:5,kappa:0.1:1.0:4" \
--method linspace \
--output-dir ./dt_kappa_sweep \
--json
- Initialize campaign:
python3 scripts/campaign_manager.py \
--action init \
--config-dir ./dt_kappa_sweep \
--command "python phase_field.py --config {config}" \
--json
- After user runs simulations, aggregate results:
python3 scripts/result_aggregator.py \
--campaign-dir ./dt_kappa_sweep \
--metric interface_width \
--json
Error Handling
| Error |
Cause |
Resolution |
Base config not found |
Invalid file path |
Verify base config file exists |
Invalid parameter format |
Malformed param string |
Use format name:min:max:count or name:min:max |
Output directory exists |
Would overwrite |
Use --force or choose new directory |
No completed jobs |
No results to aggregate |
Wait for jobs to complete or check for failures |
Metric not found |
Result files missing field |
Verify metric name in result JSON |
Integration with Other Skills
The simulation-orchestrator works with other simulation-workflow skills:
parameter-optimization simulation-orchestrator
│ │
│ DOE samples ────────────────>│ Generate configs
│ │
│ │ Run simulations
│ │
│<──────────────────────────── │ Aggregate results
│ │
│ Sensitivity analysis │
│ Optimizer selection │
Typical Combined Workflow
- Use
parameter-optimization/doe_generator.py to get sample points
- Use
simulation-orchestrator/sweep_generator.py to create configs
- Run simulations (user's responsibility)
- Use
simulation-orchestrator/result_aggregator.py to collect results
- Use
parameter-optimization/sensitivity_summary.py to analyze
Security
Input Validation
- Metric names are validated against
[a-zA-Z_][a-zA-Z0-9_.]* to prevent traversal or injection via crafted keys
campaign_manager.py validates command templates to reject shell chaining operators (;, |, &, backticks, $)
--params format strings are parsed and validated (name:min:max:count with finite numeric bounds and positive integer counts)
--method is validated against a fixed allowlist (grid, linspace, lhs)
--samples is validated as a positive integer with an upper bound
--action is validated against a fixed allowlist (init, status)
File Access
sweep_generator.py reads a single base config file (JSON) specified by --base-config and writes generated configs to --output-dir
result_aggregator.py enforces a 10 MB file-size limit per result file, maximum JSON nesting depth, and strict numeric type checking (rejects bool, NaN, Inf)
- All string values from result files are sanitized (truncated, control characters stripped) before surfacing them
- Config paths interpolated into shell commands are validated against a safe-character allowlist and escaped with
shlex.quote()
Tool Restrictions
- Read: Used to inspect script source, references, base configs, and campaign status files
- Write: Used to save generated sweep configs, campaign manifests, and aggregated results; writes are scoped to the user's working directory
- Grep/Glob: Used to locate campaign files, result files, and search references
- The skill's
allowed-tools excludes Bash to prevent the agent from executing arbitrary commands when processing untrusted simulation outputs
Safety Measures
- No
eval(), exec(), or dynamic code generation
- All subprocess calls use explicit argument lists (no
shell=True)
- Reduced tool surface (no Bash) limits the agent to read/write operations only
- Command templates are validated but never executed by the skill itself; execution is the user's responsibility
Limitations
- Not a job scheduler: Does not submit jobs to SLURM/PBS; generates configs and tracks status
- No parallel execution: User must run simulations externally (can use GNU parallel, SLURM, etc.)
- File-based tracking: Status tracked via files; no database or real-time monitoring
- Local filesystem: Assumes all files accessible from local machine
References
references/campaign_patterns.md - Common campaign structures
references/sweep_strategies.md - Parameter sweep design guidance
references/aggregation_methods.md - Result aggregation techniques
Version History
- v1.0.0 (2024-12-24): Initial release with sweep, campaign, tracking, and aggregation
1---2name: simulation-orchestrator3description: >4 Orchestrate multi-simulation campaigns — generate parameter sweep5 configurations (grid, linspace, or Latin Hypercube sampling), initialize6 and track batch job campaigns, monitor job completion status, and aggregate7 results with summary statistics across all runs. Use when running a8 parameter study across dt, kappa, or other simulation inputs, managing9 dozens or hundreds of simulation configurations, combining outputs from10 completed batch runs to find the best result, or automating the11 generate-run-collect workflow for systematic studies, even if the user12 only says "I need to try many parameter combinations" or "how do I13 organize a sweep."14allowed-tools: Read, Write, Grep, Glob15metadata:16 source: https://skillmd.com/skills/zuoguyoupan2023/simulation-orchestrator17 author: HeshamFS18 version: "1.1.0"19 security_tier: medium20 security_reviewed: true21 tested_with:22 - claude-code23 - gemini-cli24 - vs-code-copilot25 eval_cases: 526 last_reviewed: "2026-03-26"27---2829# Simulation Orchestrator3031## Goal3233Provide tools to manage multi-simulation campaigns: generate parameter sweeps, track job execution status, and aggregate results from completed runs.3435## Requirements3637- Python 3.10+38- No external dependencies (uses Python standard library only)39- Works on Linux, macOS, and Windows4041## Inputs to Gather4243Before running orchestration scripts, collect from the user:4445| Input | Description | Example |46|-------|-------------|---------|47| Base config | Template simulation configuration | `base_config.json` |48| Parameter ranges | Parameters to sweep with bounds | `dt:[1e-4,1e-2],kappa:[0.1,1.0]` |49| Sweep method | How to sample parameter space | `grid`, `lhs`, `linspace` |50| Output directory | Where to store campaign files | `./campaign_001` |51| Simulation command | Command to run each simulation | `python sim.py --config {config}` |5253## Decision Guidance5455### Choosing a Sweep Method5657```58Need every combination (full factorial)?59├── YES → Use grid (warning: exponential growth with parameters)60└── NO → Is space-filling coverage needed?61 ├── YES → Use lhs (Latin Hypercube Sampling)62 └── NO → Use linspace for uniform sampling per parameter63```6465| Method | Best For | Sample Count |66|--------|----------|--------------|67| `grid` | Low dimensions (1-3), need exact corners | n^d (exponential) |68| `linspace` | 1D sweeps, uniform spacing | n per parameter |69| `lhs` | High dimensions, space-filling | user-specified budget |7071### Campaign Size Guidelines7273| Parameters | Grid Points Each | Total Runs | Recommendation |74|------------|------------------|------------|----------------|75| 1 | 10 | 10 | Grid is fine |76| 2 | 10 | 100 | Grid acceptable |77| 3 | 10 | 1,000 | Consider LHS |78| 4+ | 10 | 10,000+ | Use LHS or DOE |7980## Script Outputs (JSON Fields)8182| Script | Output Fields |83|--------|---------------|84| `scripts/sweep_generator.py` | `configs`, `parameter_space`, `sweep_method`, `total_runs` |85| `scripts/campaign_manager.py` | `campaign_id`, `status`, `jobs`, `progress` |86| `scripts/job_tracker.py` | `job_id`, `status`, `start_time`, `end_time`, `exit_code` |87| `scripts/result_aggregator.py` | `summary`, `statistics`, `best_run`, `failed_runs` |8889## Workflow9091### Step 1: Generate Parameter Sweep9293Create configurations for all parameter combinations:9495```bash96python3 scripts/sweep_generator.py \97 --base-config base_config.json \98 --params "dt:1e-4:1e-2:5,kappa:0.1:1.0:3" \99 --method linspace \100 --output-dir ./campaign_001 \101 --json102```103104### Step 2: Initialize Campaign105106Create campaign tracking structure:107108```bash109python3 scripts/campaign_manager.py \110 --action init \111 --config-dir ./campaign_001 \112 --command "python sim.py --config {config}" \113 --json114```115116### Step 3: Track Job Status117118Monitor running jobs:119120```bash121python3 scripts/job_tracker.py \122 --campaign-dir ./campaign_001 \123 --update \124 --json125```126127### Step 4: Aggregate Results128129Combine results from completed runs:130131```bash132python3 scripts/result_aggregator.py \133 --campaign-dir ./campaign_001 \134 --metric objective_value \135 --json136```137138## CLI Examples139140```bash141# Generate 5x3=15 runs varying dt (5 values) and kappa (3 values)142python3 scripts/sweep_generator.py \143 --base-config sim.json \144 --params "dt:1e-4:1e-2:5,kappa:0.1:1.0:3" \145 --method linspace \146 --output-dir ./sweep_001 \147 --json148149# Generate LHS samples for 4 parameters with budget of 20 runs150python3 scripts/sweep_generator.py \151 --base-config sim.json \152 --params "dt:1e-4:1e-2,kappa:0.1:1.0,M:1e-6:1e-4,W:0.5:2.0" \153 --method lhs \154 --samples 20 \155 --output-dir ./lhs_001 \156 --json157158# Check campaign status159python3 scripts/campaign_manager.py \160 --action status \161 --config-dir ./sweep_001 \162 --json163164# Get summary statistics from completed runs165python3 scripts/result_aggregator.py \166 --campaign-dir ./sweep_001 \167 --metric final_energy \168 --json169```170171## Conversational Workflow Example172173**User**: I want to run a parameter sweep on dt and kappa for my phase-field simulation. I want to try 5 values of dt between 1e-4 and 1e-2, and 4 values of kappa between 0.1 and 1.0.174175**Agent workflow**:1761. Calculate total runs: 5 x 4 = 20 runs1772. Generate sweep configurations:178 ```bash179 python3 scripts/sweep_generator.py \180 --base-config simulation.json \181 --params "dt:1e-4:1e-2:5,kappa:0.1:1.0:4" \182 --method linspace \183 --output-dir ./dt_kappa_sweep \184 --json185 ```1863. Initialize campaign:187 ```bash188 python3 scripts/campaign_manager.py \189 --action init \190 --config-dir ./dt_kappa_sweep \191 --command "python phase_field.py --config {config}" \192 --json193 ```1944. After user runs simulations, aggregate results:195 ```bash196 python3 scripts/result_aggregator.py \197 --campaign-dir ./dt_kappa_sweep \198 --metric interface_width \199 --json200 ```201202## Error Handling203204| Error | Cause | Resolution |205|-------|-------|------------|206| `Base config not found` | Invalid file path | Verify base config file exists |207| `Invalid parameter format` | Malformed param string | Use format `name:min:max:count` or `name:min:max` |208| `Output directory exists` | Would overwrite | Use `--force` or choose new directory |209| `No completed jobs` | No results to aggregate | Wait for jobs to complete or check for failures |210| `Metric not found` | Result files missing field | Verify metric name in result JSON |211212## Integration with Other Skills213214The simulation-orchestrator works with other simulation-workflow skills:215216```217parameter-optimization simulation-orchestrator218 │ │219 │ DOE samples ────────────────>│ Generate configs220 │ │221 │ │ Run simulations222 │ │223 │<──────────────────────────── │ Aggregate results224 │ │225 │ Sensitivity analysis │226 │ Optimizer selection │227```228229### Typical Combined Workflow2302311. Use `parameter-optimization/doe_generator.py` to get sample points2322. Use `simulation-orchestrator/sweep_generator.py` to create configs2333. Run simulations (user's responsibility)2344. Use `simulation-orchestrator/result_aggregator.py` to collect results2355. Use `parameter-optimization/sensitivity_summary.py` to analyze236237## Security238239### Input Validation240- Metric names are validated against `[a-zA-Z_][a-zA-Z0-9_.]*` to prevent traversal or injection via crafted keys241- `campaign_manager.py` validates command templates to reject shell chaining operators (`;`, `|`, `&`, backticks, `$`)242- `--params` format strings are parsed and validated (`name:min:max:count` with finite numeric bounds and positive integer counts)243- `--method` is validated against a fixed allowlist (`grid`, `linspace`, `lhs`)244- `--samples` is validated as a positive integer with an upper bound245- `--action` is validated against a fixed allowlist (`init`, `status`)246247### File Access248- `sweep_generator.py` reads a single base config file (JSON) specified by `--base-config` and writes generated configs to `--output-dir`249- `result_aggregator.py` enforces a 10 MB file-size limit per result file, maximum JSON nesting depth, and strict numeric type checking (rejects `bool`, `NaN`, `Inf`)250- All string values from result files are sanitized (truncated, control characters stripped) before surfacing them251- Config paths interpolated into shell commands are validated against a safe-character allowlist and escaped with `shlex.quote()`252253### Tool Restrictions254- **Read**: Used to inspect script source, references, base configs, and campaign status files255- **Write**: Used to save generated sweep configs, campaign manifests, and aggregated results; writes are scoped to the user's working directory256- **Grep/Glob**: Used to locate campaign files, result files, and search references257- The skill's `allowed-tools` excludes `Bash` to prevent the agent from executing arbitrary commands when processing untrusted simulation outputs258259### Safety Measures260- No `eval()`, `exec()`, or dynamic code generation261- All subprocess calls use explicit argument lists (no `shell=True`)262- Reduced tool surface (no Bash) limits the agent to read/write operations only263- Command templates are validated but never executed by the skill itself; execution is the user's responsibility264265## Limitations266267- **Not a job scheduler**: Does not submit jobs to SLURM/PBS; generates configs and tracks status268- **No parallel execution**: User must run simulations externally (can use GNU parallel, SLURM, etc.)269- **File-based tracking**: Status tracked via files; no database or real-time monitoring270- **Local filesystem**: Assumes all files accessible from local machine271272## References273274- `references/campaign_patterns.md` - Common campaign structures275- `references/sweep_strategies.md` - Parameter sweep design guidance276- `references/aggregation_methods.md` - Result aggregation techniques277278## Version History279280- **v1.0.0** (2024-12-24): Initial release with sweep, campaign, tracking, and aggregation