Parameter Optimization
Goal
Provide a workflow to design experiments, rank parameter influence, and select optimization strategies for materials simulation calibration.
Requirements
- Python 3.8+
- No external dependencies (uses Python standard library only)
Inputs to Gather
Before running any scripts, collect from the user:
| Input |
Description |
Example |
| Parameter bounds |
Min/max for each parameter with units |
kappa: [0.1, 10.0] W/mK |
| Evaluation budget |
Max number of simulations allowed |
50 runs |
| Noise level |
Stochasticity of simulation outputs |
low, medium, high |
| Constraints |
Feasibility rules or forbidden regions |
kappa + mobility < 5 |
Decision Guidance
Choosing a DOE Method
Is dimension <= 3 AND full coverage needed?
├── YES → Use factorial
└── NO → Is sensitivity analysis the goal?
├── YES → Use quasi-random (preferred; "sobol" is accepted but deprecated)
└── NO → Use lhs (Latin Hypercube)
| Method |
Best For |
Avoid When |
lhs |
General exploration, moderate dimensions (3-20) |
Need exact grid coverage |
sobol |
Sensitivity analysis, uniform coverage |
Very high dimensions (>20) |
factorial |
Low dimension (<4), need all corners |
High dimension (exponential growth) |
Choosing an Optimizer
Is dimension <= 5 AND budget <= 100?
├── YES → Bayesian Optimization
└── NO → Is dimension <= 20?
├── YES → CMA-ES
└── NO → Random Search with screening
| Noise Level |
Recommendation |
| Low |
Gradient-based if derivatives available, else Bayesian Optimization |
| Medium |
Bayesian Optimization with noise model |
| High |
Evolutionary algorithms or robust Bayesian Optimization |
Script Outputs (JSON Fields)
| Script |
Output Fields |
scripts/doe_generator.py |
samples, method, coverage |
scripts/optimizer_selector.py |
recommended, expected_evals, notes |
scripts/sensitivity_summary.py |
ranking, notes |
scripts/surrogate_builder.py |
model_type, metrics, notes |
Workflow
- Generate DOE with
scripts/doe_generator.py
- Run simulations at DOE sample points (user's responsibility)
- Summarize sensitivity with
scripts/sensitivity_summary.py
- Choose optimizer using
scripts/optimizer_selector.py
- (Optional) Fit surrogate with
scripts/surrogate_builder.py
CLI Examples
# Generate 20 LHS samples for 3 parameters
python3 scripts/doe_generator.py --params 3 --budget 20 --method lhs --json
# Rank parameters by sensitivity scores
python3 scripts/sensitivity_summary.py --scores 0.2,0.5,0.3 --names kappa,mobility,W --json
# Get optimizer recommendation for 3D problem with 50 eval budget
python3 scripts/optimizer_selector.py --dim 3 --budget 50 --noise low --json
# Build surrogate model from simulation data
python3 scripts/surrogate_builder.py --x 0,1,2 --y 10,12,15 --model rbf --json
Conversational Workflow Example
User: I need to calibrate thermal conductivity and diffusivity for my FEM simulation. I can run about 30 simulations.
Agent workflow:
- Identify 2 parameters →
--params 2
- Budget is 30 →
--budget 30
- Use LHS for general exploration:
python3 scripts/doe_generator.py --params 2 --budget 30 --method lhs --json
- After user runs simulations and provides outputs, summarize sensitivity:
python3 scripts/sensitivity_summary.py --scores 0.7,0.3 --names conductivity,diffusivity --json
- Recommend optimizer:
python3 scripts/optimizer_selector.py --dim 2 --budget 30 --noise low --json
Error Handling
| Error |
Cause |
Resolution |
params must be positive |
Zero or negative dimension |
Ask user for valid parameter count |
budget must be positive |
Zero or negative budget |
Ask user for realistic simulation budget |
method must be lhs, sobol, or factorial |
Invalid method |
Use decision guidance to pick valid method |
scores must be comma-separated |
Malformed input |
Reformat as 0.1,0.2,0.3 |
Limitations
- Not for real-time optimization: Scripts provide recommendations, not live optimization loops
- Surrogate is a placeholder:
surrogate_builder.py computes basic metrics; replace with actual model for production
- No automatic simulation execution: User must run simulations externally and provide results
References
references/doe_methods.md - Detailed DOE method comparisonreferences/optimizer_selection.md - Optimizer algorithm detailsreferences/sensitivity_guidelines.md - Sensitivity analysis interpretationreferences/surrogate_guidelines.md - Surrogate model selection
Version History
- v1.1.0 (2024-12-24): Enhanced documentation, decision guidance, conversational examples
- v1.0.0: Initial release with core scripts
1---2name: parameter-optimization3description: Explore and optimize simulation parameters via design of experiments (DOE), sensitivity analysis, and optimizer selection. Use for calibration, uncertainty studies, parameter sweeps, LHS sampling, Sobol analysis, surrogate modeling, or Bayesian optimization setup.4---5
6# Parameter Optimization
7
8## Goal
9
10Provide a workflow to design experiments, rank parameter influence, and select optimization strategies for materials simulation calibration.
11
12## Requirements
13
14- Python 3.8+
15- No external dependencies (uses Python standard library only)
16
17## Inputs to Gather
18
19Before running any scripts, collect from the user:
20
21| Input | Description | Example |
22|-------|-------------|---------|
23| Parameter bounds | Min/max for each parameter with units | `kappa: [0.1, 10.0] W/mK` |
24| Evaluation budget | Max number of simulations allowed | `50 runs` |
25| Noise level | Stochasticity of simulation outputs | `low`, `medium`, `high` |
26| Constraints | Feasibility rules or forbidden regions | `kappa + mobility < 5` |
27
28## Decision Guidance
29
30### Choosing a DOE Method
31
32```
33Is dimension <= 3 AND full coverage needed?
34├── YES → Use factorial
35└── NO → Is sensitivity analysis the goal?
36 ├── YES → Use quasi-random (preferred; "sobol" is accepted but deprecated)
37 └── NO → Use lhs (Latin Hypercube)
38```
39
40| Method | Best For | Avoid When |
41|--------|----------|------------|
42| `lhs` | General exploration, moderate dimensions (3-20) | Need exact grid coverage |
43| `sobol` | Sensitivity analysis, uniform coverage | Very high dimensions (>20) |
44| `factorial` | Low dimension (<4), need all corners | High dimension (exponential growth) |
45
46### Choosing an Optimizer
47
48```
49Is dimension <= 5 AND budget <= 100?
50├── YES → Bayesian Optimization
51└── NO → Is dimension <= 20?
52 ├── YES → CMA-ES
53 └── NO → Random Search with screening
54```
55
56| Noise Level | Recommendation |
57|-------------|----------------|
58| Low | Gradient-based if derivatives available, else Bayesian Optimization |
59| Medium | Bayesian Optimization with noise model |
60| High | Evolutionary algorithms or robust Bayesian Optimization |
61
62## Script Outputs (JSON Fields)
63
64| Script | Output Fields |
65|--------|---------------|
66| `scripts/doe_generator.py` | `samples`, `method`, `coverage` |
67| `scripts/optimizer_selector.py` | `recommended`, `expected_evals`, `notes` |
68| `scripts/sensitivity_summary.py` | `ranking`, `notes` |
69| `scripts/surrogate_builder.py` | `model_type`, `metrics`, `notes` |
70
71## Workflow
72
731. **Generate DOE** with `scripts/doe_generator.py`
742. **Run simulations** at DOE sample points (user's responsibility)
753. **Summarize sensitivity** with `scripts/sensitivity_summary.py`
764. **Choose optimizer** using `scripts/optimizer_selector.py`
775. **(Optional)** Fit surrogate with `scripts/surrogate_builder.py`
78
79## CLI Examples
80
81```bash
82# Generate 20 LHS samples for 3 parameters
83python3 scripts/doe_generator.py --params 3 --budget 20 --method lhs --json
84
85# Rank parameters by sensitivity scores
86python3 scripts/sensitivity_summary.py --scores 0.2,0.5,0.3 --names kappa,mobility,W --json
87
88# Get optimizer recommendation for 3D problem with 50 eval budget
89python3 scripts/optimizer_selector.py --dim 3 --budget 50 --noise low --json
90
91# Build surrogate model from simulation data
92python3 scripts/surrogate_builder.py --x 0,1,2 --y 10,12,15 --model rbf --json
93```
94
95## Conversational Workflow Example
96
97**User**: I need to calibrate thermal conductivity and diffusivity for my FEM simulation. I can run about 30 simulations.
98
99**Agent workflow**:
1001. Identify 2 parameters → `--params 2`
1012. Budget is 30 → `--budget 30`
1023. Use LHS for general exploration:
103 ```bash
104 python3 scripts/doe_generator.py --params 2 --budget 30 --method lhs --json
105 ```
1064. After user runs simulations and provides outputs, summarize sensitivity:
107 ```bash
108 python3 scripts/sensitivity_summary.py --scores 0.7,0.3 --names conductivity,diffusivity --json
109 ```
1105. Recommend optimizer:
111 ```bash
112 python3 scripts/optimizer_selector.py --dim 2 --budget 30 --noise low --json
113 ```
114
115## Error Handling
116
117| Error | Cause | Resolution |
118|-------|-------|------------|
119| `params must be positive` | Zero or negative dimension | Ask user for valid parameter count |
120| `budget must be positive` | Zero or negative budget | Ask user for realistic simulation budget |
121| `method must be lhs, sobol, or factorial` | Invalid method | Use decision guidance to pick valid method |
122| `scores must be comma-separated` | Malformed input | Reformat as `0.1,0.2,0.3` |
123
124## Limitations
125
126- **Not for real-time optimization**: Scripts provide recommendations, not live optimization loops
127- **Surrogate is a placeholder**: `surrogate_builder.py` computes basic metrics; replace with actual model for production
128- **No automatic simulation execution**: User must run simulations externally and provide results
129
130## References
131
132- `references/doe_methods.md` - Detailed DOE method comparison
133- `references/optimizer_selection.md` - Optimizer algorithm details
134- `references/sensitivity_guidelines.md` - Sensitivity analysis interpretation
135- `references/surrogate_guidelines.md` - Surrogate model selection
136
137## Version History
138
139- **v1.1.0** (2024-12-24): Enhanced documentation, decision guidance, conversational examples
140- **v1.0.0**: Initial release with core scripts
tools-only