1---2name: parameter-optimization3description: >4 Explore and optimize simulation parameters via design of experiments (DOE),5 sensitivity analysis, and optimizer selection — generate Latin Hypercube,6 quasi-random, or factorial sample plans, rank parameter influence with7 sensitivity scores, recommend Bayesian optimization, CMA-ES, or gradient-8 based methods based on dimension and budget, and fit surrogate models for9 expensive evaluations. Use when calibrating material properties against10 experimental data, planning a parameter sweep, performing uncertainty11 quantification, or choosing an optimization strategy for a simulation12 with a limited evaluation budget, even if the user only says "which13 parameters matter most" or "how do I calibrate my model."14allowed-tools: Read, Write, Grep, Glob15metadata:16 source: https://skillmd.com/skills/heshamfs/parameter-optimization17 author: HeshamFS18 version: "1.2.2"19 security_tier: medium20 security_reviewed: true21 tested_with:22 - claude-code23 last_evaluated: "2026-06-24"24 eval_cases: 525 last_reviewed: "2026-06-23"26 standards:27 - "Latin Hypercube Sampling (McKay, Beckman & Conover 1979)"28 - "Sobol (1967) low-discrepancy quasi-random sequences"29 - "Morris (1991) Elementary Effects screening method"30 - "Saltelli et al. (2008), Global Sensitivity Analysis: The Primer (Sobol indices / Saltelli estimator)"31 - "CMA-ES (Hansen & Ostermeier 2001)"32---3334# Parameter Optimization3536## Goal3738Provide a workflow to design experiments, rank parameter influence, and select optimization strategies for materials simulation calibration.3940## Requirements4142- Python 3.10+43- No external dependencies (uses Python standard library only)4445## Inputs to Gather4647Before running any scripts, collect from the user:4849| Input | Description | Example |50|-------|-------------|---------|51| Parameter bounds | Min/max for each parameter with units | `kappa: [0.1, 10.0] W/mK` |52| Evaluation budget | Max number of simulations allowed | `50 runs` |53| Noise level | Stochasticity of simulation outputs | `low`, `medium`, `high` |54| Constraints | Feasibility rules or forbidden regions | `kappa + mobility < 5` |5556## Decision Guidance5758### Choosing a DOE Method5960```61Is dimension <= 3 AND full coverage needed?62├── YES → Use factorial63└── NO → Is sensitivity analysis the goal?64 ├── YES → Use quasi-random (preferred; "sobol" is accepted but deprecated)65 └── NO → Use lhs (Latin Hypercube)66```6768| Method | Best For | Avoid When |69|--------|----------|------------|70| `lhs` | General exploration, moderate dimensions (3-20) | Need exact grid coverage |71| `quasi-random` | Sensitivity analysis, uniform coverage (preferred) | Very high dimensions (>20) |72| `sobol` | Deprecated alias of `quasi-random` (emits a warning) | New code (use `quasi-random`) |73| `factorial` | Low dimension (<4), need all corners | High dimension (exponential growth) |7475> **Factorial sizing:** the factorial grid is `levels` evenly spaced values per76> parameter, producing exactly `levels ** params` samples. Set the resolution77> explicitly with `--levels` (e.g. `--params 2 --levels 4` -> 16 samples). If you78> use `--budget` instead, the script back-computes `levels = round(budget ** (1/params))`79> and **warns** whenever the realized sample count differs from the requested80> budget (e.g. `--budget 20 --params 2` realizes 16 samples). For an exact design,81> pass a perfect power (`--budget 16`) or, preferably, `--levels`.8283### Choosing an Optimizer8485```86Is dimension <= 10 AND budget <= 100?87├── YES → Bayesian Optimization88└── NO → Is dimension <= 20?89 ├── YES → CMA-ES90 └── NO → Random Search with screening91```9293| Noise Level | Recommendation |94|-------------|----------------|95| Low | Gradient-based if derivatives available, else Bayesian Optimization |96| Medium | Bayesian Optimization with noise model |97| High | Evolutionary algorithms or robust Bayesian Optimization |9899## Script Outputs (JSON Fields)100101| Script | Output Fields |102|--------|---------------|103| `scripts/doe_generator.py` | `samples`, `method`, `coverage` (`count`, `dimension`; plus `levels` and a top-level `requested_budget`/`note` for factorial) |104| `scripts/optimizer_selector.py` | `recommended`, `expected_evals`, `notes` |105| `scripts/sensitivity_summary.py` | `ranking`, `notes` |106| `scripts/surrogate_builder.py` | `model_type`, `metrics` (`mse`, `cv_error`, `output_variance`), `notes` |107108## Workflow1091101. **Generate DOE** with `scripts/doe_generator.py`1112. **Run simulations** at DOE sample points (user's responsibility)1123. **Summarize sensitivity** with `scripts/sensitivity_summary.py`1134. **Choose optimizer** using `scripts/optimizer_selector.py`1145. **(Optional)** Fit surrogate with `scripts/surrogate_builder.py`115116## CLI Examples117118```bash119# Generate 20 LHS samples for 3 parameters120python3 scripts/doe_generator.py --params 3 --budget 20 --method lhs --json121122# Full factorial with 4 levels per parameter (2 params -> 16 samples)123python3 scripts/doe_generator.py --params 2 --levels 4 --method factorial --json124125# Rank parameters by sensitivity scores126python3 scripts/sensitivity_summary.py --scores 0.2,0.5,0.3 --names kappa,mobility,W --json127128# Get optimizer recommendation for 3D problem with 50 eval budget129python3 scripts/optimizer_selector.py --dim 3 --budget 50 --noise low --json130131# Build surrogate model from simulation data132python3 scripts/surrogate_builder.py --x 0,1,2 --y 10,12,15 --model rbf --json133```134135## Conversational Workflow Example136137**User**: I need to calibrate thermal conductivity and diffusivity for my FEM simulation. I can run about 30 simulations.138139**Agent workflow**:1401. Identify 2 parameters → `--params 2`1412. Budget is 30 → `--budget 30`1423. Use LHS for general exploration:143 ```bash144 python3 scripts/doe_generator.py --params 2 --budget 30 --method lhs --json145 ```1464. After user runs simulations and provides outputs, summarize sensitivity:147 ```bash148 python3 scripts/sensitivity_summary.py --scores 0.7,0.3 --names conductivity,diffusivity --json149 ```1505. Recommend optimizer:151 ```bash152 python3 scripts/optimizer_selector.py --dim 2 --budget 30 --noise low --json153 ```154155## Error Handling156157| Error | Cause | Resolution |158|-------|-------|------------|159| `params must be positive` | Zero or negative dimension | Ask user for valid parameter count |160| `budget must be positive` | Zero or negative budget | Ask user for realistic simulation budget |161| `argument --method: invalid choice: <value> (choose from lhs, sobol, quasi-random, factorial)` | Invalid method (argparse) | Use decision guidance to pick a valid method |162| `could not convert string to float: <token>` | Non-numeric value in `--scores`/`--x`/`--y` | Reformat as `0.1,0.2,0.3` |163| `scores must be a comma-separated list` | Empty `--scores` input | Provide at least one numeric score |164165## Verification checklist166167- [ ] Recorded the exact `doe_generator.py` `coverage.count` and confirmed it matches the intended design — for `factorial`, verified `count == levels ** params` and that no `note`/`requested_budget` mismatch warning was emitted (or that the realized count is acceptable).168- [ ] Confirmed the chosen `--method` matches the Decision Guidance for the actual dimension/goal, and that `quasi-random` was used instead of the deprecated `sobol` alias (no `DeprecationWarning` in output).169- [ ] Recorded the `optimizer_selector.py` `recommended` strategy and `expected_evals`, and verified `expected_evals <= budget` so the plan is feasible within the stated evaluation budget.170- [ ] Logged the `sensitivity_summary.py` `ranking` and checked whether the top sensitivity is `< 0.1` (the "All sensitivities are low" note); if so, did not over-interpret the ranking and revisited the output metric.171- [ ] For surrogate fits, judged quality with `metrics.cv_error` (leave-one-out), NOT in-sample `mse` — especially for `rbf`, where `mse` is near zero by construction — and compared `cv_error` against `metrics.output_variance` to confirm the surrogate beats the constant-mean baseline.172- [ ] Confirmed any reported `cv_error` is a finite number (not `NaN`), i.e. there were enough samples for leave-one-out (`poly`: `n > degree+1`; `rbf`: `n >= 3`).173174## Common pitfalls & rationalizations175176| Tempting shortcut | Why it's wrong / what to do |177|-------------------|------------------------------|178| "RBF surrogate `mse` is ~0, so the model is excellent." | RBF is an exact interpolant — in-sample `mse` is near zero by construction and says nothing about generalization. Judge fit with `metrics.cv_error` and compare it to `output_variance`. |179| "I asked for `--budget 20` factorial, so I got 20 samples." | Factorial honors `levels ** params`, not the budget; `--budget 20 --params 2` realizes 16 samples and emits a `note`/warning. Use `--levels` for an exact, intended design. |180| "`sobol` gives me a true Sobol low-discrepancy sequence." | `sobol` is a deprecated alias that emits a `DeprecationWarning` and uses a simplified golden-ratio additive recurrence, not a true Sobol sequence. Use `quasi-random`; for production Sobol use `scipy.stats.qmc`. |181| "The optimizer recommendation is just advice — budget doesn't matter." | The recommendation is gated on dimension AND budget (BO only for `dim<=10 AND budget<=100`), and `expected_evals` is capped at the budget. Record both and confirm the plan fits the real budget. |182| "One sensitivity score is highest, so that parameter dominates." | The script only sorts the scores you pass in; it computes no sensitivity itself. If the top score is `< 0.1` it flags that all sensitivities are low — get the scores from a real screening/Sobol analysis before trusting the ranking. |183| "It printed JSON without erroring, so the result is valid." | Exit success only means inputs parsed. Verify the design size, `expected_evals <= budget`, a finite `cv_error`, and that the surrogate beats `output_variance` before trusting any output. |184185## Security186187### Input Validation188- `sensitivity_summary.py` validates `--names` against `[a-zA-Z_][a-zA-Z0-9_ .-]*` with a 200-char limit, preventing shell metacharacter injection via crafted parameter names189- All numeric list inputs are validated as finite numbers (`NaN`/`Inf` rejected)190- Comma-separated value lists are capped (10,000 for scores, 100,000 for surrogate data) to prevent resource exhaustion191- `doe_generator.py` caps dimension at 1,000 and budget at 1,000,000; `optimizer_selector.py` caps dimension at 100,000 and budget at 10,000,000192- `--method` is validated against a fixed allowlist (`lhs`, `quasi-random`/`sobol`, `factorial`); `sobol` is an accepted but deprecated alias of `quasi-random`193- `--noise` is validated against a fixed allowlist (`low`, `medium`, `high`)194- `--model` (surrogate type) is validated against a fixed allowlist (`rbf`, `poly`)195- `--levels` (factorial grid resolution) is validated as an integer in `[2, 1000]`196197### File Access198- Scripts read no external files; all inputs are provided via CLI arguments199- Scripts write only to stdout (JSON output); no files are created unless the agent explicitly uses the Write tool200201### Tool Restrictions202- **Read**: Used to inspect script source, references, and user data files203- **Write**: Used to save DOE sample plans, sensitivity rankings, or optimizer recommendations; writes are scoped to the user's working directory204- **Grep/Glob**: Used to locate relevant files and search references205- The skill's `allowed-tools` excludes `Bash` to prevent the agent from executing arbitrary commands when processing user-provided parameter names and constraints206207### Safety Measures208- No `eval()`, `exec()`, or dynamic code generation209- All subprocess calls use explicit argument lists (no `shell=True`)210- Reduced tool surface (no Bash) limits the agent to read/write operations only211- Parameter names are sanitized before use, preventing injection via crafted identifiers212213## Limitations214215- **Not for real-time optimization**: Scripts provide recommendations, not live optimization loops216- **Surrogate is lightweight**: `surrogate_builder.py` fits a real 1-D least-squares polynomial (`poly`) or Gaussian RBF interpolant (`rbf`) using only the standard library and reports honest residual `mse`, leave-one-out `cv_error`, and the data `output_variance`; for production use scipy/scikit-learn/GPyTorch. For `rbf`, in-sample `mse` is near zero by construction (exact interpolation) — judge fit quality with `cv_error`217- **No automatic simulation execution**: User must run simulations externally and provide results218219## References220221- `references/doe_methods.md` - Detailed DOE method comparison222- `references/optimizer_selection.md` - Optimizer algorithm details223- `references/sensitivity_guidelines.md` - Sensitivity analysis interpretation224- `references/surrogate_guidelines.md` - Surrogate model selection225226## Version History227228- **v1.2.2** (2026-06-24): Added Verification checklist and Common pitfalls & rationalizations sections to drive evidence-based use of the DOE, optimizer, sensitivity, and surrogate scripts229- **v1.2.0** (2026-06-23): Real surrogate fits (`poly` least-squares, `rbf` interpolation) with honest `mse`/`cv_error`/`output_variance`; explicit factorial `--levels` with budget-mismatch warnings; BO dimension cutoff harmonized to dim<=10; corrected Security/Error-Handling/output-field docs to match script behavior230- **v1.1.0** (2024-12-24): Enhanced documentation, decision guidance, conversational examples231- **v1.0.0**: Initial release with core scripts