Benchmark And MMS Planner
Goal
Design a verification and validation plan before trusting simulation results. The skill helps agents choose manufactured solutions, benchmark cases, refinement protocols, uncertainty checks, and pass/fail criteria.
Requirements
- Python 3.10+
- No external dependencies
- Works on Linux, macOS, and Windows
Inputs to Gather
| Input |
Description |
Example |
| PDE or model class |
Governing family |
diffusion, elasticity, phase-field |
| Quantity of interest |
Metric to validate |
interface velocity, L2 temperature error |
| Dimension |
1, 2, or 3 |
2 |
| Expected order |
Formal discretization order |
2 |
| Reference availability |
Analytic, benchmark, or none |
analytic |
| Risk level |
Cost or consequence of wrong result |
high |
Decision Guidance
- Use MMS when code correctness is uncertain and an analytic solution can be injected.
- Use canonical benchmarks when physical model validation matters more than code verification.
- Use grid/time refinement whenever the result is used for a claim, design decision, or comparison.
- Use uncertainty propagation when inputs are calibrated, noisy, or experimentally measured.
Script Outputs
scripts/benchmark_mms_planner.py emits inputs and results with:
verification_strategy
mms_plan
benchmark_cases
refinement_protocol
acceptance_criteria
warnings
Workflow
- Collect the governing model, quantity of interest, and risk level.
- Run
benchmark_mms_planner.py --json.
- Treat warnings as blockers for high-risk claims.
- Convert the returned protocol into tests, simulation runs, or review checklist items.
python3 skills/verification-validation/benchmark-and-mms-planner/scripts/benchmark_mms_planner.py \
--model diffusion \
--quantity "L2 error in temperature" \
--dimension 2 \
--expected-order 2 \
--reference analytic \
--risk high \
--json
Error Handling
- If the dimension or expected order is invalid, stop and correct the model description.
- If no reference exists, use conservation and convergence checks but do not call the result validated.
Limitations
This skill plans verification work; it does not run the solver or prove that a physical model is appropriate for an experiment.
Security
- Inputs are scalar strings and finite numeric values only.
- The script does not execute external solvers.
- File writes are not performed.
- The skill uses
Bash only to run its bundled script.
References
- See
references/vv_patterns.md for MMS, benchmark, and uncertainty planning notes.
Version History
- 1.0.0: Initial benchmark and MMS planning skill.
1---2name: benchmark-and-mms-planner3description: >4 Plan verification and validation campaigns for simulation codes using5 manufactured solutions, canonical benchmark problems, grid/time refinement,6 uncertainty propagation, and pass/fail acceptance criteria. Use when an7 agent needs to prove a solver, model, or result is trustworthy rather than8 only plausible.9allowed-tools: Read, Bash, Write, Grep, Glob10metadata:11 source: https://skillmd.com/skills/zuoguyoupan2023/benchmark-and-mms-planner12 author: HeshamFS13 version: "1.0.0"14 security_tier: high15 security_reviewed: true16 tested_with:17 - claude-code18 - gemini-cli19 - vs-code-copilot20 eval_cases: 321 last_reviewed: "2026-05-18"22---2324# Benchmark And MMS Planner2526## Goal2728Design a verification and validation plan before trusting simulation results. The skill helps agents choose manufactured solutions, benchmark cases, refinement protocols, uncertainty checks, and pass/fail criteria.2930## Requirements3132- Python 3.10+33- No external dependencies34- Works on Linux, macOS, and Windows3536## Inputs to Gather3738| Input | Description | Example |39|-------|-------------|---------|40| PDE or model class | Governing family | `diffusion`, `elasticity`, `phase-field` |41| Quantity of interest | Metric to validate | `interface velocity`, `L2 temperature error` |42| Dimension | 1, 2, or 3 | `2` |43| Expected order | Formal discretization order | `2` |44| Reference availability | Analytic, benchmark, or none | `analytic` |45| Risk level | Cost or consequence of wrong result | `high` |4647## Decision Guidance4849- Use **MMS** when code correctness is uncertain and an analytic solution can be injected.50- Use **canonical benchmarks** when physical model validation matters more than code verification.51- Use **grid/time refinement** whenever the result is used for a claim, design decision, or comparison.52- Use **uncertainty propagation** when inputs are calibrated, noisy, or experimentally measured.5354## Script Outputs5556`scripts/benchmark_mms_planner.py` emits `inputs` and `results` with:5758- `verification_strategy`59- `mms_plan`60- `benchmark_cases`61- `refinement_protocol`62- `acceptance_criteria`63- `warnings`6465## Workflow66671. Collect the governing model, quantity of interest, and risk level.682. Run `benchmark_mms_planner.py --json`.693. Treat warnings as blockers for high-risk claims.704. Convert the returned protocol into tests, simulation runs, or review checklist items.7172```bash73python3 skills/verification-validation/benchmark-and-mms-planner/scripts/benchmark_mms_planner.py \74 --model diffusion \75 --quantity "L2 error in temperature" \76 --dimension 2 \77 --expected-order 2 \78 --reference analytic \79 --risk high \80 --json81```8283## Error Handling8485- If the dimension or expected order is invalid, stop and correct the model description.86- If no reference exists, use conservation and convergence checks but do not call the result validated.8788## Limitations8990This skill plans verification work; it does not run the solver or prove that a physical model is appropriate for an experiment.9192## Security9394- Inputs are scalar strings and finite numeric values only.95- The script does not execute external solvers.96- File writes are not performed.97- The skill uses `Bash` only to run its bundled script.9899## References100101- See `references/vv_patterns.md` for MMS, benchmark, and uncertainty planning notes.102103## Version History104105- 1.0.0: Initial benchmark and MMS planning skill.