Performance Profiling
Goal
Provide tools to analyze simulation performance, identify bottlenecks, and recommend optimization strategies for computational materials science simulations.
Requirements
- Python 3.10+
- No external dependencies (uses Python standard library only)
- Works on Linux, macOS, and Windows
Inputs to Gather
Before running profiling scripts, collect from the user:
| Input |
Description |
Example |
| Simulation log |
Log file with timing information |
simulation.log |
| Scaling data |
JSON with multi-run performance data |
scaling_data.json |
| Simulation parameters |
JSON with mesh, fields, solver config |
params.json |
| Available memory |
System memory in GB (optional) |
16.0 |
Decision Guidance
When to Use Each Script
Need to identify slow phases?
├── YES → Use timing_analyzer.py
│ └── Parse simulation logs for timing data
│
Need to understand parallel performance?
├── YES → Use scaling_analyzer.py
│ └── Analyze strong or weak scaling efficiency
│
Need to estimate memory requirements?
├── YES → Use memory_profiler.py
│ └── Estimate memory from problem parameters
│
Need optimization recommendations?
└── YES → Use bottleneck_detector.py
└── Combine analyses and get actionable advice
Choosing Analysis Thresholds
| Metric |
Good |
Acceptable |
Poor |
| Phase dominance |
<30% |
30-50% |
>50% |
| Parallel efficiency |
>0.80 |
0.70-0.80 |
<0.70 |
| Memory usage |
<60% |
60-80% |
>80% |
Script Outputs (JSON Fields)
| Script |
Key Outputs |
timing_analyzer.py |
timing_data.phases, timing_data.slowest_phase, timing_data.total_time |
scaling_analyzer.py |
scaling_analysis.results, scaling_analysis.efficiency_threshold_processors |
memory_profiler.py |
memory_profile.total_memory_gb, memory_profile.per_process_gb, memory_profile.warnings |
bottleneck_detector.py |
bottlenecks, recommendations |
Workflow
Complete Profiling Workflow
- Analyze timing from simulation logs
- Analyze scaling from multi-run data (if available)
- Profile memory from simulation parameters
- Detect bottlenecks and get recommendations
- Implement optimizations based on recommendations
- Re-profile to verify improvements
Quick Profiling (Timing Only)
- Run timing analyzer on simulation log
- Identify dominant phases (>50% of runtime)
- Apply targeted optimizations to dominant phases
CLI Examples
Timing Analysis
# Basic timing analysis
python3 scripts/timing_analyzer.py \
--log simulation.log \
--json
# Custom timing pattern
python3 scripts/timing_analyzer.py \
--log simulation.log \
--pattern 'Step\s+(\w+)\s+took\s+([\d.]+)s' \
--json
Scaling Analysis
# Strong scaling (fixed problem size)
python3 scripts/scaling_analyzer.py \
--data scaling_data.json \
--type strong \
--json
# Weak scaling (constant work per processor)
python3 scripts/scaling_analyzer.py \
--data scaling_data.json \
--type weak \
--json
Memory Profiling
# Estimate memory requirements
python3 scripts/memory_profiler.py \
--params simulation_params.json \
--available-gb 16.0 \
--json
Bottleneck Detection
# Detect bottlenecks from timing only
python3 scripts/bottleneck_detector.py \
--timing timing_results.json \
--json
# Comprehensive analysis with all inputs
python3 scripts/bottleneck_detector.py \
--timing timing_results.json \
--scaling scaling_results.json \
--memory memory_results.json \
--json
Conversational Workflow Example
User: My simulation is taking too long. Can you help me identify what's slow?
Agent workflow:
- Ask for simulation log file
- Run timing analyzer:
python3 scripts/timing_analyzer.py --log simulation.log --json
- Interpret results:
- If solver dominates (>50%): Recommend preconditioner tuning
- If assembly dominates: Recommend caching or vectorization
- If I/O dominates: Recommend reducing output frequency
- If user has multi-run data, analyze scaling:
python3 scripts/scaling_analyzer.py --data scaling.json --type strong --json
- Generate comprehensive recommendations:
python3 scripts/bottleneck_detector.py --timing timing.json --scaling scaling.json --json
Interpretation Guidance
Timing Analysis
| Scenario |
Meaning |
Action |
| Solver >70% |
Solver-dominated |
Tune preconditioner, check tolerance |
| Assembly >50% |
Assembly-dominated |
Cache matrices, vectorize, parallelize |
| I/O >30% |
I/O-dominated |
Reduce frequency, use parallel I/O |
| Balanced (<30% each) |
Well-balanced |
Look for algorithmic improvements |
Scaling Analysis
| Efficiency |
Meaning |
Action |
| >0.80 |
Excellent scaling |
Continue scaling up |
| 0.70-0.80 |
Good scaling |
Monitor at larger scales |
| 0.50-0.70 |
Poor scaling |
Investigate communication/load balance |
| <0.50 |
Very poor scaling |
Reduce processor count or redesign |
Memory Profile
| Usage |
Meaning |
Action |
| <60% available |
Safe |
No action needed |
| 60-80% available |
Moderate |
Monitor, consider optimization |
| >80% available |
High |
Reduce resolution or increase processors |
| >100% available |
Exceeds capacity |
Must reduce problem size |
Error Handling
| Error |
Cause |
Resolution |
Log file not found |
Invalid path |
Verify log file path |
No timing data found |
Pattern mismatch |
Provide custom pattern with --pattern |
At least 2 runs required |
Insufficient data |
Provide more scaling runs |
Missing required parameters |
Incomplete params |
Add mesh and fields to params file |
Optimization Strategies by Bottleneck Type
Solver Bottlenecks
- Use algebraic multigrid (AMG) preconditioner
- Tighten solver tolerance if over-solving
- Consider direct solver for small problems
- Profile matrix assembly vs solve time
Assembly Bottlenecks
- Cache element matrices if geometry is static
- Use vectorized assembly routines
- Consider matrix-free methods
- Parallelize assembly with coloring
I/O Bottlenecks
- Reduce output frequency
- Use parallel I/O (HDF5, MPI-IO)
- Write to fast scratch storage
- Compress output data
Scaling Bottlenecks
- Investigate communication overhead
- Check for load imbalance
- Reduce synchronization points
- Use asynchronous communication
- Consider hybrid MPI+OpenMP
Memory Bottlenecks
- Reduce mesh resolution
- Use iterative solver (lower memory than direct)
- Enable out-of-core computation
- Increase number of processors
- Use single precision where appropriate
Security
Input Validation
- User-supplied
--pattern regex values are validated for length (500 chars max) and rejected if they contain constructs prone to catastrophic backtracking (ReDoS)
- Scaling data entries are validated for finite time values, integer processor counts, and bounded run count (10,000 max)
available_gb is validated as a positive finite number; mesh dimensions and field parameters are validated as positive integers
--type (scaling type) is validated against a fixed allowlist (strong, weak)
- All loaded JSON files must have an object (dict) as root element
File Access
timing_analyzer.py reads a single log file specified by --log; log files are capped at 500 MB and rejected before parsing
scaling_analyzer.py, memory_profiler.py, and bottleneck_detector.py read JSON files capped at 100 MB
- Phase names extracted from log files are truncated to 200 characters and stripped of control characters to prevent prompt-injection payloads from propagating into agent context
- No scripts write to the filesystem; all output goes to stdout
Tool Restrictions
- Read: Used to inspect script source, references, simulation logs, and result files
- Write: Used to save profiling reports or optimization recommendations; writes are scoped to the user's working directory
- Grep/Glob: Used to locate log files, result files, and search references
- The skill's
allowed-tools excludes Bash to prevent the agent from executing arbitrary commands when processing untrusted simulation logs or result files
Safety Measures
- No
eval(), exec(), or dynamic code generation
- All subprocess calls use explicit argument lists (no
shell=True)
- Reduced tool surface (no Bash) limits the agent to read/write operations only
- Phase names and diagnostic strings are sanitized before inclusion in output to prevent injection
Limitations
- Log parsing: Depends on pattern matching; may miss unusual formats
- Scaling analysis: Requires at least 2 runs for meaningful results
- Memory estimation: Approximate; actual usage may vary
- Recommendations: General guidance; may need domain-specific tuning
References
references/profiling_guide.md - Profiling concepts and interpretation
references/optimization_strategies.md - Detailed optimization approaches
Version History
- v1.0.0 (2025-01-22): Initial release with 4 profiling scripts
1---2name: performance-profiling3description: >4 Identify computational bottlenecks, analyze parallel scaling, estimate memory5 requirements, and generate optimization recommendations for materials6 simulations — parse timing logs to find dominant phases (solver, assembly,7 I/O), evaluate strong and weak scaling efficiency, profile memory from mesh8 and field parameters, and detect bottlenecks with actionable fix suggestions.9 Use when a simulation is running slower than expected, investigating MPI10 scaling efficiency, planning HPC resource allocation, deciding whether to11 tune the preconditioner or reduce I/O frequency, or estimating if a problem12 fits in available RAM, even if the user only says "my simulation is too13 slow" or "how many nodes do I need."14allowed-tools: Read, Write, Grep, Glob15metadata:16 source: https://skillmd.com/skills/zuoguyoupan2023/performance-profiling17 author: HeshamFS18 version: "1.1.0"19 security_tier: medium20 security_reviewed: true21 tested_with:22 - claude-code23 - gemini-cli24 - vs-code-copilot25 eval_cases: 526 last_reviewed: "2026-03-26"27---2829# Performance Profiling3031## Goal3233Provide tools to analyze simulation performance, identify bottlenecks, and recommend optimization strategies for computational materials science simulations.3435## Requirements3637- Python 3.10+38- No external dependencies (uses Python standard library only)39- Works on Linux, macOS, and Windows4041## Inputs to Gather4243Before running profiling scripts, collect from the user:4445| Input | Description | Example |46|-------|-------------|---------|47| Simulation log | Log file with timing information | `simulation.log` |48| Scaling data | JSON with multi-run performance data | `scaling_data.json` |49| Simulation parameters | JSON with mesh, fields, solver config | `params.json` |50| Available memory | System memory in GB (optional) | `16.0` |5152## Decision Guidance5354### When to Use Each Script5556```57Need to identify slow phases?58├── YES → Use timing_analyzer.py59│ └── Parse simulation logs for timing data60│61Need to understand parallel performance?62├── YES → Use scaling_analyzer.py63│ └── Analyze strong or weak scaling efficiency64│65Need to estimate memory requirements?66├── YES → Use memory_profiler.py67│ └── Estimate memory from problem parameters68│69Need optimization recommendations?70└── YES → Use bottleneck_detector.py71 └── Combine analyses and get actionable advice72```7374### Choosing Analysis Thresholds7576| Metric | Good | Acceptable | Poor |77|--------|------|------------|------|78| Phase dominance | <30% | 30-50% | >50% |79| Parallel efficiency | >0.80 | 0.70-0.80 | <0.70 |80| Memory usage | <60% | 60-80% | >80% |8182## Script Outputs (JSON Fields)8384| Script | Key Outputs |85|--------|-------------|86| `timing_analyzer.py` | `timing_data.phases`, `timing_data.slowest_phase`, `timing_data.total_time` |87| `scaling_analyzer.py` | `scaling_analysis.results`, `scaling_analysis.efficiency_threshold_processors` |88| `memory_profiler.py` | `memory_profile.total_memory_gb`, `memory_profile.per_process_gb`, `memory_profile.warnings` |89| `bottleneck_detector.py` | `bottlenecks`, `recommendations` |9091## Workflow9293### Complete Profiling Workflow94951. **Analyze timing** from simulation logs962. **Analyze scaling** from multi-run data (if available)973. **Profile memory** from simulation parameters984. **Detect bottlenecks** and get recommendations995. **Implement optimizations** based on recommendations1006. **Re-profile** to verify improvements101102### Quick Profiling (Timing Only)1031041. **Run timing analyzer** on simulation log1052. **Identify dominant phases** (>50% of runtime)1063. **Apply targeted optimizations** to dominant phases107108## CLI Examples109110### Timing Analysis111112```bash113# Basic timing analysis114python3 scripts/timing_analyzer.py \115 --log simulation.log \116 --json117118# Custom timing pattern119python3 scripts/timing_analyzer.py \120 --log simulation.log \121 --pattern 'Step\s+(\w+)\s+took\s+([\d.]+)s' \122 --json123```124125### Scaling Analysis126127```bash128# Strong scaling (fixed problem size)129python3 scripts/scaling_analyzer.py \130 --data scaling_data.json \131 --type strong \132 --json133134# Weak scaling (constant work per processor)135python3 scripts/scaling_analyzer.py \136 --data scaling_data.json \137 --type weak \138 --json139```140141### Memory Profiling142143```bash144# Estimate memory requirements145python3 scripts/memory_profiler.py \146 --params simulation_params.json \147 --available-gb 16.0 \148 --json149```150151### Bottleneck Detection152153```bash154# Detect bottlenecks from timing only155python3 scripts/bottleneck_detector.py \156 --timing timing_results.json \157 --json158159# Comprehensive analysis with all inputs160python3 scripts/bottleneck_detector.py \161 --timing timing_results.json \162 --scaling scaling_results.json \163 --memory memory_results.json \164 --json165```166167## Conversational Workflow Example168169**User**: My simulation is taking too long. Can you help me identify what's slow?170171**Agent workflow**:1721. Ask for simulation log file1732. Run timing analyzer:174 ```bash175 python3 scripts/timing_analyzer.py --log simulation.log --json176 ```1773. Interpret results:178 - If solver dominates (>50%): Recommend preconditioner tuning179 - If assembly dominates: Recommend caching or vectorization180 - If I/O dominates: Recommend reducing output frequency1814. If user has multi-run data, analyze scaling:182 ```bash183 python3 scripts/scaling_analyzer.py --data scaling.json --type strong --json184 ```1855. Generate comprehensive recommendations:186 ```bash187 python3 scripts/bottleneck_detector.py --timing timing.json --scaling scaling.json --json188 ```189190## Interpretation Guidance191192### Timing Analysis193194| Scenario | Meaning | Action |195|----------|---------|--------|196| Solver >70% | Solver-dominated | Tune preconditioner, check tolerance |197| Assembly >50% | Assembly-dominated | Cache matrices, vectorize, parallelize |198| I/O >30% | I/O-dominated | Reduce frequency, use parallel I/O |199| Balanced (<30% each) | Well-balanced | Look for algorithmic improvements |200201### Scaling Analysis202203| Efficiency | Meaning | Action |204|------------|---------|--------|205| >0.80 | Excellent scaling | Continue scaling up |206| 0.70-0.80 | Good scaling | Monitor at larger scales |207| 0.50-0.70 | Poor scaling | Investigate communication/load balance |208| <0.50 | Very poor scaling | Reduce processor count or redesign |209210### Memory Profile211212| Usage | Meaning | Action |213|-------|---------|--------|214| <60% available | Safe | No action needed |215| 60-80% available | Moderate | Monitor, consider optimization |216| >80% available | High | Reduce resolution or increase processors |217| >100% available | Exceeds capacity | Must reduce problem size |218219## Error Handling220221| Error | Cause | Resolution |222|-------|-------|------------|223| `Log file not found` | Invalid path | Verify log file path |224| `No timing data found` | Pattern mismatch | Provide custom pattern with --pattern |225| `At least 2 runs required` | Insufficient data | Provide more scaling runs |226| `Missing required parameters` | Incomplete params | Add mesh and fields to params file |227228## Optimization Strategies by Bottleneck Type229230### Solver Bottlenecks231- Use algebraic multigrid (AMG) preconditioner232- Tighten solver tolerance if over-solving233- Consider direct solver for small problems234- Profile matrix assembly vs solve time235236### Assembly Bottlenecks237- Cache element matrices if geometry is static238- Use vectorized assembly routines239- Consider matrix-free methods240- Parallelize assembly with coloring241242### I/O Bottlenecks243- Reduce output frequency244- Use parallel I/O (HDF5, MPI-IO)245- Write to fast scratch storage246- Compress output data247248### Scaling Bottlenecks249- Investigate communication overhead250- Check for load imbalance251- Reduce synchronization points252- Use asynchronous communication253- Consider hybrid MPI+OpenMP254255### Memory Bottlenecks256- Reduce mesh resolution257- Use iterative solver (lower memory than direct)258- Enable out-of-core computation259- Increase number of processors260- Use single precision where appropriate261262## Security263264### Input Validation265- User-supplied `--pattern` regex values are validated for length (500 chars max) and rejected if they contain constructs prone to catastrophic backtracking (ReDoS)266- Scaling data entries are validated for finite time values, integer processor counts, and bounded run count (10,000 max)267- `available_gb` is validated as a positive finite number; mesh dimensions and field parameters are validated as positive integers268- `--type` (scaling type) is validated against a fixed allowlist (`strong`, `weak`)269- All loaded JSON files must have an object (dict) as root element270271### File Access272- `timing_analyzer.py` reads a single log file specified by `--log`; log files are capped at 500 MB and rejected before parsing273- `scaling_analyzer.py`, `memory_profiler.py`, and `bottleneck_detector.py` read JSON files capped at 100 MB274- Phase names extracted from log files are truncated to 200 characters and stripped of control characters to prevent prompt-injection payloads from propagating into agent context275- No scripts write to the filesystem; all output goes to stdout276277### Tool Restrictions278- **Read**: Used to inspect script source, references, simulation logs, and result files279- **Write**: Used to save profiling reports or optimization recommendations; writes are scoped to the user's working directory280- **Grep/Glob**: Used to locate log files, result files, and search references281- The skill's `allowed-tools` excludes `Bash` to prevent the agent from executing arbitrary commands when processing untrusted simulation logs or result files282283### Safety Measures284- No `eval()`, `exec()`, or dynamic code generation285- All subprocess calls use explicit argument lists (no `shell=True`)286- Reduced tool surface (no Bash) limits the agent to read/write operations only287- Phase names and diagnostic strings are sanitized before inclusion in output to prevent injection288289## Limitations290291- **Log parsing**: Depends on pattern matching; may miss unusual formats292- **Scaling analysis**: Requires at least 2 runs for meaningful results293- **Memory estimation**: Approximate; actual usage may vary294- **Recommendations**: General guidance; may need domain-specific tuning295296## References297298- `references/profiling_guide.md` - Profiling concepts and interpretation299- `references/optimization_strategies.md` - Detailed optimization approaches300301## Version History302303- **v1.0.0** (2025-01-22): Initial release with 4 profiling scripts