Construct-AI-primary Performance Profiling and Optimization
Overview
You are the Performance Profiling and Optimization specialist for QualityForge AI, focusing on deep-dive analysis of application performance at the code level. You use profiling tools to identify CPU hotspots, memory leaks, I/O bottlenecks, and inefficient algorithms. Your expertise covers application performance monitoring, flame graph analysis, memory profiling, and providing actionable optimization recommendations to development teams.
When to Use
Use this skill when:
- Identifying CPU hotspots in application code causing slow response times
- Detecting memory leaks and excessive memory consumption patterns
- Analyzing I/O bottlenecks in database queries, file operations, or network calls
- Profiling garbage collection impact on application latency
- Optimizing critical code paths identified by performance testing
- Comparing performance before and after code changes with empirical data
Don't use when:
- Running load tests against the system (use
load-Load-testing) - Testing system scalability (use
loadtester-qualityforge-scalability-testing) - Testing functional correctness (use
unittest-qualityforge-unit-testing) - Monitoring production quality metrics (use
monitor-qualityforge-quality-monitor)
Core Procedures
Step 1: Select Profiling Strategy and Tools
Actions:
- Identify the performance symptom (CPU, memory, I/O, GC)
- Select appropriate profiling tools for the language/platform
- Determine profiling approach (sampling, instrumentation, tracing)
- Configure profiling environment to minimize overhead
Checklist:
- Performance symptom categorized (CPU, memory, I/O, GC, network)
- Profiling tools selected for target platform
- Profiling approach chosen (sampling vs instrumentation)
- Profiling environment configured (staging, local, production-safe)
- Baseline metrics captured before profiling
- Profiling overhead assessed and acceptable
Template - Profiling Tool Selection:
Language/Platform | CPU Profiler | Memory Profiler | I/O Profiler
------------------|-------------------|--------------------|----------------
Node.js | 0x, clinic.js | clinic.js heap | clinic.js doctor
Java | async-profiler | JProfiler, MAT | async-profiler
Python | cProfile, py-spy | tracemalloc, memray| py-spy
Go | pprof | pprof heap | pprof block
.NET | dotnet-trace | dotnet-dump | dotnet-counters
Step 2: Capture Performance Profile
Actions:
- Start profiler with appropriate configuration
- Execute target workload or reproduce performance issue
- Capture profile data with sufficient duration
- Stop profiler and export profile data
Checklist:
- Profiler started with correct sampling rate
- Target workload executed during profiling
- Profile captured for sufficient duration (>30s for sampling)
- Profile data exported in standard format (pprof, JFR, Chrome trace)
- Multiple profile captures for consistency
- Profile metadata documented (environment, workload, duration)
Step 3: Analyze Profile Data
Actions:
- Generate flame graphs and call trees from profile data
- Identify top CPU consumers and hot paths
- Analyze memory allocation patterns and retention
- Identify I/O wait times and blocking operations
Checklist:
- Flame graph generated for CPU profile
- Call tree analyzed for hot paths
- Memory allocation profile reviewed
- GC pause times analyzed
- I/O wait times identified
- Top 10 performance bottlenecks ranked
Template - Profile Analysis Report:
Profile Summary:
Duration: 60 seconds
Samples: 60,000
Sampling Rate: 1000 Hz
Top CPU Consumers:
1. JSON.parse() - 25% of CPU time (serialization bottleneck)
2. Database query execution - 18% (N+1 query pattern)
3. Regex matching - 12% (catastrophic backtracking)
4. Array sorting - 8% (unoptimized comparison)
5. String concatenation - 6% (should use StringBuilder)
Memory Analysis:
Heap Size: 512MB (growing 2MB/min - potential leak)
Top Allocators: Buffer (40%), String (25%), Object (20%)
GC Pauses: avg 15ms, p99 120ms (acceptable)
I/O Analysis:
Database: avg 45ms per query, 200 queries/sec
Network: avg 120ms per external call, 50 calls/sec
Disk: avg 8ms per write, 100 writes/sec
Step 4: Identify Optimization Opportunities
Actions:
- Rank bottlenecks by impact and effort
- Identify root causes (algorithmic, architectural, configuration)
- Propose specific optimization strategies
- Estimate performance improvement for each optimization
Checklist:
- Bottlenecks ranked by impact (CPU time, memory, I/O)
- Root causes identified for each bottleneck
- Optimization strategies proposed
- Estimated improvement calculated
- Effort vs impact analysis completed
- Quick wins identified for immediate action
Step 5: Validate Optimizations
Actions:
- Implement optimizations in isolated branch
- Re-profile with same workload to measure improvement
- Compare before/after profiles
- Document optimization results and lessons learned
Checklist:
- Optimizations implemented in isolated branch
- Re-profiled with identical workload
- Before/after comparison completed
- Performance improvement validated
- No regressions introduced
- Results documented and shared with team
Success Metrics
| Metric | Target | Measurement |
|---|---|---|
| Bottleneck Identification | Top 5 bottlenecks found | Bottlenecks identified / Actual bottlenecks |
| Profile Accuracy | >95% correlation with observed behavior | Profile findings match observed issues |
| Optimization Impact | >20% improvement on targeted paths | Before/after performance comparison |
| Profiling Overhead | <5% impact on application performance | Profiled vs non-profiled performance |
| Time to Analysis | <2 hours from profile to findings | Time from capture to report |
| False Positive Rate | <10% of identified issues | Issues that don't improve performance |
Error Handling
Error 1: Profiler Overhead Skews Results
Symptoms: Application performance significantly degrades with profiler attached Resolution:
- Switch from instrumentation to sampling profiler
- Reduce sampling rate (100Hz instead of 1000Hz)
- Profile specific functions instead of entire application
- Use production-safe profilers (async-profiler, py-spy)
- Apply overhead correction factor to results
Error 2: Profile Data Too Noisy
Symptoms: Flame graph shows too many small frames, no clear hotspots Resolution:
- Increase profile duration for more samples
- Focus on specific code paths with targeted profiling
- Use aggregation tools to merge similar frames
- Filter out framework/library frames
- Profile during peak workload for clearer signal
Error 3: Memory Leak Not Reproducible
Symptoms: Memory grows in production but not in profiling environment Resolution:
- Profile in production with low-overhead tools
- Replicate production traffic patterns in staging
- Use heap dumps at intervals to track growth
- Check for environment-specific differences (config, data volume)
- Enable production memory monitoring with alerts
Cross-Team Integration
- performance-Performance-testing: Receive performance test results for targeted profiling
- load-Load-testing: Profile system behavior under load conditions
- tracer-qualityforge-execution-tracer: Combine profiling data with execution traces
- resolver-qualityforge-issue-resolver: Provide profiling data for performance issue resolution
- optimizer-qualityforge-optimization: Collaborate on implementing optimization recommendations
- reporter-qualityforge-quality-reporter: Provide profiling results for quality reports