Performance investigation
Define the problem
Record:
- User-visible metric and target.
- Baseline and regression size.
- Workload, concurrency, dataset, and environment.
- Measurement window and statistical variation.
- Relevant code, configuration, dependency, or traffic changes.
Do not compare results produced by materially different workloads.
Decompose the cost
Break the path into:
- Queueing and scheduling.
- CPU and instruction cost.
- Allocation and memory pressure.
- Synchronization and contention.
- Storage and filesystem I/O.
- Network transfer and remote dependency time.
- Retries, timeouts, and duplicated work.
Gather evidence
Prefer:
- Profiles and traces.
- Histograms and percentiles.
- Hardware and runtime counters.
- Allocation and lock-contention data.
- Controlled A/B experiments.
Avoid optimizing from intuition or isolated microbenchmarks.
Recommend an optimization
Include:
- Identified bottleneck.
- Supporting measurement.
- Proposed change.
- Expected effect and assumptions.
- Correctness and operational risks.
- Benchmark and production validation plan.
- Rollback or disablement strategy.