Performance Modeling
Build performance models to predict latency, throughput, and identify bottlenecks under various load profiles.
Context
You are optimizing system performance or planning capacity. The user has latency issues or wants to predict performance. Read their current metrics and workload characteristics.
Domain Context
Based on queueing theory and performance modeling research:
- Little's Law: Latency = Queue Depth / Throughput
- Queueing Models: M/M/1 (simple), M/M/c (multi-server) help predict latency distribution
- Response Time Hierarchy: Network < Cache < In-Memory < Disk → Database
- Throughput Ceiling: Database connections, CPU cores, network bandwidth impose limits
- Tail Latency: p99 and p99.9 matter more than average; optimize for tail, not mean
Instructions
Measure Current Performance: Establish baseline latency (p50, p95, p99) and throughput (requests/sec). Identify bottleneck component (database, cache, service).
Build Queueing Model: Model each bottleneck resource (database, cache, API server) as M/M/1 queue. Predict latency at 2x, 5x current load.
Identify Breaking Points: At what load does latency exceed acceptable threshold? At what load do errors appear? Plot latency vs load curve.
Model Optimizations: For each identified bottleneck, model impact of optimization:
- Database: Add read replicas, caching, indexes
- Network: Add CDN, compression
- CPU: Add instances, optimize code
Compare Trade-offs: Cost of optimization (resources, effort) vs latency gain. Choose changes with best ROI.
Anti-Patterns
- Optimizing Average Instead of Tail: p50 is 10ms but p99 is 500ms. Optimize p50 (easy) while ignoring p99 (impacts users). Guard: Always track and optimize p99 latency.
- Missing Measurement Points: Only measure at entry point, missing internal latency. Result: can't see where time is spent. Guard: Measure at every component boundary; trace requests.
- Modeling Without Validation: Build model, predict 2x scaling works, it doesn't. Guard: Validate model against actual load test; adjust assumptions if wrong.
- Forgetting Overhead: Model predicts 50ms latency but actual is 200ms. Result: overhead (GC, context switching) not accounted. Guard: Measure wall-clock time including all overhead.
Further Reading
- The Art of Computer Systems Performance Analysis by Raj Jain — queueing theory and modeling
- Designing Data-Intensive Applications by Martin Kleppmann — performance analysis at scale
- Google's Performance Optimization — case studies and methodologies
1---2name: performance-modeling3description: Model system performance, predict latency under load, identify bottlenecks. Use when optimizing performance or capacity planning.4---56# Performance Modeling78Build performance models to predict latency, throughput, and identify bottlenecks under various load profiles.910## Context1112You are optimizing system performance or planning capacity. The user has latency issues or wants to predict performance. Read their current metrics and workload characteristics.1314## Domain Context1516Based on queueing theory and performance modeling research:1718- **Little's Law**: Latency = Queue Depth / Throughput19- **Queueing Models**: M/M/1 (simple), M/M/c (multi-server) help predict latency distribution20- **Response Time Hierarchy**: Network < Cache < In-Memory < Disk → Database21- **Throughput Ceiling**: Database connections, CPU cores, network bandwidth impose limits22- **Tail Latency**: p99 and p99.9 matter more than average; optimize for tail, not mean2324## Instructions25261. **Measure Current Performance**: Establish baseline latency (p50, p95, p99) and throughput (requests/sec). Identify bottleneck component (database, cache, service).27282. **Build Queueing Model**: Model each bottleneck resource (database, cache, API server) as M/M/1 queue. Predict latency at 2x, 5x current load.29303. **Identify Breaking Points**: At what load does latency exceed acceptable threshold? At what load do errors appear? Plot latency vs load curve.31324. **Model Optimizations**: For each identified bottleneck, model impact of optimization:33 - Database: Add read replicas, caching, indexes34 - Network: Add CDN, compression35 - CPU: Add instances, optimize code36375. **Compare Trade-offs**: Cost of optimization (resources, effort) vs latency gain. Choose changes with best ROI.3839## Anti-Patterns4041- **Optimizing Average Instead of Tail**: p50 is 10ms but p99 is 500ms. Optimize p50 (easy) while ignoring p99 (impacts users). **Guard**: Always track and optimize p99 latency.42- **Missing Measurement Points**: Only measure at entry point, missing internal latency. Result: can't see where time is spent. **Guard**: Measure at every component boundary; trace requests.43- **Modeling Without Validation**: Build model, predict 2x scaling works, it doesn't. **Guard**: Validate model against actual load test; adjust assumptions if wrong.44- **Forgetting Overhead**: Model predicts 50ms latency but actual is 200ms. Result: overhead (GC, context switching) not accounted. **Guard**: Measure wall-clock time including all overhead.4546## Further Reading4748- _The Art of Computer Systems Performance Analysis_ by Raj Jain — queueing theory and modeling49- _Designing Data-Intensive Applications_ by Martin Kleppmann — performance analysis at scale50- _Google's Performance Optimization_ — case studies and methodologies
sethdford