Performance

Performance

A structured guide to profiling, caching, database optimization, async patterns, and performance budgets for production services.

When to Activate

• Profiling a slow endpoint or service
• Implementing a caching layer (in-process, Redis, or HTTP)
• Optimizing a database query or fixing N+1 problems
• Setting a performance budget for an API endpoint
• Reducing memory usage or GC pressure
• Choosing between sync and async patterns for a workload

Profiling

When to Profile

• Profile before optimizing — never guess where the bottleneck is
• CPU profiling — where is time spent (function call time)?
• Memory profiling — what objects are consuming heap space?
• I/O profiling — what is blocking on disk or network?

Python — cProfile + snakeviz

import cProfile
import pstats
import io

pr = cProfile.Profile()
pr.enable()
result = my_slow_function()
pr.disable()

s = io.StringIO()
ps = pstats.Stats(pr, stream=s).sort_stats('cumulative')
ps.print_stats(20)  # top 20 slowest functions
print(s.getvalue())

# Profile a whole script from the command line:
# python -m cProfile -o output.prof script.py
# snakeviz output.prof  # opens interactive flame graph in browser

Memory profiling with memory_profiler:

# pip install memory-profiler
from memory_profiler import profile

@profile
def my_function():
    # annotated line-by-line memory usage
    data = [x for x in range(10_000_000)]
    return data

TypeScript/Node.js — clinic.js + 0x

# CPU flame graph
npx 0x -- node dist/server.js
# Opens a generated .html flame graph in the browser

# Heap snapshot + event loop lag
npx clinic doctor -- node dist/server.js

# CPU flame graph via clinic
npx clinic flame -- node dist/server.js

# Async waterfall / I/O bottlenecks
npx clinic bubbleprof -- node dist/server.js

Go — pprof

import (
    "net/http"
    _ "net/http/pprof" // side-effect import registers /debug/pprof handlers
)

// In main(), run alongside your app server:
go func() {
    http.ListenAndServe("localhost:6060", nil)
}()

# CPU profile (30-second sample)
go tool pprof http://localhost:6060/debug/pprof/profile?seconds=30

# Memory (heap) profile
go tool pprof http://localhost:6060/debug/pprof/heap

# In the pprof interactive prompt:
# (pprof) top10          — top 10 functions by CPU or memory
# (pprof) web            — open flame graph in browser (requires graphviz)
# (pprof) list FuncName  — annotated source with per-line costs

Reading Flame Graphs

• X-axis — time (box width = proportion of total execution time)
• Y-axis — call stack depth (parent calls children above it)
• Wide flat boxes near the top — hot code paths; primary optimization targets
• Long stacks with narrow top boxes — deep recursion; usually not a problem
• Plateaus — the widest boxes in the middle of a stack often hide the real work

Caching Strategies

Strategy Comparison

Cache-Aside Pattern (most common)

def get_user(user_id: str) -> User:
    # 1. Check cache first
    cached = redis.get(f"user:{user_id}")
    if cached:
        return User.from_json(cached)

    # 2. Cache miss — fetch from DB
    user = db.query(User).filter(User.id == user_id).first()

    # 3. Populate cache with TTL
    redis.setex(f"user:{user_id}", 300, user.to_json())  # 5 min TTL
    return user

Caching Pattern Comparison

Cache Invalidation

• TTL (time-to-live) — simplest; accept stale data up to TTL duration
• Event-driven — invalidate on write (redis.delete(f"user:{user_id}") after UPDATE)
• Write-through — always write to both cache and DB; no stale data, but slower writes
• Avoid — invalidating cache on reads is an anti-pattern; adds latency to hot paths

In-Process LRU Cache

# Python
from functools import lru_cache

@lru_cache(maxsize=1000)
def get_config(key: str) -> str:
    return db.get_config(key)

// TypeScript
import LRU from 'lru-cache';

const cache = new LRU<string, string>({ max: 1000, ttl: 1000 * 60 * 5 });

function getConfig(key: string): string {
  if (cache.has(key)) return cache.get(key)!;
  const value = db.getConfig(key);
  cache.set(key, value);
  return value;
}

// Go
import "github.com/hashicorp/golang-lru/v2"

cache, _ := lru.New[string, string](1000)

func getConfig(key string) string {
    if val, ok := cache.Get(key); ok {
        return val
    }
    val := db.GetConfig(key)
    cache.Add(key, val)
    return val
}

HTTP Cache Headers

Key Cache-Control Directives

Conditional Requests (ETag)

# First request
GET /api/products/123
→ 200 OK
   ETag: "v2-abc123"
   Cache-Control: max-age=60

# After TTL expires — client sends ETag back
GET /api/products/123
If-None-Match: "v2-abc123"
→ 304 Not Modified   (no response body — saves bandwidth)
# or, if product changed:
→ 200 OK
   ETag: "v3-def456"

Database N+1 Problem

The Problem

# BAD: N+1 — 1 query for orders + 1 query per order for its user
orders = db.query(Order).all()      # 1 query
for order in orders:
    print(order.user.name)          # N queries (lazy load per order)

With 500 orders this emits 501 queries. Use EXPLAIN ANALYZE or ORM query logging to detect this in review.

Fix Per ORM

Python — SQLAlchemy

from sqlalchemy.orm import selectinload, joinedload

# selectinload: 2 queries total — 1 for orders, 1 IN query for all related users
orders = db.query(Order).options(selectinload(Order.user)).all()

# joinedload: 1 query with a JOIN (better for single related object)
orders = db.query(Order).options(joinedload(Order.user)).all()

TypeScript — Prisma

// BAD
const orders = await prisma.order.findMany();
for (const order of orders) {
  const user = await prisma.user.findUnique({ where: { id: order.userId } });
}

// GOOD — Prisma batches the related fetches automatically
const orders = await prisma.order.findMany({
  include: { user: true },
});

Go — GORM

var orders []Order

// BAD — N separate queries inside the loop
db.Find(&orders)
for i := range orders {
    db.First(&orders[i].User, orders[i].UserID)
}

// GOOD — Preload issues a single IN query for all users
db.Preload("User").Find(&orders)

Detecting N+1 in Practice

Async Patterns

I/O-Bound vs CPU-Bound

Python — asyncio for I/O-Bound Work

import asyncio
import aiohttp

async def fetch_all(urls: list[str]) -> list[dict]:
    async with aiohttp.ClientSession() as session:
        tasks = [fetch(session, url) for url in urls]
        return await asyncio.gather(*tasks)  # concurrent, not parallel

async def fetch(session: aiohttp.ClientSession, url: str) -> dict:
    async with session.get(url) as response:
        return await response.json()

CPU-bound work in Python must use ProcessPoolExecutor to escape the GIL:

from concurrent.futures import ProcessPoolExecutor

def cpu_heavy(data: list) -> int:
    return sum(x ** 2 for x in data)

async def process_many(chunks: list[list]) -> list[int]:
    loop = asyncio.get_event_loop()
    with ProcessPoolExecutor() as pool:
        results = await asyncio.gather(
            *[loop.run_in_executor(pool, cpu_heavy, chunk) for chunk in chunks]
        )
    return results

TypeScript/Node.js — Protect the Event Loop

import fs from 'fs';
import { Worker, isMainThread, workerData, parentPort } from 'worker_threads';

// BAD: sync read blocks the event loop for all requests
const data = fs.readFileSync('large-file.json', 'utf8');

// GOOD: async I/O — yields control back to event loop
const data = await fs.promises.readFile('large-file.json', 'utf8');

// BAD: CPU-heavy work in the main thread stalls all requests
const result = heavyComputation(data);

// GOOD: offload CPU work to a worker thread
function runInWorker(payload: unknown): Promise<unknown> {
  return new Promise((resolve, reject) => {
    const worker = new Worker(__filename, { workerData: payload });
    worker.on('message', resolve);
    worker.on('error', reject);
  });
}

Go — Goroutines for Concurrency

// Fan-out: fire N goroutines, collect with WaitGroup + channel
func fetchAll(urls []string) []Result {
    results := make(chan Result, len(urls))
    var wg sync.WaitGroup

    for _, url := range urls {
        wg.Add(1)
        go func(u string) {
            defer wg.Done()
            resp, err := http.Get(u)
            results <- Result{URL: u, Err: err, Body: readBody(resp)}
        }(url)
    }

    wg.Wait()
    close(results)

    var out []Result
    for r := range results {
        out = append(out, r)
    }
    return out
}

Performance Budgets

Deriving a Budget from SLOs

• If the SLO is p99 < 500 ms, the internal service call budget is ~200 ms (leave headroom for network, serialization, retries)
• Decompose latency: total = DB + cache + downstream API + serialization + middleware
• Assign each component a share; the tightest constraint sets the overall shape

Endpoint Budget Reference

CI Regression Detection

# k6 threshold example — fails the PR if p99 regresses
import http from 'k6/http';
import { check } from 'k6';

export const options = {
  thresholds: {
    http_req_duration: ['p(99)<500'],  // fail if p99 > 500ms
  },
};

export default function () {
  const res = http.get('http://localhost:3000/api/users/1');
  check(res, { 'status 200': (r) => r.status === 200 });
}

Run this in CI on every PR:

k6 run --vus 50 --duration 30s load-test.js
# Exit code non-zero if any threshold is breached

Fail the build if p99 degrades more than 20% from the baseline captured on main.

See also: database-design, observability, performance-testing

Red Flags

• Optimizing before profiling — intuition targets the wrong 5% of runtime; always profile with representative load before touching any code
• Profiling with 1K rows when production has 10M — hotspots at small scale vanish or invert at large scale; profile with production-representative data volume
• Blanket eager loading to fix N+1 — fetching every relationship on every query loads data you never use; apply selectinload/joinedload surgically to proven hotspots
• In-process LRU cache across forked workers — forked processes maintain separate memory spaces; a cache write in one worker is invisible to others; use Redis for cross-process caching
• Async for CPU-bound work — Python asyncio and Node.js event loops don't parallelize CPU; CPU-bound work blocks the loop; offload to ProcessPoolExecutor or a task queue
• ETags set but If-None-Match not handled server-side — setting ETag without handling conditional requests means clients never get 304; implement both sides of the exchange
• Mean latency as the primary metric — mean hides tail problems; always track p95 and p99; the slowest 1% of requests represents the worst user experience

Checklist

• Profiled before optimizing — no premature optimization
• Flame graph or profile output captured to identify the actual bottleneck
• N+1 queries detected with EXPLAIN ANALYZE or ORM query logging
• Eager loading configured for all related-entity fetches
• Cache layer added for hot read paths (in-process for single-instance, Redis for distributed)
• Cache keys include version or tenant identifier to prevent cross-user data leaks
• Cache-Control headers set for all public API responses
• ETags implemented for cacheable resources (304 responses save bandwidth)
• async/await or equivalent used for all I/O-bound operations
• CPU-bound work offloaded to worker processes or threads
• Performance budget defined per endpoint category and documented
• Baseline p50/p95/p99 measured before and after changes