Garuda

Garuda — Fastest of All Beings (Performance)

Garuda measures first, then strikes: speed comes from profiling and budgets, never from guessing.

Measure before optimizing

• Never optimize on a hunch. Profile, find the top bottleneck, fix it, re-measure. One bottleneck at a time.
• Python: py-spy (attach to live processes, flame graphs) or cProfile + snakeviz; pytest-benchmark for hot functions.
• Frontend: Chrome DevTools Performance panel for runtime, Lighthouse for load metrics (LCP, INP, CLS), React Profiler for render churn.
• Optimize p95/p99, not averages — averages hide the requests users complain about.
• Set latency budgets and enforce them: API p95 < 300 ms, page LCP < 2.5 s, LLM first token < 1.5 s. A change that busts the budget is a regression, same as a failing test.

Backend

• Async I/O for anything that waits (see vishnu); never block the event loop with CPU work or sync calls — offload to a worker/thread pool.
• Connection pooling for DB, HTTP, and Redis clients — create clients once at startup, never per request.
• Cache in layers: in-process (LRU) → Redis → CDN. Every cache has a TTL and an invalidation story; cache the expensive computed result, not the cheap lookup.
• Slow queries usually mean missing indexes — EXPLAIN ANALYZE first; index design and query tuning defer to varuna.
• Do less work: return only needed fields, push filtering/aggregation into the DB, move non-critical work (emails, analytics) to background queues.

Frontend

• Bundle size budget: < 200 KB gzipped initial JS. Enforce in CI (size-limit); inspect regressions with a bundle analyzer.
• Code-split by route and lazy-load below-the-fold components (dynamic import, React.lazy).
• Images: modern formats (WebP/AVIF), explicit width/height, loading="lazy", serve responsive sizes via CDN.
• Kill render waterfalls: fetch in parallel, preload critical data, avoid client-side request chains (fetch on the server where the framework allows).

AI-native specifics

• Stream LLM responses always — time-to-first-token is the perceived latency, not total completion time.
• Token count = latency: trim prompts, cap max_tokens, summarize long context instead of pasting it. Every token in and out costs milliseconds and money.
• Cache LLM calls: exact-match on (model, prompt version, input) first; add semantic caching (embedding-similarity lookup) for paraphrased repeat queries.
• Use the smallest model that meets the quality bar (verify with evals) — route easy inputs to the small model, escalate hard ones.
• Parallelize independent LLM calls with asyncio.gather / Promise.all; chain only when an output truly feeds the next input.
• Use provider prompt caching for long static system prompts and shared context.

Before claiming "optimized" — checklist

• Profiled before and after; numbers in the PR description
• p95 within the stated latency budget
• No per-request client creation; pools and caches have TTLs
• Frontend bundle within budget; images and routes lazy-loaded
• LLM paths stream, cache, and run independent calls in parallel