Performance And Cost Optimization

Performance and Cost Optimization

Overview

Define what "fast enough" and "cheap enough" mean before optimizing anything. Output is .forge/performance-budget.md — latency budgets per request path, LLM cost budgets per call type, the caching strategy (key schema, TTLs, invalidation), frontend bundle budgets, and the profiling targets. Pairs with observability (the dashboards that measure compliance) and scalability-analysis (the growth projection that decides when to invest).

When to Use

• A new service is being designed and there's no latency or cost target
• An LLM-heavy feature is shipping and per-call cost is unbounded
• The frontend bundle is growing organically and ship time is increasing
• A query or endpoint is approaching its SLO and the team is debating where to optimize
• Caching is being added ad-hoc with no key convention or invalidation story

When NOT to Use

• A one-off script with no user-facing latency
• Pure refactoring or rename work with no perf surface
• "It feels slow" with no measurement — start with observability first

Common Rationalizations

Red Flags

• An endpoint in production with no latency SLO and no dashboard
• An LLM call with no max_tokens, no model selection rationale, and no cost tracking
• A cache layer with no documented TTL or invalidation event
• A frontend route loading >500KB of initial JS
• A backend hot path that has never been profiled
• A "background job" that takes >30s with no chunking or progress reporting
• Optimization work happening before measurement
• An LLM prompt that grows unbounded with conversation history

Core Process

Step 1: Define latency budgets per request type

In .forge/performance-budget.md:

Adjust per project, but write the numbers down. Every hot path gets a budget.

Step 2: Set LLM cost budgets per call type

For every LLM call in the system, document:

• Model selection rationale (why Sonnet vs Haiku vs local)
• Input token cap (truncate or summarize context past N tokens)
• Output token cap (max_tokens always set)
• Cost target per call (e.g., "<$0.01 for the autocomplete call, <$0.05 for the summary call")
• Cost-per-user-per-day budget (the rate-limit threshold)
• Caching strategy if the same prompt recurs (prompt cache, response cache, or both)

Append to .forge/performance-budget.md.

Step 3: Design caching strategy

For every cache layer:

• Key schema — explicit, deterministic, includes version (v1:user:{id}:profile)
• TTL — short for frequently-changing data (60s), long for slowly-changing (1h), forever for immutable (with versioned keys)
• Invalidation events — which mutations bust which cache keys, documented as a table
• Negative caching — cache "not found" for short TTL to absorb thundering-herd lookups
• Stale-while-revalidate for read-heavy paths where eventual consistency is acceptable
• Failure mode — what happens when the cache is down (fall through to source, or fail closed)

A cache without a documented invalidation event is a stale-data bug waiting to ship.

Step 4: Set frontend bundle budgets

• Initial JS for any user-facing route: <200KB gzip.
• Initial CSS: <50KB.
• LCP-impacting resources (above-the-fold images): preloaded, properly sized.
• Code-split per route. Lazy-load heavy components (charts, editors, maps).
• Bundle analyzer in CI; budget breach blocks the build.

Step 5: Identify profiling targets

For each service:

• Top 3 endpoints by traffic — profiled with production-shaped data
• Top 3 endpoints by latency — flame graphs captured, reviewed
• Top 3 database queries by cost — EXPLAIN plans attached to .forge/database-design.md
• Any code path inside a setInterval, setTimeout, or render loop

Profile before optimizing. Optimization without a baseline is decoration.

Step 6: Define cost-tracking dashboards (cross-ref observability)

• Per-endpoint cost per request (compute + DB + cache + LLM + egress)
• Cost-per-active-user per day
• LLM spend per feature per day
• Top 10 most-expensive users per week (for usage-based pricing or abuse detection)

Verification

• .forge/performance-budget.md written
• Every hot-path endpoint has a documented p95 and p99 latency budget
• Every LLM call type has model rationale, max_tokens, and cost target
• Every cache layer has documented key schema, TTL, and invalidation events
• Frontend bundle is <200KB initial JS per route (CI-enforced)
• Top traffic and top latency endpoints have been profiled with flame graphs
• Cost-per-user-per-day dashboard exists and has an alert threshold
• No LLM prompt grows unbounded with user input or conversation length