Fastapi Performance And Resilience

FastAPI Performance and Resilience

When to use

Invoke when a scaffolded FastAPI service must hold an SLO under load: blocking calls are stalling the event loop, downstream failures cascade, the connection pool is mis-sized, there is no caching posture or backpressure under burst, retries amplify outages, or there is no load-test gate proving the service meets its latency/throughput budget before release.

Do not use for: the service shell, settings, or error tiers (use fastapi-service-scaffold), auth or OWASP review (use fastapi-auth-and-security-review), OpenTelemetry/metrics/SLO definition (use fastapi-observability-readiness), or broker/task/event integration (use fastapi-async-and-task-integration).

Inputs

Required:

• A service with the fastapi-service-scaffold baseline (settings, logging, ASGI-lifespan shutdown, health probes present).
• Approved architecture/performance budgets and architecture/reliability SLO targets (latency/throughput objective, error budget) — or explicit confirmation they are intentionally deferred.

Optional:

• The observability layer from fastapi-observability-readiness (for event-loop and resilience metrics) if already present.
• Known blocking/CPU-bound operations and downstream dependencies with their own SLAs.
• Expected peak traffic shape (RPS, burst factor, payload sizes) for the load test.

Operating rules

• Never invent the budget or SLO. Latency/throughput targets, the error budget, and acceptable degradation belong to architecture/performance and architecture/reliability. If silent on a number this skill needs, pause and raise an ADR candidate rather than guessing a threshold.
• Extend the scaffold; do not duplicate it. Pool sizes, breaker thresholds, and timeout settings read from the validated settings seam, log through the scaffold logger, and tie shutdown to the scaffold ASGI-lifespan. Resilience metrics reuse the observability seam — do not wire a second telemetry stack.
• The event loop is sacred: no synchronous blocking or CPU-bound work in an async def path. Blocking I/O uses an async driver or run_in_threadpool/anyio.to_thread; CPU work goes to a process pool. Event-loop lag is measured, not assumed.
• The worker model is explicit and matches the workload: Gunicorn + UvicornWorker with a worker count derived from cores per architecture/performance — not a single Uvicorn process pretending to use all cores, and not threads where processes are required.
• Every external call is bounded and isolated: an explicit timeout, a circuit breaker, and a bulkhead (bounded concurrent calls) so one slow dependency cannot consume the whole pool. No unbounded await on a network call.
• Connection pools are sized, not defaulted: DB and HTTP client pool sizes derive from the worker model and the downstream limits, documented against architecture/performance. An unbounded or default pool under load is an outage.
• Retries have a budget: capped attempts, exponential backoff with jitter, and a global retry-budget cap so retries cannot amplify a downstream outage into a self-inflicted DDoS. No retry on non-idempotent operations without an idempotency key.
• Caching posture is deliberate: what is cached, the TTL, the invalidation trigger, and the stampede protection are explicit — never an unbounded process-local dict.
• Backpressure is enforced, not hoped for: bounded concurrency with a shed-load path (429/503 with Retry-After) when saturated, and /readyz reflecting saturation — degrade deliberately rather than collapse.
• Resilience is verified, not asserted: a load-test gate runs the expected traffic shape and a dependency-failure scenario, and passes only if the measured result meets the architecture/reliability SLO. A service without a passing load-test gate is not done.
• A change that does not pass mypy, ruff, tests, the boot smoke check, and the load-test gate is not done. Fix and re-run.

Output contract

The performance and resilience layer MUST conform to:

• observability-standards — event-loop lag, breaker state, pool saturation, shed count, and retry rate are observable via the scaffold/observability seam.
• deployment-standards — the worker model and pool sizes are settings-driven and environment-agnostic; resource limits documented.

Upstream contract: architecture/performance is the source of truth for latency/throughput budgets and pool sizing; architecture/reliability is the source of truth for SLO targets, the error budget, and acceptable degradation. If either is silent on a decision this skill needs, pause and raise an ADR candidate rather than guessing.

Progressive references

• Read references/fastapi-performance-playbook.md when implementing any owned area or checking the anti-pattern list.
• Read references/fastapi-performance-quality-rubric.md before declaring the work complete.
• Use assets/fastapi-performance-and-resilience.template.md as the worker, breaker, caching, and load-test reference.

Process

1. Gather context: load architecture/performance (latency/throughput budgets, pool sizing, payload limits) and architecture/reliability (SLO, error budget, acceptable degradation). Confirm the scaffold baseline. If a needed number is missing, raise an ADR candidate before proceeding — do not pick a threshold.
2. Extend settings: add the worker count, per-dependency timeouts, breaker thresholds, bulkhead concurrency limits, pool sizes, retry budget, cache TTLs, and backpressure limits to the scaffold Settings model and .env.example (placeholders only).
3. Enforce async-path discipline: identify synchronous blocking or CPU-bound operations in async def paths; move blocking I/O to async drivers or run_in_threadpool/anyio.to_thread, CPU work to a process pool. Add an event-loop-lag probe wired into /readyz and the metrics seam.
4. Apply the worker model: Gunicorn + UvicornWorker with a worker count from settings (per architecture/performance); graceful shutdown propagates to all workers via the scaffold ASGI-lifespan.
5. Bound and isolate downstream calls: wrap each external dependency in an explicit timeout, a circuit breaker (open/half-open/closed with documented thresholds), and a bulkhead (bounded concurrent calls). A tripped breaker returns a defined degraded response, not a hang.
6. Size connection pools and set caching posture: DB/HTTP client pool sizes derived from the worker model and downstream limits; a deliberate caching layer with explicit TTL, invalidation, and stampede protection. Document both against architecture/performance.
7. Budget retries: capped attempts with exponential backoff + jitter and a global retry-budget limiter; no retry on non-idempotent calls without an idempotency key. Document the worst-case amplification factor and confirm it stays within the dependency's headroom.
8. Enforce backpressure and build the load-test gate: bounded concurrency with a 429/503 + Retry-After shed path and /readyz reflecting saturation; a load-test script (k6/Locust) running the expected traffic shape plus a dependency-failure scenario, asserting measured p99/throughput against the architecture/reliability SLO and failing the build if missed. Wire it into CI as a gate.
9. Build verification (mandatory): run mypy, ruff check, tests, and the boot smoke check; then run the load-test gate and confirm it passes against the SLO and that breakers/backpressure behave under the failure scenario. Fix and re-run on failure. Validate against the Output contract; document any unresolved gap in the service README.

Outputs

Required:

• Async-path discipline: blocking/CPU work off the event loop; an event-loop-lag probe in /readyz and the metrics seam.
• A settings-driven worker model (Gunicorn + UvicornWorker) with shutdown propagated to all workers.
• Per-dependency timeout + circuit breaker + bulkhead with a defined degraded response.
• Sized connection pools and a deliberate caching posture (TTL, invalidation, stampede protection).
• A retry budget (capped attempts, backoff+jitter, global cap; idempotency-gated) with a documented amplification factor.
• Enforced backpressure with a shed-load path and saturation reflected in /readyz.
• A CI load-test gate asserting measured performance against the architecture/reliability SLO.

Output rules:

• No invented threshold — every budget/SLO number traces to architecture/performance/architecture/reliability or an open ADR candidate.
• The scaffold settings, logging, lifespan, and observability seam are extended, not duplicated.
• No synchronous blocking call on an async def path; no unbounded await on a network call; no retry on a non-idempotent call without an idempotency key.
• The service sheds load deliberately under saturation rather than collapsing.

Quality checks

• mypy, ruff check, tests, and the boot smoke check all pass (or a skip is documented with reason).
• Every budget/SLO number traces to architecture/performance/architecture/reliability (no invented threshold).
• No synchronous blocking or CPU-bound work runs on an async def path; blocking I/O is async or threadpooled, CPU work is in a process pool.
• An event-loop-lag probe feeds /readyz and the metrics seam.
• The worker model (Gunicorn + UvicornWorker, worker count) is settings-driven and shutdown propagates to all workers.
• Every external call has an explicit timeout, a circuit breaker, and a bulkhead; a tripped breaker returns a defined degraded response, not a hang.
• Connection pool sizes are derived and documented; no default/unbounded pool under load.
• Caching posture is explicit (TTL, invalidation trigger, stampede protection) — no unbounded process-local cache.
• Retries are capped with backoff+jitter and a global budget; non-idempotent calls are not retried without an idempotency key; the amplification factor is documented.
• Backpressure sheds load with 429/503 + Retry-After under saturation, and /readyz reflects saturation.
• The CI load-test gate runs the expected traffic shape plus a dependency-failure scenario and fails the build if the SLO is missed (verified passing).
• Event-loop lag, breaker state, pool saturation, shed count, and retry rate are observable via the scaffold/observability seam.
• The service README documents the worker model, pool sizes, caching posture, resilience thresholds, and the load-test gate.

References

• Upstream: architecture/performance, architecture/reliability.
• Baseline this extends: fastapi-service-scaffold. Related: fastapi-auth-and-security-review, fastapi-observability-readiness, fastapi-async-and-task-integration.
• Standards: observability-standards, deployment-standards.