Vayu

Vayu — The Swift Wind (FastAPI)

Vayu is speed with direction. FastAPI earns its name only if nothing blocks the event loop and every request knows exactly who it serves.

Project layout

• Structure by layer, imported downward only:
  • app/routers/ — HTTP concerns: parse, authorize, delegate, respond. No business logic.
  • app/services/ — business logic and LLM orchestration. No Request/Response objects.
  • app/repositories/ — all SQL/ORM against Render Postgres. No business logic.
  • app/schemas/ — Pydantic models. app/core/ — settings, auth, deps.
• A router function longer than ~30 lines is hiding a service. Extract it.
• One APIRouter per resource, mounted under /api/v1. Version from day one — /api/v1 costs nothing now and saves a migration later.

Pydantic everywhere

• Pydantic v2 models for every request body and every response (response_model=). Raw dict in a signature or return is a review comment.
• Separate XCreate / XUpdate / XOut models — never expose the DB row shape directly, never accept fields the client shouldn't set (user_id, created_at).
• model_config = ConfigDict(extra="forbid") on request models: unknown fields are client bugs — surface them, don't swallow them.
• Settings via pydantic-settings in app/core/config.py, read from env once at startup, injected via Depends — no os.environ reads scattered through code, fail fast on missing config (see kubera for what's secret).

Async discipline

• async def routes must never block: no requests, no time.sleep, no sync ORM calls, no sync LLM SDK calls. One blocking call stalls every request on the worker.
• Use async clients end to end: httpx.AsyncClient, async SQLAlchemy/asyncpg, the provider's async SDK. Unavoidably-sync work goes through run_in_threadpool.
• Shared clients (HTTP, DB engine, LLM) are created once in the lifespan handler and injected — never constructed per request.
• Every outbound call has an explicit timeout. LLM calls get generous but finite ones; the default of "forever" is an outage waiting (see dhanvantari).

Auth — trust only the verified token

• Every non-public route depends on a CurrentUser dependency that verifies the Supabase JWT: signature against the project JWKS (cached, with refresh), exp, and audience authenticated (see yama).
• The user ID comes from the verified sub claim, nowhere else. A user_id in a body, path, or query is never trusted — ignore it or 403 on mismatch.
• No auth branching inside handlers: authorization lives in dependencies, so a route's security is visible in its signature.
• 401 for missing/invalid token, 403 for valid token without rights, 404 to hide whether another user's resource exists.

Errors

• Raise HTTPException with a stable machine-readable shape: {"detail": {"code": "conversation_not_found", "message": …}}. The frontend switches on code, not on prose.
• One global exception handler logs the full traceback with request ID (see chitragupta) and returns a generic 500 — internals, SQL, stack frames, and provider error bodies never reach the client.
• Map upstream LLM failures deliberately: provider 429 → 503 with Retry-After; provider timeout → 504; content-filter refusal → a typed 200 the UI can render honestly. Never let a raw provider exception become your response.

Streaming chat — the core endpoint

• Stream completions to the frontend as SSE via StreamingResponse (media_type="text/event-stream"); no buffering the full completion server-side.
• Send typed events, not bare text: event: token, event: done (with message ID and token counts), event: error. The client should never have to guess whether a dropped connection was completion or failure.
• Handle client disconnects: check await request.is_disconnected() in the relay loop and cancel the upstream provider call — abandoned generations still bill (see lakshmi).
• Persist the user message before generation starts; persist the assistant message with usage on done, and persist partial output flagged interrupted on disconnect or stop (see shesha for the schema).
• Model ID, temperature, and system prompt version are recorded with every generation — debugging an LLM answer starts from what was actually sent (see durga).

CORS

• Explicit allow_origins: the production domain plus the Vercel preview pattern via allow_origin_regex (e.g. https://.*-<team>\.vercel\.app). Never ["*"] with credentials.
• CORS config lives in settings, per environment — not hardcoded in main.py.

Operational endpoints

• /healthz: cheap liveness, no dependencies, for Render's health check (see airavata). /readyz: checks DB connectivity, for deploy gating.
• OpenAPI stays accurate because it's generated — but summaries, tags, and response models are written; an endpoint without a response_model documents nothing.

Before merging backend work — checklist

• No blocking calls in async routes; all clients async and lifespan-managed
• Every route has response_model; request models forbid extra fields
• Auth via verified-JWT dependency; no user ID trusted from the client
• Streaming endpoints emit typed events and handle disconnect + provider failure
• Every outbound call has a timeout; provider errors mapped, not leaked
• CORS origins explicit and environment-driven