Surya

Surya — Who Illuminates All (Monitoring & Observability)

Surya lights every corner of production: nothing runs unmeasured, nothing fails unseen.

What to measure

• Four golden signals for every system: latency, traffic, errors, saturation.
• RED method per service: Rate (req/s), Errors (failed/s), Duration (latency distribution).
• Measure latency as distributions — p50/p95/p99 — never averages. Averages hide the users who suffer.
• Saturation: connection pool usage, queue depth, memory/CPU headroom, disk %.

Instrumentation

• Use OpenTelemetry for first-party code — traces, metrics, and logs through one SDK, vendor-neutral.
  • Python: opentelemetry-sdk + auto-instrumentation for FastAPI/requests/SQLAlchemy.
  • Node/TS: @opentelemetry/sdk-node + auto-instrumentations for http/express/pg.
• Every service emits RED metrics out of the box; custom business metrics (signups, orders) added deliberately.
• Trace sampling: head-sample at ~10% for high-traffic paths, but always keep error traces (tail sampling if available).
• Trace context propagates across services and queues — same IDs as chitragupta logging.

Dashboards

• One dashboard per service, same layout everywhere: RED at top, saturation below, dependencies at bottom.
• Show p50/p95/p99 latency, error rate, and traffic on every service dashboard.
• Dashboards answer "is it healthy?" in 10 seconds. If you need tribal knowledge to read it, fix the dashboard.

SLOs and checks

• Define SLOs before defining alerts (e.g. 99.5% of requests < 500ms over 30 days). Alert on burn rate, not raw blips (see narada).
• Uptime checks (external synthetic probes) for every public endpoint and critical user flow, from outside your own infra.
• Health endpoints: /healthz (liveness, cheap) and /readyz (readiness, checks dependencies).

AI-native specifics

• Per-endpoint token usage and cost dashboards — cost is a golden signal for LLM products. Track $/request and $/day per feature.
• Model latency breakdown: TTFT (time to first token) vs total generation time; TTFT drives perceived UX for streaming.
• Track eval scores on sampled production traffic, not just offline — a model or prompt change shows up here first.
• Drift detection on inputs: monitor input length, language mix, and topic distribution; alert on shifts that invalidate prompts or evals.
• Tag all LLM metrics with model ID and prompt version so regressions are attributable.

Before shipping monitoring — checklist

• RED metrics + saturation emitted via OpenTelemetry
• Dashboard with p50/p95/p99, standard layout
• SLO defined; alerts derive from it
• External uptime check on public endpoints
• LLM cost, TTFT, and eval-score tracking tagged by model + prompt version