# Std Monitoring

> Monitoring/observability standards — structured logs with request_id, no sensitive data, health checks, CloudWatch alarms, Sentry. Use when adding logging, jobs, or controllers.

- Skill: `kaakati/std-monitoring` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add kaakati/std-monitoring`
- Raw SKILL.md: https://api.skillmd.com/api/skills/kaakati/std-monitoring/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: Kaakati (https://skillmd.com/u/kaakati)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/kaakati/std-monitoring

---


# Monitoring & Observability Standards

**Enforcement**: code-reviewer skill (Step 8: Stack-Specific Checks), devops-engineer agent (Step 8: Configure Monitoring), deploy skill (Step 7: Monitoring).

## Structured Logging
- Use lograge gem for Rails to produce structured JSON logs
- Every log entry must include: timestamp, level, request_id, user_id, action
- Never log sensitive data (passwords, tokens, PII)
- Use log levels consistently: DEBUG for dev, INFO for requests, WARN for recoverable issues, ERROR for failures

## Request Tracing
- **You do not build this — Rails already does.** `ActionDispatch::RequestId` is in the default
  middleware stack: it adopts an incoming `X-Request-Id` (typically set by the load balancer, so
  your logs join to the LB's for free) or generates a uuid, exposes it as `request.request_id`,
  and returns it to the client in the `X-Request-Id` header.
- **`config.log_tags = [:request_id]`** — one line, and every log line in the request carries it.
- Include request_id in all log entries, error reports, and API responses.
- **The trace breaks at the async boundary.** A Sidekiq job runs in another process with no
  request: `request_id` is `nil` there unless you propagate it through client/server middleware.
  That is exactly where you need it — the async work is what fails at 3am.
- Centrifugo messages must carry request_id for end-to-end tracing.
- The id is **outside input** (Rails sanitizes it to 255 alphanumeric-and-dash chars because a
  client can send it) — never interpolate it into SQL or a shell command.
- Deep guide → `references/request-tracing.md`

## Health Check Endpoints
- `/health` — application liveness (returns 200 if app process is running)
- `/health/db` — database connectivity (PostgreSQL/PostGIS query check)
- `/health/redis` — Redis connectivity and latency check
- `/health/centrifugo` — Centrifugo WebSocket server availability
- All health checks must respond within 5 seconds or return 503

## Key Metrics
- Request rate (requests/second by endpoint)
- Error rate (4xx and 5xx by endpoint)
- Latency percentiles: p50, p95, p99
- Sidekiq: queue depth, job execution time, retry count, dead jobs
- Database: active connections, query duration, slow queries (>100ms)
- Redis: memory usage, hit rate, connection count

## CloudWatch Alarms
- ECS task health: alarm if running tasks < desired tasks for >2 minutes
- RDS connections: alarm if connections > 80% of max
- ElastiCache memory: alarm if memory usage > 75%
- Error rate spikes: alarm if 5xx rate > 1% for >5 minutes
- Sidekiq dead jobs: alarm if dead job count increases

## Centrifugo Monitoring
- Track active WebSocket connection count
- Monitor message throughput (messages/second)
- Track channel count and subscriptions per channel
- Alert on connection drops or message delivery failures
- Monitor Centrifugo process memory and CPU usage

## Error Tracking
- Sentry for both Rails backend and React Native mobile
- Configure source maps for React Native for readable stack traces
- Set up release tracking to correlate errors with deployments
- Tag errors with: environment, user_id, device_info (mobile), request_id

## Grafana Dashboards
- Overview dashboard: request rate, error rate, latency, active users
- Infrastructure dashboard: ECS, RDS, ElastiCache, Centrifugo metrics
- Background jobs dashboard: Sidekiq queues, execution times, failures
- Mobile dashboard: crash-free rate, API latency from client, active sessions

## Deep guides (read on demand, do not preload)

- Where `request_id` comes from (`ActionDispatch::RequestId`), `config.log_tags`, `Current`
  attributes for services, **propagating the id into Sidekiq** (the boundary where the trace
  breaks), returning it to clients, and proving it end to end with one `curl`
  → `references/request-tracing.md`

Related, owned elsewhere — do not duplicate: *querying* the logs this produces (Logs Insights,
`gcloud logging read`) → `../log-search`; the API error envelope the id belongs in →
`../std-api-design/references/errors-rails.md`; what must never be logged → `../std-security`.