# Observability And Instrumentation

> Add structured logging, metrics, and tracing so systems are debuggable and operable in production. Use when implementing new services, adding important features, or improving production readiness.

- Skill: `itsual/observability-and-instrumentation` (Agent Skill)
- Install (CLI): `npx skillmds@latest add itsual/observability-and-instrumentation`
- Raw SKILL.md: https://api.skillmd.com/api/skills/itsual/observability-and-instrumentation/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: itsual (https://skillmd.com/u/itsual)
- Updated: 2026-09-21
- Page: https://skillmd.com/skills/itsual/observability-and-instrumentation

---


# Observability and Instrumentation

## Overview

If you cannot observe it, you cannot operate it. Instrument as you build, not as an afterthought.

## When to Use

- Building new services or major features
- Preparing for production
- Investigating production issues (to improve future signal)
- Defining SLOs or alerts

## Core Practices

### Logging
- Structured (JSON) preferred
- Include request/correlation IDs
- Log at appropriate levels
- Avoid logging secrets or high-cardinality sensitive data

### Metrics
- RED (Rate, Errors, Duration) for services
- USE (Utilization, Saturation, Errors) for resources
- Business metrics where they aid decisions

### Tracing
- Propagate context across service boundaries
- Focus on critical paths first

### Alerting
- Alert on symptoms (user-impacting) more than causes
- Make alerts actionable
- Avoid noisy alerts that train people to ignore them

## Checklist

- [ ] Key operations have timing and error metrics
- [ ] Logs allow tracing a request across components
- [ ] Dashboards exist for the golden signals
- [ ] Alerts are defined for true pain points

## Verification

- You can answer “is it working?” and “why is it slow/failing?” from the instrumentation
- New code paths are not invisible in production