Grafana
Query, analyze, and report on observability data.
PromQL Patterns
- Use
rate(metric[5m])for counters; adjust window to traffic granularity. - Aggregate with
by (label)to reduce cardinality noise. - Use
histogram_quantile(0.99, rate(metric_bucket[5m]))for latency percentiles.
LogQL Patterns
- Filter first with label selectors
{app="service"}, then pipe|= "error". - Use
| jsonor| patternto extract fields for grouping. - Aggregate log volume with
sum(rate({app="service"}[1m])).
Dashboard Analysis
- Identify the relevant panel by correlating time ranges with incident timestamps.
- Export panel data as CSV for offline spreadsheet analysis when needed.
Alert Investigation
- Read the alert expression and compare current values against thresholds.
- Check for label matches between firing alerts and affected workloads.
Automation
- Use the Grafana API (
/api/datasources/proxy/...or snapshot endpoints) to programmatically fetch query results. - Return findings as structured Markdown with query, value, and interpretation.