incidentfox Grafana Dashboard Analysis
Authentication
IMPORTANT: Credentials are injected automatically by a proxy layer. Do NOT check for GRAFANA_API_KEY in environment variables - it won't be visible to you. Just run the scripts directly; authentication is handled transparently.
Configuration environment variables you CAN check (non-secret):
GRAFANA_URL- Grafana instance URL (e.g.,https://grafana.company.com)
Available Scripts
All scripts are in .claude/skills/observability-grafana/scripts/
list_dashboards.py - Search/List Dashboards
python .claude/skills/observability-grafana/scripts/list_dashboards.py [--query "kubernetes"]
get_dashboard.py - Get Dashboard Details
python .claude/skills/observability-grafana/scripts/get_dashboard.py --uid DASHBOARD_UID
query_prometheus.py - Query Prometheus via Grafana
python .claude/skills/observability-grafana/scripts/query_prometheus.py --query "PROMQL" [--time-range 60] [--step 1m] [--datasource-id ID]
# Examples (auto-detects Prometheus datasource):
python .claude/skills/observability-grafana/scripts/query_prometheus.py --query "rate(http_requests_total[5m])"
python .claude/skills/observability-grafana/scripts/query_prometheus.py --query "up{job='api'}" --time-range 120
# Explicit datasource ID (use list_datasources.py to find it):
python .claude/skills/observability-grafana/scripts/query_prometheus.py --query "up" --datasource-id 8
Note: The Prometheus datasource ID is auto-detected by default. Use list_datasources.py if you need to target a specific datasource.
list_datasources.py - List Configured Datasources
python .claude/skills/observability-grafana/scripts/list_datasources.py
get_alerts.py - Get Active Alerts
python .claude/skills/observability-grafana/scripts/get_alerts.py
create_dashboard.py - Create Dashboard from Template
python .claude/skills/observability-grafana/scripts/create_dashboard.py \
--title "My Dashboard" \
--template .claude/skills/observability-grafana/templates/vercel-overview.json
# With folder:
python .claude/skills/observability-grafana/scripts/create_dashboard.py \
--title "My Dashboard" \
--template .claude/skills/observability-grafana/templates/vercel-overview.json \
--folder-uid FOLDER_UID
# Overwrite existing:
python .claude/skills/observability-grafana/scripts/create_dashboard.py \
--title "My Dashboard" \
--template .claude/skills/observability-grafana/templates/vercel-overview.json \
--overwrite
Dashboard Templates
Pre-built templates are in .claude/skills/observability-grafana/templates/:
| Template | Description |
|---|---|
vercel-overview.json |
4-panel Vercel monitoring: error count, error rate, log stream, request summary |
PromQL Quick Reference
# Request rate
rate(http_requests_total{service="api"}[5m])
# Error rate %
sum(rate(http_requests_total{status=~"5.."}[5m])) / sum(rate(http_requests_total[5m])) * 100
# P95 latency
histogram_quantile(0.95, rate(http_duration_seconds_bucket{service="api"}[5m]))
# CPU usage
rate(process_cpu_seconds_total{service="api"}[5m])
Investigation Workflow
1. list_datasources.py # Find available datasources
2. list_dashboards.py --query "api" # Find relevant dashboards
3. get_dashboard.py --uid <uid> # Get dashboard details
4. query_prometheus.py --query "rate(http_requests_total{service='api',status=~'5..'}[5m])"
5. get_alerts.py # Check active alerts