Prometheus API Skill
Query Prometheus monitoring systems via HTTP API at /api/v1.
Quick Reference
Instant Query
curl 'http://<prometheus>:9090/api/v1/query?query=<promql>&time=<timestamp>'
Range Query
curl 'http://<prometheus>:9090/api/v1/query_range?query=<promql>&start=<ts>&end=<ts>&step=<duration>'
Response Format
All responses return JSON:
{
"status": "success" | "error",
"data": <result>,
"errorType": "<string>",
"error": "<string>",
"warnings": ["<string>"]
}
HTTP codes: 400 (bad params), 422 (expression error), 503 (timeout).
Query Endpoints
| Endpoint |
Purpose |
Key Parameters |
/api/v1/query |
Instant query |
query, time, timeout, limit |
/api/v1/query_range |
Range query |
query, start, end, step, timeout, limit |
/api/v1/format_query |
Format PromQL |
query |
/api/v1/series |
Find series by labels |
match[], start, end, limit |
/api/v1/labels |
List label names |
start, end, match[], limit |
/api/v1/label/<name>/values |
Label values |
start, end, match[], limit |
/api/v1/query_exemplars |
Query exemplars |
query, start, end |
Metadata & Status Endpoints
| Endpoint |
Purpose |
/api/v1/targets |
Target discovery status (state=active|dropped|any) |
/api/v1/targets/metadata |
Metric metadata from targets |
/api/v1/metadata |
All metric metadata |
/api/v1/rules |
Alerting/recording rules |
/api/v1/alerts |
Active alerts |
/api/v1/alertmanagers |
Alertmanager discovery |
/api/v1/status/config |
Current config YAML |
/api/v1/status/flags |
CLI flags |
/api/v1/status/runtimeinfo |
Runtime info |
/api/v1/status/buildinfo |
Build info |
/api/v1/status/tsdb |
TSDB cardinality stats |
/api/v1/status/walreplay |
WAL replay progress |
Admin Endpoints (require --web.enable-admin-api)
| Endpoint |
Method |
Purpose |
/api/v1/admin/tsdb/snapshot |
POST |
Create TSDB snapshot |
/api/v1/admin/tsdb/delete_series |
POST |
Delete series (match[], start, end) |
/api/v1/admin/tsdb/clean_tombstones |
POST |
Clean deleted data |
Common PromQL Patterns
# Rate of counter over 5m
rate(http_requests_total[5m])
# Sum by label
sum by (job) (rate(http_requests_total[5m]))
# Percentile from histogram
histogram_quantile(0.95, rate(http_request_duration_seconds_bucket[5m]))
# Filter by label
up{job="prometheus", instance=~".*:9090"}
# Increase over time
increase(http_requests_total[1h])
# Average over time range
avg_over_time(process_cpu_seconds_total[5m])
Result Types
- vector:
[{"metric": {...}, "value": [timestamp, "value"]}]
- matrix:
[{"metric": {...}, "values": [[ts, "val"], ...]}]
- scalar:
[timestamp, "value"]
- string:
[timestamp, "string"]
Scripts
Query script: scripts/prom_query.py
# Instant query
python scripts/prom_query.py http://localhost:9090 'up'
# Range query
python scripts/prom_query.py http://localhost:9090 'rate(http_requests_total[5m])' \
--start '2024-01-01T00:00:00Z' --end '2024-01-01T01:00:00Z' --step '1m'
# Output: table, json, csv
python scripts/prom_query.py http://localhost:9090 'up' --format table
Health check: scripts/prom_health.py
python scripts/prom_health.py http://localhost:9090
Detailed Reference
For complete API documentation: references/api_reference.md
For PromQL functions: references/promql_functions.md
Gotchas
rate() over a counter that resets too often: math is correct but meaningless — use increase() and divide by interval explicitly when counters don't survive scrapes.up{} per-target gauge: a flaky target shows up=0 but doesn't trigger alerts unless for is met. Set short for for liveness, long for noise.- Recording rules evaluate at fixed interval; missed evaluations don't backfill — gaps in the recording series during incidents.
- Federation
match[] parameter requires ALL matchers to match — an empty matcher returns no series, which looks like a working query with no data.
- Stale-marker semantics: a series stops being scraped → stale marker after 5 min by default → queries see "no data" not "0". Affects alerts on
absent().
- Service Discovery + relabel_config: a bad regex in
keep action silently drops all targets — verify with /api/v1/targets after each config change.
1---2name: prometheus3description: Query and interact with Prometheus HTTP API for monitoring data. Use when Claude needs to query Prometheus metrics, execute PromQL queries, retrieve targets/alerts/rules status, access metadata about series/labels, manage TSDB operations, or troubleshoot monitoring infrastructure. Supports instant queries, range queries, metadata endpoints, admin APIs, and alerting information.4---5
6# Prometheus API Skill
7
8Query Prometheus monitoring systems via HTTP API at `/api/v1`.
9
10## Quick Reference
11
12### Instant Query
13
14```bash
15curl 'http://<prometheus>:9090/api/v1/query?query=<promql>&time=<timestamp>'
16```
17
18### Range Query
19
20```bash
21curl 'http://<prometheus>:9090/api/v1/query_range?query=<promql>&start=<ts>&end=<ts>&step=<duration>'
22```
23
24## Response Format
25
26All responses return JSON:
27
28```json
29{
30 "status": "success" | "error",
31 "data": <result>,
32 "errorType": "<string>",
33 "error": "<string>",
34 "warnings": ["<string>"]
35}
36```
37
38HTTP codes: `400` (bad params), `422` (expression error), `503` (timeout).
39
40## Query Endpoints
41
42| Endpoint | Purpose | Key Parameters |
43|----------|---------|----------------|
44| `/api/v1/query` | Instant query | `query`, `time`, `timeout`, `limit` |
45| `/api/v1/query_range` | Range query | `query`, `start`, `end`, `step`, `timeout`, `limit` |
46| `/api/v1/format_query` | Format PromQL | `query` |
47| `/api/v1/series` | Find series by labels | `match[]`, `start`, `end`, `limit` |
48| `/api/v1/labels` | List label names | `start`, `end`, `match[]`, `limit` |
49| `/api/v1/label/<name>/values` | Label values | `start`, `end`, `match[]`, `limit` |
50| `/api/v1/query_exemplars` | Query exemplars | `query`, `start`, `end` |
51
52## Metadata & Status Endpoints
53
54| Endpoint | Purpose |
55|----------|---------|
56| `/api/v1/targets` | Target discovery status (`state=active\|dropped\|any`) |
57| `/api/v1/targets/metadata` | Metric metadata from targets |
58| `/api/v1/metadata` | All metric metadata |
59| `/api/v1/rules` | Alerting/recording rules |
60| `/api/v1/alerts` | Active alerts |
61| `/api/v1/alertmanagers` | Alertmanager discovery |
62| `/api/v1/status/config` | Current config YAML |
63| `/api/v1/status/flags` | CLI flags |
64| `/api/v1/status/runtimeinfo` | Runtime info |
65| `/api/v1/status/buildinfo` | Build info |
66| `/api/v1/status/tsdb` | TSDB cardinality stats |
67| `/api/v1/status/walreplay` | WAL replay progress |
68
69## Admin Endpoints (require `--web.enable-admin-api`)
70
71| Endpoint | Method | Purpose |
72|----------|--------|---------|
73| `/api/v1/admin/tsdb/snapshot` | POST | Create TSDB snapshot |
74| `/api/v1/admin/tsdb/delete_series` | POST | Delete series (`match[]`, `start`, `end`) |
75| `/api/v1/admin/tsdb/clean_tombstones` | POST | Clean deleted data |
76
77## Common PromQL Patterns
78
79```promql
80# Rate of counter over 5m
81rate(http_requests_total[5m])
82
83# Sum by label
84sum by (job) (rate(http_requests_total[5m]))
85
86# Percentile from histogram
87histogram_quantile(0.95, rate(http_request_duration_seconds_bucket[5m]))
88
89# Filter by label
90up{job="prometheus", instance=~".*:9090"}
91
92# Increase over time
93increase(http_requests_total[1h])
94
95# Average over time range
96avg_over_time(process_cpu_seconds_total[5m])
97```
98
99## Result Types
100
101- **vector**: `[{"metric": {...}, "value": [timestamp, "value"]}]`
102- **matrix**: `[{"metric": {...}, "values": [[ts, "val"], ...]}]`
103- **scalar**: `[timestamp, "value"]`
104- **string**: `[timestamp, "string"]`
105
106## Scripts
107
108Query script: `scripts/prom_query.py`
109
110```bash
111# Instant query
112python scripts/prom_query.py http://localhost:9090 'up'
113
114# Range query
115python scripts/prom_query.py http://localhost:9090 'rate(http_requests_total[5m])' \
116 --start '2024-01-01T00:00:00Z' --end '2024-01-01T01:00:00Z' --step '1m'
117
118# Output: table, json, csv
119python scripts/prom_query.py http://localhost:9090 'up' --format table
120```
121
122Health check: `scripts/prom_health.py`
123
124```bash
125python scripts/prom_health.py http://localhost:9090
126```
127
128## Detailed Reference
129
130For complete API documentation: [references/api_reference.md](references/api_reference.md)
131
132For PromQL functions: [references/promql_functions.md](references/promql_functions.md)
133
134---
135
136## Gotchas
137
138- **`rate()` over a counter that resets too often: math is correct but meaningless** — use `increase()` and divide by interval explicitly when counters don't survive scrapes.
139- **`up{}` per-target gauge**: a flaky target shows up=0 but doesn't trigger alerts unless `for` is met. Set short `for` for liveness, long for noise.
140- **Recording rules evaluate at fixed interval**; missed evaluations don't backfill — gaps in the recording series during incidents.
141- **Federation `match[]` parameter requires ALL matchers to match** — an empty matcher returns no series, which looks like a working query with no data.
142- **Stale-marker semantics**: a series stops being scraped → stale marker after 5 min by default → queries see "no data" not "0". Affects alerts on `absent()`.
143- **Service Discovery + relabel_config**: a bad regex in `keep` action silently drops all targets — verify with `/api/v1/targets` after each config change.
julianobarbosa