Query Metrics

Querying Axiom Metrics

All script paths are relative to this skill's folder; invoke as scripts/<name>. The target dataset must be of kind otel:metrics:v1.

Setup, prerequisites, and ~/.axiom.toml configuration: see README.md. Edge-deployment routing is automatic — the scripts read each dataset's edgeDeployment and route to the right regional endpoint without configuration.

Workflow

1. scripts/datasets <deploy> --kind otel:metrics:v1 — list metrics datasets.
2. scripts/metrics-spec — required before composing any query. MPL evolves; the spec is the source of truth. Also use it to answer general MPL/metrics questions.
3. scripts/metrics-info <deploy> <dataset> metrics — list metrics with {type, temporality, unit} metadata. Read this before writing the query (see Choosing a Query Shape).
4. scripts/metrics-info <deploy> <dataset> tags [<tag> values] — explore filter dimensions.
5. scripts/metrics-query <deploy> '<MPL>' <start> <end> — execute. Iterate.

If the user names a specific entity (service, host, …), scripts/metrics-info <deploy> <dataset> find-metrics "<value>" finds the metrics carrying it. find-metrics searches tag values, not metric names — don't use it for general discovery.

Choosing a Query Shape

The metrics-info listing returns each metric's {type, temporality, unit}. Read these before composing — never assume a metric is a simple scalar.

Rules per type (consult metrics-spec for exact operator names — they evolve):

• Gauge — instantaneous value. Align directly with avg/min/max/sum. Don't apply a rate; you'd be averaging meaningless deltas of an instantaneous value.
• CounterMonotonic + Cumulative — running total (resets aside). The raw values are rarely what you want. Convert to a per-second rate first, then align/aggregate.
• CounterMonotonic + Delta — already per-interval. Sum/align without a rate step.
• CounterNonMonotonic — can go up or down (queue depth, balance). Intent is ambiguous: rate, delta, or current value all make sense for different questions. Ask the user before picking one.
• Histogram — not a scalar. align using avg produces nonsense. Use bucket … using with the histogram functions from metrics-spec; quantiles are float specs to those functions, and temporality selects the variant (Cumulative vs Delta interpolation). Consult metrics-spec for the exact signatures.
• temporality: null — "not applicable for this instrument type" (the norm for Gauges), not "missing data".

When surfacing numbers, attach the unit (treat null as unitless). If you combine metrics with mismatched units in arithmetic, warn rather than silently producing a meaningless number.

Query Metrics

scripts/metrics-query [-w pixels] [--pixel-per-point n] <deploy> '<MPL>' <start> <end>

Always single-quote the MPL string in the shell. MPL is full of backticks; inside double quotes the shell executes them as command substitution, silently mangling the query (or running whatever the identifier names).

Bound the output before grouping. group by <tag> returns one series per tag value with no cap — on a high-cardinality tag this floods the output. Check cardinality first (describe, or tags <tag> values) and prefer plain group using <agg> while exploring.

Examples:

scripts/metrics-query prod -w 1200 \
  '`my-dataset`:`http.server.duration` | align to $__interval using avg' \
  now-1h now

scripts/metrics-query prod -w 1200 \
  '`my-dataset`:`http.server.duration`
   | where `service.name` == "frontend" and method == "GET"
   | align to $__interval using avg
   | group by status_code using sum' \
  now-1d now

Adaptive resolution ($__interval)

Hardcoding a step (align to 5m) makes charts look wrong at other zoom levels — too sparse zoomed in, too dense zoomed out. Prefer the system parameter $__interval wherever a Duration is expected, and pass the chart width so the server picks the step:

scripts/metrics-query prod -w 1200 \
  '`my-dataset`:`http.server.duration` | align to $__interval using avg' \
  now-7d now

The metrics service computes $__interval from the query's time range and the target chart width, then snaps it up to a nice resolution from the ladder 1s, 5s, 10s, 15s, 30s, 1m, 5m, 10m, 15m, 30m, 1h, 12h, 1d, 1w, 1M, 1Y. It never drops below a metric's stored resolution.

• No declaration needed — the server auto-registers $__interval; do not add param $__interval: Duration; (the edge forwards the query verbatim and the metrics service injects the parameter).
• Bucket count ≈ chart-width / pixel-per-point (pixel-per-point default 10). Omit -w and the server targets ~500 buckets.
• Works anywhere a Duration is valid, e.g. bucket to $__interval using histogram(0.5, 0.95).
• Set -w to your render width (e.g. the metrics-chart skill's plot width) so one bucket ≈ one pixel column. The value is forwarded under the request body's queryOptions (chart-width, pixel-per-point).

Parameters

MPL can declare parameters (param $svc: string;). Pass values with repeated -p name=value. The script applies the API's param__ prefix; values are forwarded verbatim as MPL literals (string literals include their quotes).

scripts/metrics-query \
  -p svc='"frontend"' \
  -p window='5m' \
  prod \
  'param $svc: string; param $window: Duration;
   `otel-metrics`:`http.server.duration` | where `service.name` == $svc | align to $window using avg' \
  now-1h now

Required parameters must be supplied; optional ones may be omitted. Resulting request body shape:

{
  "apl": "param $svc: string; …",
  "startTime": "now-1h",
  "endTime": "now",
  "params": { "param__svc": "\"frontend\"", "param__window": "5m" }
}

Literal syntax per type lives in metrics-spec.

Discovery (metrics-info)

Time range defaults to the last 24h; override with --start / --end. Both accept RFC3339 (offsets allowed) or relative now / now-<N><unit> with <unit> in s m h d w, resolved to RFC3339 UTC client-side. This is narrower than metrics-query, which forwards times to the server unparsed and so also accepts forms like now-1y; in metrics-info anything outside now / now-<N>[smhdw] must already be RFC3339 or the request 400s.

Error Handling

HTTP errors return JSON with code and message; some include a detail object:

{"code": 400, "message": "MPL syntax error: …"}

Syntax errors (400) include an annotated source pointer listing the valid operators at the failure position — read it, it usually names the fix.

Requests time out client-side after 120s (AXIOM_MAX_TIME to override; AXIOM_CONNECT_TIMEOUT for the 10s connect timeout).

On 500, re-run with curl -v to capture the traceparent / x-axiom-trace-id header and report it — the trace ID is what the backend team needs to debug.

Scripts

Run any script without arguments for full usage.