Bsamiee Parametric Portal Logql Generator

[H1][LOGQL-GENERATOR]

Dictum: Pipeline order determines query performance.

Generate LogQL for Loki 3.0-3.6 (bloom filters, structured metadata, approx_topk). Cross-references: observability-stack for Loki deployment, promql-generator for PromQL metric queries.

Tasks:

1. Goal -- Error analysis, performance, security, debugging? Dashboard, alert, ad-hoc?
2. Sources -- Labels (job, namespace, app), log format (JSON/logfmt/plain), time range.
3. Query type -- Log query (return lines) or metric query (calculate values)?
4. Plan -- Plain-English plan, confirm with user before generating.
5. Generate -- Apply patterns below; consult references/best_practices.md.
6. Deliver -- Final query + explanation + usage context (Grafana panel, alert rule, logcli, HTTP API).

[1][VERSION_MATRIX]

Dictum: Version awareness prevents deprecated patterns.

Guidance:

• Promtail EOL March 2, 2026 -- migrate to Alloy.
• BoltDB deprecated -- use TSDB with v13 schema.

[2][PIPELINE_ORDER]

Dictum: Each stage filters BEFORE the next -- moving expensive operations earlier wastes resources.

{stream} -> line filter -> decolorize -> struct metadata -> parser -> label filter -> keep/drop -> format -> aggregate
 cheapest                                                                                              most expensive

Guidance:

• Structured metadata filters BEFORE parsers enables bloom filter acceleration (3.3+).
• Line filters (|=, !=) are O(1) substring checks -- place before parsers.
• Pattern match |> / !> uses <_> wildcards, 10x faster than regex (3.0+).

Best-Practices:

• Extract only needed JSON fields: | json level, status not | json -- reduces allocations (3.6).
• | decolorize before | logfmt -- ANSI codes break key=value parsing.
• | drop instance, pod before aggregation -- reduces series cardinality.

[3][STREAM_AND_FILTERS]

Dictum: Specific stream selectors minimize chunk scanning.

[4][PARSERS]

Dictum: Parser selection affects per-line cost.

[5][AGGREGATIONS]

Dictum: Metric queries aggregate log entries into time series.

[5.1][LOG_RANGE]

[5.2][UNWRAPPED_RANGE]

Operators: sum, avg, min, max, count, stddev, topk, bottomk, approx_topk, sort, sort_desc. Grouping: sum by (label1, label2) (...) or sum without (label1) (...).

[5.3][APPROX_TOPK]

approx_topk(k, expr) -- probabilistic top-K via count-min sketch, instant queries only. Requires limits_config.shard_aggregations: [approx_topk] on querier.

[6][STRUCTURED_METADATA]

Dictum: Structured metadata enables high-cardinality filtering without index impact.

Filter AFTER stream selector, BEFORE parsers for bloom acceleration. NOT indexed -- no cardinality impact.

{app="api"} | trace_id="abc123" | json | level="error"     # CORRECT -- bloom accelerated
# WRONG: {app="api", trace_id="abc123"}                     # trace_id is NOT an indexed label

Guidance:

• Bloom filters (3.3+) provide O(1) chunk skipping for string equality and OR filters placed before parsers.
• Automatic labels: service_name (from container/OTel), detected_level (when discover_log_levels: true).
• vector(0) fallback in alerting prevents "no data" flapping on sparse logs.

[7][METRIC_PATTERNS]

Dictum: Metric queries are pre-aggregated -- prefer for dashboards and alerts.

[8][NON-EXISTENT_FEATURES]

Dictum: Generating non-existent operators wastes user time.

[9][ALLOY_PIPELINE_STAGES]

Dictum: Alloy pipeline stages shape labels and metadata arriving in Loki.

[REFERENCE] infrastructure/src/deploy.ts lines 29-36.

[10][RESOURCES]

Dictum: Reference files provide copy-paste patterns and performance guidance.

• examples/log_queries.logql -- log parsing, filtering, structured metadata, template functions.
• examples/metric_queries.logql -- aggregation, alerting, rate/counter functions, label operations.
• references/best_practices.md -- performance, anti-patterns, recording rules.
• context7 MCP (grafana loki) -- authoritative docs for unclear syntax.

Converted and distributed by TomeVault — claim your Tome and manage your conversions.