Prometheus Metrics Specialist
Compact skill for instrumenting HuleEdu services with Prometheus metrics and querying them with PromQL.
When to Use
Activate when the user:
- Needs to add metrics instrumentation to a service
- Wants to write PromQL queries for dashboards or alerts
- Asks about metrics naming conventions
- Needs help with metrics middleware setup
- Wants to understand business vs operational metrics
- Mentions Prometheus, PromQL, metrics, instrumentation, or monitoring
- Needs to expose
/metrics endpoint
I Need To...
Get Started (First Time)
- Learn Prometheus basics → fundamentals.md - Counter, Histogram, Gauge concepts
- Set up HuleEdu service → huleedu-patterns.md - metrics.py module, /metrics endpoint
- Pick an example → See "Add Metrics to Service" below
Add Metrics to Service
| I need to instrument... |
Read this file |
| HTTP endpoints |
examples/http-and-kafka.md |
| Kafka event processing |
examples/http-and-kafka.md |
| LLM API calls |
examples/llm-and-batch.md |
| Batch processing |
examples/llm-and-batch.md |
| Database queries |
examples/database-and-business.md |
| Business logic |
examples/database-and-business.md |
Choose Naming Pattern
→ huleedu-patterns.md § Naming Patterns
Write PromQL Queries
| I need to... |
Read this section |
| Learn PromQL syntax |
promql-guide.md § Basics |
| Calculate error rate |
promql-guide.md § Error Rates |
| Get P95 latency |
promql-guide.md § Percentiles |
| Troubleshoot issues |
promql-guide.md § Troubleshooting |
Reference
| What I need to know |
Where to find it |
| Metric types (Counter/Histogram/Gauge) |
fundamentals.md § Metric Types |
| Label design principles |
fundamentals.md § Labels |
| Histogram bucket design |
fundamentals.md § Buckets |
| HuleEdu architecture |
huleedu-patterns.md § Architecture |
Quick Metric Type Cheatsheet
# Counter: Events that only increase (resets to 0 on restart)
operations_total = Counter("spell_checker_operations_total", "desc", ["status"])
operations_total.labels(status="success").inc()
# Histogram: Distributions (latency, sizes, scores)
duration_seconds = Histogram("spell_checker_duration_seconds", "desc")
duration_seconds.observe(0.23) # 230ms
# Gauge: Point-in-time values that go up/down
active_connections = Gauge("spell_checker_active_connections", "desc")
active_connections.set(15)
active_connections.inc() # 16
HuleEdu Naming Cheatsheet
# Pattern 1: Service-Prefixed (most common, operational metrics)
spell_checker_operations_total
spell_checker_http_request_duration_seconds
# Pattern 2: Business Metrics (cross-service, business logic)
huleedu_llm_prompt_tokens_total
huleedu_spellcheck_corrections_made
# Pattern 3: Legacy (being phased out)
request_count
Decision: Service-specific metric? → Pattern 1. Cross-service business metric? → Pattern 2.
Core Capabilities
- Naming Conventions: 3 patterns (service-prefixed, business, legacy)
- Instrumentation: Counter, Histogram, Gauge, Summary usage
- Middleware Setup: Standard HTTP metrics middleware
- Service-Specific Metrics: Business logic patterns
- PromQL Queries: Troubleshooting and dashboard queries
- Scrape Configuration: Prometheus targets and discovery
- Best Practices: Metric types, cardinality, aggregation
- Context7 Integration: Latest Prometheus/PromQL docs
Documentation Structure
- fundamentals.md: Universal Prometheus concepts + prometheus_client API
- huleedu-patterns.md: HuleEdu naming, architecture, setup patterns
- promql-guide.md: PromQL syntax + HuleEdu query patterns
- examples/: Real-world instrumentation examples
1---2name: prometheus-metrics-specialist3description: Instrument services with Prometheus metrics and write PromQL queries. Guides HuleEdu naming conventions, metrics middleware setup, and business vs operational metrics. Integrates with Context7 for latest Prometheus documentation.4---5
6# Prometheus Metrics Specialist
7
8Compact skill for instrumenting HuleEdu services with Prometheus metrics and querying them with PromQL.
9
10## When to Use
11
12Activate when the user:
13- Needs to add metrics instrumentation to a service
14- Wants to write PromQL queries for dashboards or alerts
15- Asks about metrics naming conventions
16- Needs help with metrics middleware setup
17- Wants to understand business vs operational metrics
18- Mentions Prometheus, PromQL, metrics, instrumentation, or monitoring
19- Needs to expose `/metrics` endpoint
20
21## I Need To...
22
23### Get Started (First Time)
241. **Learn Prometheus basics** → [fundamentals.md](fundamentals.md) - Counter, Histogram, Gauge concepts
252. **Set up HuleEdu service** → [huleedu-patterns.md](huleedu-patterns.md) - metrics.py module, /metrics endpoint
263. **Pick an example** → See "Add Metrics to Service" below
27
28### Add Metrics to Service
29
30| I need to instrument... | Read this file |
31|------------------------|----------------|
32| **HTTP endpoints** | [examples/http-and-kafka.md](examples/http-and-kafka.md#http-instrumentation) |
33| **Kafka event processing** | [examples/http-and-kafka.md](examples/http-and-kafka.md#kafka-event-metrics) |
34| **LLM API calls** | [examples/llm-and-batch.md](examples/llm-and-batch.md#llm-provider-metrics) |
35| **Batch processing** | [examples/llm-and-batch.md](examples/llm-and-batch.md#batch-processing-metrics) |
36| **Database queries** | [examples/database-and-business.md](examples/database-and-business.md#database-metrics) |
37| **Business logic** | [examples/database-and-business.md](examples/database-and-business.md#business-logic-metrics) |
38
39### Choose Naming Pattern
40→ [huleedu-patterns.md § Naming Patterns](huleedu-patterns.md#naming-patterns)
41
42### Write PromQL Queries
43
44| I need to... | Read this section |
45|--------------|-------------------|
46| **Learn PromQL syntax** | [promql-guide.md § Basics](promql-guide.md#promql-basics) |
47| **Calculate error rate** | [promql-guide.md § Error Rates](promql-guide.md#error-rate-patterns) |
48| **Get P95 latency** | [promql-guide.md § Percentiles](promql-guide.md#percentile-queries) |
49| **Troubleshoot issues** | [promql-guide.md § Troubleshooting](promql-guide.md#troubleshooting-patterns) |
50
51### Reference
52
53| What I need to know | Where to find it |
54|---------------------|------------------|
55| **Metric types (Counter/Histogram/Gauge)** | [fundamentals.md § Metric Types](fundamentals.md#metric-types) |
56| **Label design principles** | [fundamentals.md § Labels](fundamentals.md#label-design) |
57| **Histogram bucket design** | [fundamentals.md § Buckets](fundamentals.md#histogram-bucket-design) |
58| **HuleEdu architecture** | [huleedu-patterns.md § Architecture](huleedu-patterns.md#architecture) |
59
60## Quick Metric Type Cheatsheet
61
62```python
63# Counter: Events that only increase (resets to 0 on restart)
64operations_total = Counter("spell_checker_operations_total", "desc", ["status"])
65operations_total.labels(status="success").inc()
66
67# Histogram: Distributions (latency, sizes, scores)
68duration_seconds = Histogram("spell_checker_duration_seconds", "desc")
69duration_seconds.observe(0.23) # 230ms
70
71# Gauge: Point-in-time values that go up/down
72active_connections = Gauge("spell_checker_active_connections", "desc")
73active_connections.set(15)
74active_connections.inc() # 16
75```
76
77## HuleEdu Naming Cheatsheet
78
79```python
80# Pattern 1: Service-Prefixed (most common, operational metrics)
81spell_checker_operations_total
82spell_checker_http_request_duration_seconds
83
84# Pattern 2: Business Metrics (cross-service, business logic)
85huleedu_llm_prompt_tokens_total
86huleedu_spellcheck_corrections_made
87
88# Pattern 3: Legacy (being phased out)
89request_count
90```
91
92**Decision**: Service-specific metric? → Pattern 1. Cross-service business metric? → Pattern 2.
93
94## Core Capabilities
95
96- **Naming Conventions**: 3 patterns (service-prefixed, business, legacy)
97- **Instrumentation**: Counter, Histogram, Gauge, Summary usage
98- **Middleware Setup**: Standard HTTP metrics middleware
99- **Service-Specific Metrics**: Business logic patterns
100- **PromQL Queries**: Troubleshooting and dashboard queries
101- **Scrape Configuration**: Prometheus targets and discovery
102- **Best Practices**: Metric types, cardinality, aggregation
103- **Context7 Integration**: Latest Prometheus/PromQL docs
104
105## Documentation Structure
106
107- **[fundamentals.md](fundamentals.md)**: Universal Prometheus concepts + prometheus_client API
108- **[huleedu-patterns.md](huleedu-patterns.md)**: HuleEdu naming, architecture, setup patterns
109- **[promql-guide.md](promql-guide.md)**: PromQL syntax + HuleEdu query patterns
110- **[examples/](examples/)**: Real-world instrumentation examples
majiayu000