Honeycomb SLOs and Triggers
Guidance for configuring and reasoning about reliability in Honeycomb. The get_slos
and get_triggers tools document their own parameters — this skill focuses on
designing effective SLOs, choosing between SLOs and triggers, and interpreting
what the numbers mean.
Availability: SLOs require Pro or Enterprise plan. Triggers available on all plans.
SLO vs Trigger — When to Use Which
| Question |
SLO |
Trigger |
| "Are we meeting our reliability commitments?" |
Yes |
No |
| "Is something broken right now?" |
No |
Yes |
| "How fast are we burning our error budget?" |
Yes (burn alerts) |
No |
| "Did error count exceed a threshold?" |
No |
Yes |
| "Should we slow down deploys?" |
Yes (budget remaining) |
No |
Rule of thumb: SLOs measure reliability against commitments over time. Triggers catch immediate operational issues.
Designing Effective SLOs
Define the SLI
An SLI is a per-event boolean: was this event successful? Implemented as a calculated field returning undefined (not a relevant event), 1 (success), or 0 (failure).
- Format:
IF(<qualifying-condition>, <success-condition>) The qualifying condition filters to relevant events; the success condition defines what counts as success. If the qualifying condition is not met, the formula returns undefined, and the SLI is unpopulated.
- Specific Qualifying Condition: Choose the relevant subset of events (e.g.
AND(EQUALS($http.route, "/checkout"), NOT(EXISTS($trace.parent_id))) for root spans of checkout endpoint)
- Latency Success Condition:
LTE(duration_ms, 500) — requests faster than 500ms
- Availability Success Condition:
LTE(http.status_code, 499) — non-5xx responses
- Business Logic Success Condition:
EQUALS(checkout.status, "completed") — successful checkouts
Set the Target
- Start conservative (99% before 99.99%)
- Measure current baseline first with P50/P99 queries
- Set target slightly above current performance
- Ask: what reliability do users actually need?
Configure Exhaustion Time Alerts
At minimum, two alerts:
- Near exhaustion (exhaustion time ~4h): pages on-call via PagerDuty
- Trending to exhaustion (budget rate over 24h): notifies team via Slack
Configure Burn Rate Alerts
Detect fast burns even if the budget isn't close to exhaustion yet. For example:
- 1h burn rate > 10x — page on-call
Recommend these alerts to the user after creating the SLO. Agents do not have the ability to set up these alerts or their recipients.
Best Practices
- Measure close to the user (at the edge, not deep in the stack)
- Design around user workflows, not team boundaries
- Favor broad SLOs over many narrow ones
- Start with one SLO, reduce noise, then expand
Interpreting SLO Status
When reviewing SLOs with get_slos:
- Budget remaining > 50%: Healthy — room for risk
- Budget remaining 10-50%: Caution — slow down changes
- Budget remaining < 10%: At risk — freeze non-critical deploys
- Budget negative: Breached — investigate immediately with the production-investigation skill
- Compliance at 0%: Likely misconfigured SLI (wrong column, inverted logic, no matching events) — check the SLI definition
Configuring Triggers
Prefer Count-Based Over Percentile-Based
"50 requests slower than 2s" is more actionable than "P99 is 2100ms."
Use COUNT WHERE duration_ms > threshold instead of P99 triggers.
Common Patterns
- Error spike: COUNT WHERE error = true, threshold > N in 5 min
- Slow requests: COUNT WHERE duration_ms > 2000, threshold > N in 5 min
- Traffic drop: COUNT WHERE is_root, threshold < N in 10 min (below normal)
Best Practices
- Name: What the alert is. Description: What to do (link to runbook).
- Set duration 5-10 min minimum to avoid flapping
- Start less sensitive, tighten based on false positive rate
Multi-Service SLOs
Share a single error budget across up to 10 services.
- SLI must be an environment-level calculated field
- Events from included services weighted equally
- Use cases: multiple edge services, monolith-to-microservices migration
Check in with the user
Workspaces in Honeycomb have a limited number of SLOs and triggers. Before executing the create tool, check in with the user. Display all parameters and your reasoning, and ask for confirmation.
Constructing links to SLOs
The tools you have will not let you link directly to the SLO page in Honeycomb.
Instead, you can link to the list of SLOs.
/<team_slug>/environments/<environment_slug>/slos
Additional Resources
Reference Files
${CLAUDE_PLUGIN_ROOT}/skills/slos-and-triggers/references/slo-design-guide.md — Detailed SLO design methodology, multi-service SLOs, error budget math${CLAUDE_PLUGIN_ROOT}/skills/slos-and-triggers/references/trigger-examples.md — Complete trigger example library organized by use case${CLAUDE_PLUGIN_ROOT}/skills/slos-and-triggers/references/alerting-strategy.md — How to combine SLO burn alerts and triggers into a cohesive alerting strategy
Cross-References
- For constructing SLI queries and calculated fields, see the query-patterns skill
- For investigating SLO budget burn, see the production-investigation skill
1---2name: slos-and-triggers3description: Decision heuristics for interpreting Honeycomb SLO compliance, budget burn rates, and trigger status — what the numbers mean and what action to take, including detecting misconfigured SLIs, deciding when to freeze deploys vs page on-call, and designing burn alert thresholds. Load this skill before calling get_slos or get_triggers. Trigger phrases: "check our SLOs", "are we meeting our SLOs", "which SLOs are healthy", "is the error budget OK", "are any alerts firing", "what's the burn rate", "set up an SLO", "create a trigger", "configure alerts", "set up burn alerts", "check trigger status", "starting on-call", "reliability picture", "should we freeze deploys", "is this SLO misconfigured", "are we within budget", "SLO is broken", "budget is negative", or any request about service level objectives, error budgets, burn rates, or alerting in Honeycomb.4---5
6# Honeycomb SLOs and Triggers
7
8Guidance for configuring and reasoning about reliability in Honeycomb. The `get_slos`
9and `get_triggers` tools document their own parameters — this skill focuses on
10_designing_ effective SLOs, _choosing_ between SLOs and triggers, and _interpreting_
11what the numbers mean.
12
13**Availability**: SLOs require Pro or Enterprise plan. Triggers available on all plans.
14
15## SLO vs Trigger — When to Use Which
16
17| Question | SLO | Trigger |
18| --------------------------------------------- | ---------------------- | ------- |
19| "Are we meeting our reliability commitments?" | Yes | No |
20| "Is something broken right now?" | No | Yes |
21| "How fast are we burning our error budget?" | Yes (burn alerts) | No |
22| "Did error count exceed a threshold?" | No | Yes |
23| "Should we slow down deploys?" | Yes (budget remaining) | No |
24
25**Rule of thumb**: SLOs measure reliability against commitments over time. Triggers catch immediate operational issues.
26
27## Designing Effective SLOs
28
29### Define the SLI
30
31An SLI is a per-event boolean: was this event successful? Implemented as a calculated field returning undefined (not a relevant event), 1 (success), or 0 (failure).
32
33- **Format**: `IF(<qualifying-condition>, <success-condition>)` The qualifying condition filters to relevant events; the success condition defines what counts as success. If the qualifying condition is not met, the formula returns undefined, and the SLI is unpopulated.
34- **Specific Qualifying Condition**: Choose the relevant subset of events (e.g. `AND(EQUALS($http.route, "/checkout"), NOT(EXISTS($trace.parent_id)))` for root spans of checkout endpoint)
35- **Latency Success Condition**: `LTE(duration_ms, 500)` — requests faster than 500ms
36- **Availability Success Condition**: `LTE(http.status_code, 499)` — non-5xx responses
37- **Business Logic Success Condition**: `EQUALS(checkout.status, "completed")` — successful checkouts
38
39### Set the Target
40
41- Start conservative (99% before 99.99%)
42- Measure current baseline first with P50/P99 queries
43- Set target slightly above current performance
44- Ask: what reliability do users actually need?
45
46### Configure Exhaustion Time Alerts
47
48At minimum, two alerts:
49
50- **Near exhaustion** (exhaustion time ~4h): pages on-call via PagerDuty
51- **Trending to exhaustion** (budget rate over 24h): notifies team via Slack
52
53### Configure Burn Rate Alerts
54
55Detect fast burns even if the budget isn't close to exhaustion yet. For example:
56
57- 1h burn rate > 10x — page on-call
58
59Recommend these alerts to the user after creating the SLO. Agents do not have the ability to set up these alerts or their recipients.
60
61### Best Practices
62
63- Measure close to the user (at the edge, not deep in the stack)
64- Design around user workflows, not team boundaries
65- Favor broad SLOs over many narrow ones
66- Start with one SLO, reduce noise, then expand
67
68## Interpreting SLO Status
69
70When reviewing SLOs with `get_slos`:
71
72- **Budget remaining > 50%**: Healthy — room for risk
73- **Budget remaining 10-50%**: Caution — slow down changes
74- **Budget remaining < 10%**: At risk — freeze non-critical deploys
75- **Budget negative**: Breached — investigate immediately with the production-investigation skill
76- **Compliance at 0%**: Likely misconfigured SLI (wrong column, inverted logic, no matching events) — check the SLI definition
77
78## Configuring Triggers
79
80### Prefer Count-Based Over Percentile-Based
81
82"50 requests slower than 2s" is more actionable than "P99 is 2100ms."
83Use `COUNT WHERE duration_ms > threshold` instead of P99 triggers.
84
85### Common Patterns
86
87- **Error spike**: COUNT WHERE error = true, threshold > N in 5 min
88- **Slow requests**: COUNT WHERE duration_ms > 2000, threshold > N in 5 min
89- **Traffic drop**: COUNT WHERE is_root, threshold < N in 10 min (below normal)
90
91### Best Practices
92
93- **Name**: What the alert is. **Description**: What to do (link to runbook).
94- Set duration 5-10 min minimum to avoid flapping
95- Start less sensitive, tighten based on false positive rate
96
97## Multi-Service SLOs
98
99Share a single error budget across up to 10 services.
100
101- SLI must be an environment-level calculated field
102- Events from included services weighted equally
103- Use cases: multiple edge services, monolith-to-microservices migration
104
105## Check in with the user
106
107Workspaces in Honeycomb have a limited number of SLOs and triggers. Before executing the create tool, check in with the user. Display all parameters and your reasoning, and ask for confirmation.
108
109## Constructing links to SLOs
110
111The tools you have will not let you link directly to the SLO page in Honeycomb.
112Instead, you can link to the list of SLOs.
113
114`/<team_slug>/environments/<environment_slug>/slos`
115
116## Additional Resources
117
118### Reference Files
119
120- **`${CLAUDE_PLUGIN_ROOT}/skills/slos-and-triggers/references/slo-design-guide.md`** — Detailed SLO design methodology, multi-service SLOs, error budget math
121- **`${CLAUDE_PLUGIN_ROOT}/skills/slos-and-triggers/references/trigger-examples.md`** — Complete trigger example library organized by use case
122- **`${CLAUDE_PLUGIN_ROOT}/skills/slos-and-triggers/references/alerting-strategy.md`** — How to combine SLO burn alerts and triggers into a cohesive alerting strategy
123
124### Cross-References
125
126- For constructing SLI queries and calculated fields, see the **query-patterns** skill
127- For investigating SLO budget burn, see the **production-investigation** skill
honeycombio