Weekly Production Review
Use this skill to produce a source-grounded production review across Datadog,
incident.io, and Linear. Keep the output factual, table-first, and easy to
audit from the linked source rows.
Scope
- Default "last week" to the previous Monday through Sunday in the user's
timezone. State both local and UTC query windows in one short scope line.
- Cover all production environments unless the user narrows scope:
prod-us, prod-eu, prod-hipaa, and prod-jp.
- The review itself changes nothing outside the tracker. Do not touch incident.io
records, follow-ups, alerts, monitors, files, Slack messages, or production
systems unless the user explicitly asks after reviewing the findings.
- Linear is the exception: evidence may be commented onto issues that already
exist, labelled and marked as agent-written. A new issue has no parent, so it
is the one write that asks first — show the set, take one go-ahead, then file
them.
linear-agent-writes is the authority;
linear-bug-triage applies it to measured
findings.
- For chat-only reviews, avoid creating report artifacts or local analysis
workspaces unless a required tool workflow explicitly does so or the user asks
for a file. If incident.io analysis tooling requires a local playbook
workspace, mention it only when relevant and keep production systems
unchanged.
- Write
No measurements found when a requested signal cannot be queried or
measured.
Related Skills
- Use
datadog-query-recipes for
production Datadog query shapes and environment/site routing.
- Use
linear-bug-triage for the Linear
write-back: it comments measured evidence onto existing issues, and files new
ones once you approve the set.
- Use
incident-alert-tickets to check
each alert cluster against the per-monitor knowledge base and record newly
root-caused clusters there as a labelled description edit.
Workflow
- Confirm the review window and timezone. If the user says "last week", use the
previous calendar week, not a rolling seven-day window.
- Gather public/customer-facing incidents from incident.io. Prefer incident.io
for accepted incidents, public incident visibility, follow-ups, and incident
status. Always produce the incident.io table below, even when no rows are
found.
- Gather incident.io alert load for the same review window. Prefer
incident.io alert or escalation stats for the primary Langfuse escalation
path/team. If the user provides an incident.io pager-load dashboard URL, use
its
escalation_path parameter as a scope hint; use the dashboard date range
only when the user explicitly scopes the review to that range instead of the
default weekly window. Group by paged engineer and incident.io time-of-day
bucket so the table shows working-hours, evening, and night load. Always
produce the incident.io alert load table below.
- Gather Linear bugs from the
bug label first. Include all bug-labeled
tickets created, updated, completed, or still open with production evidence
during the window. Inspect likely production bugs with issue details and
comments when status, owner, or evidence is unclear. Always produce the
Linear bug table below.
- Gather Datadog alert/page signals for the window. Use incident.io alerts or
escalations when they represent pages; use Datadog monitor/event data when
available. Build the exhaustive alert universe by paginating until no more
results remain for the window. Cover every production environment in scope.
Group repeated firings by monitor/page title or ID, environment, service/team,
and trigger reason.
- For every Datadog alert/page cluster, perform the deep dive before writing the
final row. Do not stop at the monitor title or count. Check the monitor's
incident-alert ticket first (see
incident-alert-tickets); a
documented cause section may explain the cluster — cite the ticket in the
incident.io / Linear Link column. Inspect matching APM
spans, representative traces, related logs, error records, exception details,
failed job logs, dependency spans, queue backlog/delay context, and monitor
time windows. Put the relevant trace/span evidence and relevant logs/errors
directly in the Datadog alerts table. Do not create a separate Datadog Issue
Deep Dives table.
- Gather Datadog error log patterns for the window. Use logs with
status:error, scope to production environments, and group by the clustered
message pattern plus service/env where available. Always produce the
Datadog logs table below. Preserve exact Datadog patterns, including wildcard
tokens, instead of paraphrasing them.
- Classify each incident, bug, alert, alert-load row, and log pattern. Separate
production breakage from self-hosted, internal-only, duplicate, canceled,
expected/test, staging/dev, monitor-noise, or unknown signals.
- Cross-reference source rows on a best-effort basis. Link Datadog rows to
matching incident.io incidents or Linear bugs when evidence supports the
relationship. Link incident.io and Linear rows back to Datadog evidence when
available. If a relationship is inferential, say so in the row.
Output Contract
Return one short scope line followed by exactly these five source tables, in
this order:
- incident.io
- incident.io Alert Load
- Linear Bugs
- Datadog Alerts
- Datadog Logs
Do not add an executive summary, narrative summary, event-centric view, summary
table, source synthesis table, or separate Datadog Issue Deep Dives table. Put
counts and classifications inside the source tables.
If a table has no rows, keep the table heading and write one row or sentence
with No rows found or No measurements found plus the scoped source/query.
If a row is unclear, classify it as unclear or unknown/no measurements
instead of dropping it.
Cross-Source Linking
Keep incident.io incidents, incident.io alert load, Linear, Datadog alerts, and
Datadog logs as separate output tables. Use links inside each table to show
relationships instead of synthesizing a separate cross-source table.
Use Linear as the source of truth for deduplication across weeks and workflows.
Before reporting a bug, security finding, cost concern, or alert as new, search
Linear for matching issue keys, titles, source URLs, and comments — covering
both the bug label set and the incident-alert label set. If an
existing issue covers it, link to that issue and mark the row as already
tracked instead of reporting it again as fresh work.
Use short stable link labels:
Datadog monitor: <monitor name>
Datadog logs: <env/service/symptom>
Datadog spans: <env/route/symptom>
Datadog trace: <trace id or route>
incident.io: <INC reference>
incident.io alert load: <escalation path or team>
Linear: <issue key>
Do not write links, create follow-ups, or update external systems unless the
user explicitly asks for changes after reviewing the report.
incident.io Table
Use this table for incident.io incidents with public or customer-facing impact
in the review window. Query incident.io with incident_list scoped to the
review window and relevant team when known. Include summary, roles,
custom_fields, timestamps, durations, and escalation_urgency when the
tool supports them. If the tool cannot filter by visibility server-side, list
the scoped incidents and include only rows where visibility is public or
the evidence supports customer-facing impact.
| Incident |
Severity / Status |
Start / End / Duration |
Impact |
Linked Sources |
Follow-ups / Notes |
Column rules:
Incident: incident.io reference linked to the incident.
Severity / Status: severity and current lifecycle status.
Start / End / Duration: reported, identified, resolved, and duration when
available.
Impact: short impact statement grounded in the incident summary.
Linked Sources: Datadog alerts/pages, Datadog logs/spans, Linear issues, or
none found.
Follow-ups / Notes: follow-up count/status or none found.
incident.io Alert Load Table
Use this table for incident.io alert and pager load in the review window.
Prefer incident.io alert or escalation stats filtered to the Langfuse escalation
path or team. If the user provides a pager-load dashboard URL, parse and apply
the escalation_path[one_of] filter when available. Count alerts when the
source returns alert counts; otherwise count escalations/pages and label the
count source in Source / Notes.
incident.io time-of-day buckets are UTC:
working_hours: 09:00-18:00 Monday-Friday.
late_evening: 18:00-23:00 any day plus weekend daytime.
overnight: 23:00-09:00.
Render one row per paged engineer, sorted by total descending, and include a
final All engineers row when measurements exist. If user identity is missing,
use Unassigned / no responder. Do not collapse this table into a narrative
summary.
| Engineer |
Working Hours |
Late Evening |
Overnight |
Total Alerts / Pages |
Share |
Source / Notes |
Column rules:
Engineer: paged user or escalation target. Use the engineer's display name
when available.
Working Hours: count in the working_hours bucket.
Late Evening: count in the late_evening bucket.
Overnight: count in the overnight bucket.
Total Alerts / Pages: row total. Label pages versus alerts in
Source / Notes when the source does not expose alert counts directly.
Share: row total divided by the measured alert/page total.
Source / Notes: source query, escalation path/team filter, dashboard link,
or No measurements found.
Linear Bugs Table
Start from all Linear tickets with the bug label that were touched by the
window. Do not rely only on text searches for prod, incident, or Datadog;
those searches are useful for enrichment but are not the source universe. This
table is the Linear source inventory. A Linear bug can be classified as
non-production, duplicate, canceled, or no-action.
| Linear |
Title |
Summary |
Owner |
Status |
Touched Last Week Because |
Production Evidence |
Classification |
Counted? |
Column rules:
Linear: issue key linked to Linear, such as LFE-123.
Title: Linear issue title in a separate column.
Summary: one operational sentence based on the issue body, comments, and
evidence. Avoid fix guesses.
Owner: assignee if present; otherwise owning team if clear; otherwise
Unassigned.
Status: Linear status or state, plus completion timing when useful.
Touched Last Week Because: created, updated, completed, or
open production bug.
Production Evidence: prod env, customer impact, incident.io incident,
Datadog link, measured logs/spans/errors, or No measurements found.
Classification: use production/customer-impacting, internal-only,
self-hosted, staging/dev, duplicate/canceled/no-action, or unclear.
Counted?: yes only when the bug label and production/customer-impacting
evidence support including it in fixed/open production bug counts.
Datadog Alerts Table
Use this table for every production Datadog alert/page cluster found in the full
paginated alert/event pass, including clusters later classified as
expected/test, monitor noise, or unknown/no measurements.
| Monitor / Page Signal |
Env / Service |
Count / Window |
Why It Alerted |
Trace / Span Evidence |
Relevant Logs / Errors |
Verdict |
incident.io / Linear Link |
Column rules:
Monitor / Page Signal: monitor/page title or stable ID.
Env / Service: production env and service/team labels.
Count / Window: grouped firing/page count and relevant time window.
Why It Alerted: monitor threshold, trigger condition, route, queue, status
code, latency, or backlog signal.
Trace / Span Evidence: representative trace/span links, error counts,
latency, status codes, dependency spans, or No measurements found.
Relevant Logs / Errors: explicit exception class/message, exact or
normalized log message, failed job IDs when visible, DB/downstream errors,
retry exhaustion, validation failures, or No measurements found.
Verdict: use customer incident, confirmed bug, infra/dependency,
expected/test, monitor noise, or unknown/no measurements.
incident.io / Linear Link: matching incident.io reference, Linear issue key,
explicit disposition, or none found.
For API route and queue consumer errors, start from APM spans matching:
operation_name:(http.server OR bullmq.consumer) status:error env:<prod-env>
Then narrow by service, resource_name, route, queue, consumer, status code,
error type, or monitor time window. For failed trace samples, explicitly query
related Datadog logs and error records using the trace ID, span ID, service,
resource name, environment, and same time window. If no related logs or error
records are found, write No measurements found.
Before finalizing, compare the final Datadog alerts table against the full
paginated alert/event sweep. Confirm every production monitor title seen during
the window appears in the table or is explicitly excluded as non-prod.
Datadog Logs Table
Use this table for the most frequent production error-log patterns from the
review window. This is a broad log-health pass and is separate from the alert
cluster rows above, though rows should cross-link when possible.
Start from a Datadog Logs query shaped like:
status:error
Scope it to the review window and production environments in scope. Prefer the
Datadog pattern/clustering view using the log message field as the clustering
pattern field. Group or facet by service, env, and status when available,
and sort by count descending. Review at least the top 10 patterns overall, plus
any additional top pattern per production environment when the global top 10 is
dominated by one env or service.
| Exact Log Pattern |
Env / Service |
Count / Share |
Representative Error |
Related Signal / Link |
Disposition |
Column rules:
Exact Log Pattern: exact Datadog clustered pattern or exact raw log message.
Preserve Datadog wildcard syntax such as [wildcard]...[/wildcard]. Do not
paraphrase this column.
Env / Service: affected production envs and services.
Count / Share: count in the review window and share if available.
Representative Error: one exact short representative message, exception
class, or stack/log summary. Avoid long stack traces.
Related Signal / Link: related Datadog alert row, trace, log query,
incident.io incident, Linear issue, or none found.
Disposition: use known incident, tracked bug, needs investigation,
expected/test, monitor noise, or unknown.
If a high-volume pattern maps to a failed API route or queue consumer, ensure
the matching Datadog alert row includes the trace/log/error investigation. If a
high-volume pattern has no alert/page row, keep it in this table anyway and
mark it needs investigation or unknown based on evidence.
Output Format
Return valid Markdown only.
- Use valid Markdown syntax for headings, links, and tables.
- Include a space after list markers such as
-, *, and 1..
- Close links and parentheses correctly.
- Do not emit malformed tables, dangling backticks, or partially opened code
fences.
- Escape table pipes inside log patterns or error messages when needed.
- If a section would be fragile to format, prefer a plain paragraph over broken
Markdown.
1---
2name: weekly-production-review
3description: |
4 Prepare Langfuse weekly production reviews covering failures, fixes, open
5 issues, and tracking gaps. Use for "what broke last week," production bugs,
6 Datadog alerts or error patterns, incident.io activity, or pager load.
7metadata:
8 source: https://skillmd.com/skills/langfuse/weekly-production-review
9---
10
11# Weekly Production Review
12
13Use this skill to produce a source-grounded production review across Datadog,
14incident.io, and Linear. Keep the output factual, table-first, and easy to
15audit from the linked source rows.
16
17## Scope
18
19- Default "last week" to the previous Monday through Sunday in the user's
20 timezone. State both local and UTC query windows in one short scope line.
21- Cover all production environments unless the user narrows scope:
22 `prod-us`, `prod-eu`, `prod-hipaa`, and `prod-jp`.
23- The review itself changes nothing outside the tracker. Do not touch incident.io
24 records, follow-ups, alerts, monitors, files, Slack messages, or production
25 systems unless the user explicitly asks after reviewing the findings.
26- Linear is the exception: evidence may be commented onto issues that already
27 exist, labelled and marked as agent-written. A new issue has no parent, so it
28 is the one write that asks first — show the set, take one go-ahead, then file
29 them. [`linear-agent-writes`](../linear-agent-writes/SKILL.md) is the authority;
30 [`linear-bug-triage`](../linear-bug-triage/SKILL.md) applies it to measured
31 findings.
32- For chat-only reviews, avoid creating report artifacts or local analysis
33 workspaces unless a required tool workflow explicitly does so or the user asks
34 for a file. If incident.io analysis tooling requires a local playbook
35 workspace, mention it only when relevant and keep production systems
36 unchanged.
37- Write `No measurements found` when a requested signal cannot be queried or
38 measured.
39
40## Related Skills
41
42- Use [`datadog-query-recipes`](../datadog-query-recipes/SKILL.md) for
43 production Datadog query shapes and environment/site routing.
44- Use [`linear-bug-triage`](../linear-bug-triage/SKILL.md) for the Linear
45 write-back: it comments measured evidence onto existing issues, and files new
46 ones once you approve the set.
47- Use [`incident-alert-tickets`](../incident-alert-tickets/SKILL.md) to check
48 each alert cluster against the per-monitor knowledge base and record newly
49 root-caused clusters there as a labelled description edit.
50
51## Workflow
52
531. Confirm the review window and timezone. If the user says "last week", use the
54 previous calendar week, not a rolling seven-day window.
552. Gather public/customer-facing incidents from incident.io. Prefer incident.io
56 for accepted incidents, public incident visibility, follow-ups, and incident
57 status. Always produce the incident.io table below, even when no rows are
58 found.
593. Gather incident.io alert load for the same review window. Prefer
60 incident.io alert or escalation stats for the primary Langfuse escalation
61 path/team. If the user provides an incident.io pager-load dashboard URL, use
62 its `escalation_path` parameter as a scope hint; use the dashboard date range
63 only when the user explicitly scopes the review to that range instead of the
64 default weekly window. Group by paged engineer and incident.io time-of-day
65 bucket so the table shows working-hours, evening, and night load. Always
66 produce the incident.io alert load table below.
674. Gather Linear bugs from the `bug` label first. Include all `bug`-labeled
68 tickets created, updated, completed, or still open with production evidence
69 during the window. Inspect likely production bugs with issue details and
70 comments when status, owner, or evidence is unclear. Always produce the
71 Linear bug table below.
725. Gather Datadog alert/page signals for the window. Use incident.io alerts or
73 escalations when they represent pages; use Datadog monitor/event data when
74 available. Build the exhaustive alert universe by paginating until no more
75 results remain for the window. Cover every production environment in scope.
76 Group repeated firings by monitor/page title or ID, environment, service/team,
77 and trigger reason.
786. For every Datadog alert/page cluster, perform the deep dive before writing the
79 final row. Do not stop at the monitor title or count. Check the monitor's
80 `incident-alert` ticket first (see
81 [`incident-alert-tickets`](../incident-alert-tickets/SKILL.md)); a
82 documented cause section may explain the cluster — cite the ticket in the
83 `incident.io / Linear Link` column. Inspect matching APM
84 spans, representative traces, related logs, error records, exception details,
85 failed job logs, dependency spans, queue backlog/delay context, and monitor
86 time windows. Put the relevant trace/span evidence and relevant logs/errors
87 directly in the Datadog alerts table. Do not create a separate Datadog Issue
88 Deep Dives table.
897. Gather Datadog error log patterns for the window. Use logs with
90 `status:error`, scope to production environments, and group by the clustered
91 `message` pattern plus service/env where available. Always produce the
92 Datadog logs table below. Preserve exact Datadog patterns, including wildcard
93 tokens, instead of paraphrasing them.
948. Classify each incident, bug, alert, alert-load row, and log pattern. Separate
95 production breakage from self-hosted, internal-only, duplicate, canceled,
96 expected/test, staging/dev, monitor-noise, or unknown signals.
979. Cross-reference source rows on a best-effort basis. Link Datadog rows to
98 matching incident.io incidents or Linear bugs when evidence supports the
99 relationship. Link incident.io and Linear rows back to Datadog evidence when
100 available. If a relationship is inferential, say so in the row.
101
102## Output Contract
103
104Return one short scope line followed by exactly these five source tables, in
105this order:
106
1071. incident.io
1082. incident.io Alert Load
1093. Linear Bugs
1104. Datadog Alerts
1115. Datadog Logs
112
113Do not add an executive summary, narrative summary, event-centric view, summary
114table, source synthesis table, or separate Datadog Issue Deep Dives table. Put
115counts and classifications inside the source tables.
116
117If a table has no rows, keep the table heading and write one row or sentence
118with `No rows found` or `No measurements found` plus the scoped source/query.
119If a row is unclear, classify it as `unclear` or `unknown/no measurements`
120instead of dropping it.
121
122## Cross-Source Linking
123
124Keep incident.io incidents, incident.io alert load, Linear, Datadog alerts, and
125Datadog logs as separate output tables. Use links inside each table to show
126relationships instead of synthesizing a separate cross-source table.
127
128Use Linear as the source of truth for deduplication across weeks and workflows.
129Before reporting a bug, security finding, cost concern, or alert as new, search
130Linear for matching issue keys, titles, source URLs, and comments — covering
131both the `bug` label set and the `incident-alert` label set. If an
132existing issue covers it, link to that issue and mark the row as already
133tracked instead of reporting it again as fresh work.
134
135Use short stable link labels:
136
137- `Datadog monitor: <monitor name>`
138- `Datadog logs: <env/service/symptom>`
139- `Datadog spans: <env/route/symptom>`
140- `Datadog trace: <trace id or route>`
141- `incident.io: <INC reference>`
142- `incident.io alert load: <escalation path or team>`
143- `Linear: <issue key>`
144
145Do not write links, create follow-ups, or update external systems unless the
146user explicitly asks for changes after reviewing the report.
147
148## incident.io Table
149
150Use this table for incident.io incidents with public or customer-facing impact
151in the review window. Query incident.io with `incident_list` scoped to the
152review window and relevant team when known. Include `summary`, `roles`,
153`custom_fields`, `timestamps`, `durations`, and `escalation_urgency` when the
154tool supports them. If the tool cannot filter by visibility server-side, list
155the scoped incidents and include only rows where `visibility` is `public` or
156the evidence supports customer-facing impact.
157
158| Incident | Severity / Status | Start / End / Duration | Impact | Linked Sources | Follow-ups / Notes |
159| --- | --- | --- | --- | --- | --- |
160
161Column rules:
162
163- `Incident`: incident.io reference linked to the incident.
164- `Severity / Status`: severity and current lifecycle status.
165- `Start / End / Duration`: reported, identified, resolved, and duration when
166 available.
167- `Impact`: short impact statement grounded in the incident summary.
168- `Linked Sources`: Datadog alerts/pages, Datadog logs/spans, Linear issues, or
169 `none found`.
170- `Follow-ups / Notes`: follow-up count/status or `none found`.
171
172## incident.io Alert Load Table
173
174Use this table for incident.io alert and pager load in the review window.
175Prefer incident.io alert or escalation stats filtered to the Langfuse escalation
176path or team. If the user provides a pager-load dashboard URL, parse and apply
177the `escalation_path[one_of]` filter when available. Count alerts when the
178source returns alert counts; otherwise count escalations/pages and label the
179count source in `Source / Notes`.
180
181incident.io time-of-day buckets are UTC:
182
183- `working_hours`: 09:00-18:00 Monday-Friday.
184- `late_evening`: 18:00-23:00 any day plus weekend daytime.
185- `overnight`: 23:00-09:00.
186
187Render one row per paged engineer, sorted by total descending, and include a
188final `All engineers` row when measurements exist. If user identity is missing,
189use `Unassigned / no responder`. Do not collapse this table into a narrative
190summary.
191
192| Engineer | Working Hours | Late Evening | Overnight | Total Alerts / Pages | Share | Source / Notes |
193| --- | ---: | ---: | ---: | ---: | ---: | --- |
194
195Column rules:
196
197- `Engineer`: paged user or escalation target. Use the engineer's display name
198 when available.
199- `Working Hours`: count in the `working_hours` bucket.
200- `Late Evening`: count in the `late_evening` bucket.
201- `Overnight`: count in the `overnight` bucket.
202- `Total Alerts / Pages`: row total. Label pages versus alerts in
203 `Source / Notes` when the source does not expose alert counts directly.
204- `Share`: row total divided by the measured alert/page total.
205- `Source / Notes`: source query, escalation path/team filter, dashboard link,
206 or `No measurements found`.
207
208## Linear Bugs Table
209
210Start from all Linear tickets with the `bug` label that were touched by the
211window. Do not rely only on text searches for `prod`, `incident`, or `Datadog`;
212those searches are useful for enrichment but are not the source universe. This
213table is the Linear source inventory. A Linear bug can be classified as
214non-production, duplicate, canceled, or no-action.
215
216| Linear | Title | Summary | Owner | Status | Touched Last Week Because | Production Evidence | Classification | Counted? |
217| --- | --- | --- | --- | --- | --- | --- | --- | --- |
218
219Column rules:
220
221- `Linear`: issue key linked to Linear, such as `LFE-123`.
222- `Title`: Linear issue title in a separate column.
223- `Summary`: one operational sentence based on the issue body, comments, and
224 evidence. Avoid fix guesses.
225- `Owner`: assignee if present; otherwise owning team if clear; otherwise
226 `Unassigned`.
227- `Status`: Linear status or state, plus completion timing when useful.
228- `Touched Last Week Because`: `created`, `updated`, `completed`, or
229 `open production bug`.
230- `Production Evidence`: prod env, customer impact, incident.io incident,
231 Datadog link, measured logs/spans/errors, or `No measurements found`.
232- `Classification`: use `production/customer-impacting`, `internal-only`,
233 `self-hosted`, `staging/dev`, `duplicate/canceled/no-action`, or `unclear`.
234- `Counted?`: `yes` only when the bug label and production/customer-impacting
235 evidence support including it in fixed/open production bug counts.
236
237## Datadog Alerts Table
238
239Use this table for every production Datadog alert/page cluster found in the full
240paginated alert/event pass, including clusters later classified as
241`expected/test`, `monitor noise`, or `unknown/no measurements`.
242
243| Monitor / Page Signal | Env / Service | Count / Window | Why It Alerted | Trace / Span Evidence | Relevant Logs / Errors | Verdict | incident.io / Linear Link |
244| --- | --- | ---: | --- | --- | --- | --- | --- |
245
246Column rules:
247
248- `Monitor / Page Signal`: monitor/page title or stable ID.
249- `Env / Service`: production env and service/team labels.
250- `Count / Window`: grouped firing/page count and relevant time window.
251- `Why It Alerted`: monitor threshold, trigger condition, route, queue, status
252 code, latency, or backlog signal.
253- `Trace / Span Evidence`: representative trace/span links, error counts,
254 latency, status codes, dependency spans, or `No measurements found`.
255- `Relevant Logs / Errors`: explicit exception class/message, exact or
256 normalized log message, failed job IDs when visible, DB/downstream errors,
257 retry exhaustion, validation failures, or `No measurements found`.
258- `Verdict`: use `customer incident`, `confirmed bug`, `infra/dependency`,
259 `expected/test`, `monitor noise`, or `unknown/no measurements`.
260- `incident.io / Linear Link`: matching incident.io reference, Linear issue key,
261 explicit disposition, or `none found`.
262
263For API route and queue consumer errors, start from APM spans matching:
264
265```text
266operation_name:(http.server OR bullmq.consumer) status:error env:<prod-env>
267```
268
269Then narrow by `service`, `resource_name`, route, queue, consumer, status code,
270error type, or monitor time window. For failed trace samples, explicitly query
271related Datadog logs and error records using the trace ID, span ID, service,
272resource name, environment, and same time window. If no related logs or error
273records are found, write `No measurements found`.
274
275Before finalizing, compare the final Datadog alerts table against the full
276paginated alert/event sweep. Confirm every production monitor title seen during
277the window appears in the table or is explicitly excluded as non-prod.
278
279## Datadog Logs Table
280
281Use this table for the most frequent production error-log patterns from the
282review window. This is a broad log-health pass and is separate from the alert
283cluster rows above, though rows should cross-link when possible.
284
285Start from a Datadog Logs query shaped like:
286
287```text
288status:error
289```
290
291Scope it to the review window and production environments in scope. Prefer the
292Datadog pattern/clustering view using the log `message` field as the clustering
293pattern field. Group or facet by `service`, `env`, and `status` when available,
294and sort by count descending. Review at least the top 10 patterns overall, plus
295any additional top pattern per production environment when the global top 10 is
296dominated by one env or service.
297
298| Exact Log Pattern | Env / Service | Count / Share | Representative Error | Related Signal / Link | Disposition |
299| --- | --- | ---: | --- | --- | --- |
300
301Column rules:
302
303- `Exact Log Pattern`: exact Datadog clustered pattern or exact raw log message.
304 Preserve Datadog wildcard syntax such as `[wildcard]...[/wildcard]`. Do not
305 paraphrase this column.
306- `Env / Service`: affected production envs and services.
307- `Count / Share`: count in the review window and share if available.
308- `Representative Error`: one exact short representative message, exception
309 class, or stack/log summary. Avoid long stack traces.
310- `Related Signal / Link`: related Datadog alert row, trace, log query,
311 incident.io incident, Linear issue, or `none found`.
312- `Disposition`: use `known incident`, `tracked bug`, `needs investigation`,
313 `expected/test`, `monitor noise`, or `unknown`.
314
315If a high-volume pattern maps to a failed API route or queue consumer, ensure
316the matching Datadog alert row includes the trace/log/error investigation. If a
317high-volume pattern has no alert/page row, keep it in this table anyway and
318mark it `needs investigation` or `unknown` based on evidence.
319
320## Output Format
321
322Return valid Markdown only.
323
324- Use valid Markdown syntax for headings, links, and tables.
325- Include a space after list markers such as `-`, `*`, and `1.`.
326- Close links and parentheses correctly.
327- Do not emit malformed tables, dangling backticks, or partially opened code
328 fences.
329- Escape table pipes inside log patterns or error messages when needed.
330- If a section would be fragile to format, prefer a plain paragraph over broken
331 Markdown.