Dashboard design
A dashboard is not an attic for every graph you ever built. It is an instrument
for answering one operational question fast while something is on fire. A panel
with no question behind it is decoration, and during an outage decoration is
scrolling you cannot afford. Design each board as a short list of questions.
Method
- Title every panel with its question. Not "Latency" but "Is /checkout
meeting its 2s p99?". The title states what the reader should learn, and a
panel that answers no clear question gets cut. The board then reads as a list
of questions, not a pile of metrics.
- Cover services with RED: rate, errors, duration. For each
request-serving component, show request rate, error rate, and a duration
distribution. Those three answer "is it up, is it failing, is it slow" for
anything that handles requests, which is most of what pages you.
- Cover resources with USE: utilization, saturation, errors. For CPU,
memory, disk, and pools, show how full it is, how much work waits in queue,
and its error count. USE finds the bottleneck while RED shows the symptom; a
saturated pool over an idle CPU is the classic catch.
- Stack panels symptom over cause. Put user-facing RED panels at the top
and resource USE panels beneath. The eye scans down from "what hurts" to
"why", following the shape of an investigation instead of scattering it
across the screen.
- Draw the SLO line on the panel. Render the threshold as a marker so "is
this bad" is a glance, not arithmetic. A latency graph with a 2s reference
line answers itself; a bare axis makes every viewer recompute the same
judgment.
- Fix the range and template the variables. Default to a window that spans
a deploy (1h) and add a
service or region dropdown rather than cloning
the board per target. One parameterized dashboard beats twenty copies that
drift apart.
Litmus tests
- Can a fresh on-call state each panel's question from its title alone?
- Does the board carry rate, errors, and duration for every serving service?
- In the last incident, did it shorten the path to cause or just add scrolling?
Boundaries
Dashboards summarize; they do not explain a single request. Once a panel shows
something wrong, the next step is a trace or a log query, not another graph.
What crosses into a page is alerting-design, and the series these panels plot
come from metrics-instrumentation.
1---2name: dashboard-design3description: Build dashboards where each panel answers exactly one question, laid out by the RED method for services and the USE method for resources. Use when a board has grown into a wall of graphs nobody can read while an incident burns.4---56# Dashboard design78A dashboard is not an attic for every graph you ever built. It is an instrument9for answering one operational question fast while something is on fire. A panel10with no question behind it is decoration, and during an outage decoration is11scrolling you cannot afford. Design each board as a short list of questions.1213## Method14151. **Title every panel with its question.** Not "Latency" but "Is /checkout16 meeting its 2s p99?". The title states what the reader should learn, and a17 panel that answers no clear question gets cut. The board then reads as a list18 of questions, not a pile of metrics.192. **Cover services with RED: rate, errors, duration.** For each20 request-serving component, show request rate, error rate, and a duration21 distribution. Those three answer "is it up, is it failing, is it slow" for22 anything that handles requests, which is most of what pages you.233. **Cover resources with USE: utilization, saturation, errors.** For CPU,24 memory, disk, and pools, show how full it is, how much work waits in queue,25 and its error count. USE finds the bottleneck while RED shows the symptom; a26 saturated pool over an idle CPU is the classic catch.274. **Stack panels symptom over cause.** Put user-facing RED panels at the top28 and resource USE panels beneath. The eye scans down from "what hurts" to29 "why", following the shape of an investigation instead of scattering it30 across the screen.315. **Draw the SLO line on the panel.** Render the threshold as a marker so "is32 this bad" is a glance, not arithmetic. A latency graph with a 2s reference33 line answers itself; a bare axis makes every viewer recompute the same34 judgment.356. **Fix the range and template the variables.** Default to a window that spans36 a deploy (1h) and add a `service` or `region` dropdown rather than cloning37 the board per target. One parameterized dashboard beats twenty copies that38 drift apart.3940## Litmus tests4142- Can a fresh on-call state each panel's question from its title alone?43- Does the board carry rate, errors, and duration for every serving service?44- In the last incident, did it shorten the path to cause or just add scrolling?4546## Boundaries4748Dashboards summarize; they do not explain a single request. Once a panel shows49something wrong, the next step is a trace or a log query, not another graph.50What crosses into a page is alerting-design, and the series these panels plot51come from metrics-instrumentation.
Amey-Thakur