Agent Metric Dashboards
The five agent KPIs: sessions and turns per session, deflection, latency
percentiles, reasoning intensity (the observable cost driver), and quality. This
skill wires each to a real source and lays out the single-pane dashboard an
executive reviewer can read without being misled.
The Prerequisite That Gates Everything
Agentforce Session Tracing must be enabled — Setup → Einstein Audit, Analytics,
and Monitoring Setup → Agentforce Session Tracing plus the Session Tracing Data
Model — before the conversations you want to measure. Analytics appear only
for conversations occurring after setup; there is no backfill. Readers need the
Data Cloud User permission set or the dashboard is empty for them.
This makes dashboard enablement a row on the go-live checklist
(agentforce/agent-deployment-checklist), not an analytics backlog item.
Where The Numbers Actually Come From
| KPI |
Source |
Notes |
| Sessions |
AIAgentSession (Data Cloud DMO) |
Also reported directly by Agentforce Observability |
| Turns per session |
AIAgentInteraction ÷ AIAgentSession |
Turn-by-turn detail |
| Deflected sessions, average latency |
Agentforce Observability |
Reported as first-class insights |
| Escalation rate |
Action data on AIAgentInteraction |
Sessions whose action sequence includes the transfer action |
| Action failure rate |
AIAgentInteractionStep |
Group by action; sort by rate, not volume |
| Quality, feedback |
Session-tracing metric scores and feedback signals |
Calibrate against human labels quarterly |
There is no standard Conversation__c and no queryable per-token cost. A spec
that names them is a data-engineering project, not a dashboard.
Adoption Signals
Every production agent from activation onward; monthly executive review. Tracing
must be on from day zero, so this skill is consumed before launch rather than
after the first week.
Recommended Workflow
- Enable Session Tracing and the Session Tracing Data Model before activation;
assign Data Cloud User to every intended reader and verify with a real one.
- Build the volume and efficiency tiles from
AIAgentSession and
AIAgentInteraction: sessions, turns per session, LLM calls per turn, p50 and
p95 latency. Split agent latency from action latency — different owners,
different fixes.
- Make deflection causal or rename it. Randomise a holdout arm at the routing
layer and report
(rate_control − rate_treatment) / rate_control with both
arm sizes; where no holdout is possible, label the tile "Escalation rate (no
control arm)".
- Pair every proxy with its confound on the same tile: deflection with
72-hour repeat-contact rate, CSAT with
n and response rate, aggregate with
the per-subagent breakdown sorted by rate (subagent is the April 2026
rename of topic; the metadata names did not change).
- Report cost as drivers (sessions × turns/session × LLM-calls/turn) and take
absolute cost from Salesforce consumption reporting. Reconcile monthly; never
compute a currency figure from org data alone.
- Annotate every trend with agent version activations, prompt template
activations, and model version changes — model versions move without a
deployment on your side.
- Derive alert thresholds from four weeks of observed distribution, give each
one a named owner, and review at four weeks: did it fire, was every firing
actionable.
Key Considerations
- Deflection rises when the agent gets worse — a user who abandons looks
identical to one who was helped. It is never a solo KPI.
- Mean latency hides the tail that drives abandonment. Report p50 and p95.
- A quality score whose agreement with human judgement is unmeasured is an
unmonitored dependency of every decision made from it.
- The OTel export (
GET /services/data/v66.0/einstein/audit/otel/{session-id})
is one session per call, limited to the previous 72 hours, and Beta. Sample
rather than exporting everything, and alert on exporter lag.
Worked Examples (see references/examples.md)
- Deflection with baseline — Service org with 40% pre-agent escalation rate.
- Tokens/conversation trend — Costs spike after a subagent-instruction rewrite.
Common Gotchas (see references/gotchas.md)
- CSAT response bias — Only frustrated users answer the survey — CSAT looks terrible.
- Deflection = 'user gave up' — No escalation because user closed the browser in frustration.
- Cost metric without model version — Cost/conversation changes overnight due to model upgrade.
Top LLM Anti-Patterns (full list in references/llm-anti-patterns.md)
- Single-number CSAT with no context.
- Deflection without a baseline — reports vanity metrics.
- LLM-as-judge never calibrated — grades itself.
Official Sources Used
1---2name: agent-metric-dashboards3description: Build the executive KPI dashboard for Agentforce: adoption, deflection, latency, cost, quality — KPI definitions, data sources, CRM Analytics lenses, alert thresholds. NOT for the platform's own session tracing, Agent Analytics and health monitoring — use agentforce/agentforce-observability. NOT for scoring agent answer quality offline — use agentforce/agentforce-eval-harness.4---56# Agent Metric Dashboards78The five agent KPIs: sessions and turns per session, deflection, latency9percentiles, reasoning intensity (the observable cost driver), and quality. This10skill wires each to a **real** source and lays out the single-pane dashboard an11executive reviewer can read without being misled.1213## The Prerequisite That Gates Everything1415Agentforce Session Tracing must be enabled — Setup → Einstein Audit, Analytics,16and Monitoring Setup → Agentforce Session Tracing plus the Session Tracing Data17Model — **before** the conversations you want to measure. Analytics appear only18for conversations occurring after setup; there is no backfill. Readers need the19Data Cloud User permission set or the dashboard is empty for them.2021This makes dashboard enablement a row on the go-live checklist22(`agentforce/agent-deployment-checklist`), not an analytics backlog item.2324## Where The Numbers Actually Come From2526| KPI | Source | Notes |27|---|---|---|28| Sessions | `AIAgentSession` (Data Cloud DMO) | Also reported directly by Agentforce Observability |29| Turns per session | `AIAgentInteraction` ÷ `AIAgentSession` | Turn-by-turn detail |30| Deflected sessions, average latency | Agentforce Observability | Reported as first-class insights |31| Escalation rate | Action data on `AIAgentInteraction` | Sessions whose action sequence includes the transfer action |32| Action failure rate | `AIAgentInteractionStep` | Group by action; sort by rate, not volume |33| Quality, feedback | Session-tracing metric scores and feedback signals | Calibrate against human labels quarterly |3435There is no standard `Conversation__c` and no queryable per-token cost. A spec36that names them is a data-engineering project, not a dashboard.3738## Adoption Signals3940Every production agent from activation onward; monthly executive review. Tracing41must be on from day zero, so this skill is consumed before launch rather than42after the first week.4344## Recommended Workflow45461. Enable Session Tracing and the Session Tracing Data Model before activation;47 assign Data Cloud User to every intended reader and verify with a real one.482. Build the volume and efficiency tiles from `AIAgentSession` and49 `AIAgentInteraction`: sessions, turns per session, LLM calls per turn, p50 and50 p95 latency. Split agent latency from action latency — different owners,51 different fixes.523. Make deflection causal or rename it. Randomise a holdout arm at the routing53 layer and report `(rate_control − rate_treatment) / rate_control` with both54 arm sizes; where no holdout is possible, label the tile "Escalation rate (no55 control arm)".564. Pair every proxy with its confound **on the same tile**: deflection with57 72-hour repeat-contact rate, CSAT with `n` and response rate, aggregate with58 the per-subagent breakdown sorted by rate (*subagent* is the April 202659 rename of *topic*; the metadata names did not change).605. Report cost as drivers (sessions × turns/session × LLM-calls/turn) and take61 absolute cost from Salesforce consumption reporting. Reconcile monthly; never62 compute a currency figure from org data alone.636. Annotate every trend with agent version activations, prompt template64 activations, and model version changes — model versions move without a65 deployment on your side.667. Derive alert thresholds from four weeks of observed distribution, give each67 one a named owner, and review at four weeks: did it fire, was every firing68 actionable.6970## Key Considerations7172- Deflection **rises when the agent gets worse** — a user who abandons looks73 identical to one who was helped. It is never a solo KPI.74- Mean latency hides the tail that drives abandonment. Report p50 and p95.75- A quality score whose agreement with human judgement is unmeasured is an76 unmonitored dependency of every decision made from it.77- The OTel export (`GET /services/data/v66.0/einstein/audit/otel/{session-id}`)78 is one session per call, limited to the previous 72 hours, and Beta. Sample79 rather than exporting everything, and alert on exporter *lag*.8081## Worked Examples (see `references/examples.md`)8283- *Deflection with baseline* — Service org with 40% pre-agent escalation rate.84- *Tokens/conversation trend* — Costs spike after a subagent-instruction rewrite.8586## Common Gotchas (see `references/gotchas.md`)8788- **CSAT response bias** — Only frustrated users answer the survey — CSAT looks terrible.89- **Deflection = 'user gave up'** — No escalation because user closed the browser in frustration.90- **Cost metric without model version** — Cost/conversation changes overnight due to model upgrade.9192## Top LLM Anti-Patterns (full list in `references/llm-anti-patterns.md`)9394- Single-number CSAT with no context.95- Deflection without a baseline — reports vanity metrics.96- LLM-as-judge never calibrated — grades itself.9798## Official Sources Used99100- About Agentforce Session Tracing — https://help.salesforce.com/s/articleView?id=ai.generative_ai_session_trace_about.htm&type=5101- Set Up Agentforce Session Tracing — https://help.salesforce.com/s/articleView?id=ai.generative_ai_session_trace_setup.htm&type=5102- Data Model for Agentforce Session Tracing — https://help.salesforce.com/s/articleView?id=ai.generative_ai_session_trace_data_model.htm&type=5103- Export Agentforce Session Tracing Data (OTel API) — https://developer.salesforce.com/docs/ai/agentforce/guide/otel-api.html104- Data Model and Calculated Fields for Agent Analytics — https://help.salesforce.com/s/articleView?id=ai.generative_ai_agent_analytics_data_model.htm&type=5
PranavNagrecha