Agent Metric Dashboards
The five agent KPIs: sessions and turns per session, deflection, latency
percentiles, reasoning intensity (the observable cost driver), and quality. This
skill wires each to a real source and lays out the single-pane dashboard an
executive reviewer can read without being misled.
The Prerequisite That Gates Everything
Agentforce Session Tracing must be enabled — Setup → Einstein Audit, Analytics,
and Monitoring Setup → Agentforce Session Tracing plus the Session Tracing Data
Model — before the conversations you want to measure. Analytics appear only
for conversations occurring after setup; there is no backfill. Readers need the
Data Cloud User permission set or the dashboard is empty for them.
This makes dashboard enablement a row on the go-live checklist
(agentforce/agent-deployment-checklist), not an analytics backlog item.
Where The Numbers Actually Come From
| KPI |
Source |
Notes |
| Sessions |
AIAgentSession (Data Cloud DMO) |
Also reported directly by Agentforce Observability |
| Turns per session |
AIAgentInteraction ÷ AIAgentSession |
Turn-by-turn detail |
| Deflected sessions, average latency |
Agentforce Observability |
Reported as first-class insights |
| Escalation rate |
Action data on AIAgentInteraction |
Sessions whose action sequence includes the transfer action |
| Action failure rate |
AIAgentInteractionStep |
Group by action; sort by rate, not volume |
| Quality, feedback |
Session-tracing metric scores and feedback signals |
Calibrate against human labels quarterly |
There is no standard Conversation__c and no queryable per-token cost. A spec
that names them is a data-engineering project, not a dashboard.
Adoption Signals
Every production agent from activation onward; monthly executive review. Tracing
must be on from day zero, so this skill is consumed before launch rather than
after the first week.
Recommended Workflow
- Enable Session Tracing and the Session Tracing Data Model before activation;
assign Data Cloud User to every intended reader and verify with a real one.
- Build the volume and efficiency tiles from
AIAgentSession and
AIAgentInteraction: sessions, turns per session, LLM calls per turn, p50 and
p95 latency. Split agent latency from action latency — different owners,
different fixes.
- Make deflection causal or rename it. Randomise a holdout arm at the routing
layer and report
(rate_control − rate_treatment) / rate_control with both
arm sizes; where no holdout is possible, label the tile "Escalation rate (no
control arm)".
- Pair every proxy with its confound on the same tile: deflection with
72-hour repeat-contact rate, CSAT with
n and response rate, aggregate with
the per-subagent breakdown sorted by rate (subagent is the April 2026
rename of topic; the metadata names did not change).
- Report cost as drivers (sessions × turns/session × LLM-calls/turn) and take
absolute cost from Salesforce consumption reporting. Reconcile monthly; never
compute a currency figure from org data alone.
- Annotate every trend with agent version activations, prompt template
activations, and model version changes — model versions move without a
deployment on your side.
- Derive alert thresholds from four weeks of observed distribution, give each
one a named owner, and review at four weeks: did it fire, was every firing
actionable.
Key Considerations
- Deflection rises when the agent gets worse — a user who abandons looks
identical to one who was helped. It is never a solo KPI.
- Mean latency hides the tail that drives abandonment. Report p50 and p95.
- A quality score whose agreement with human judgement is unmeasured is an
unmonitored dependency of every decision made from it.
- The OTel export (
GET /services/data/v66.0/einstein/audit/otel/{session-id})
is one session per call, limited to the previous 72 hours, and Beta. Sample
rather than exporting everything, and alert on exporter lag.
Worked Examples (see references/examples.md)
- Deflection with baseline — Service org with 40% pre-agent escalation rate.
- Tokens/conversation trend — Costs spike after a subagent-instruction rewrite.
Common Gotchas (see references/gotchas.md)
- CSAT response bias — Only frustrated users answer the survey — CSAT looks terrible.
- Deflection = 'user gave up' — No escalation because user closed the browser in frustration.
- Cost metric without model version — Cost/conversation changes overnight due to model upgrade.
Top LLM Anti-Patterns (full list in references/llm-anti-patterns.md)
- Single-number CSAT with no context.
- Deflection without a baseline — reports vanity metrics.
- LLM-as-judge never calibrated — grades itself.
Official Sources Used
1---2name: agent-metric-dashboards3description: "Build the executive KPI dashboard for Agentforce: adoption, deflection, latency, cost, quality — KPI definitions, data sources, CRM Analytics lenses, alert thresholds. NOT for the platform's own session tracing, Agent Analytics and health monitoring — use agentforce/agentforce-observability. NOT for scoring agent answer quality offline — use agentforce/agentforce-eval-harness."4category: agentforce5salesforce-version: "Spring '25+"6well-architected-pillars:7 - Operational Excellence8 - Performance9triggers:10 - "what is my agent deflection rate"11 - "how much does each agent conversation cost"12 - "agent latency p95"13 - "agentforce roi dashboard"14tags:15 - agentforce16 - observability17 - dashboards18 - metrics19inputs:20 - "Conversation log access"21 - "CSAT or quality signal"22outputs:23 - "Einstein Analytics / CRM Analytics dashboard"24 - "weekly rollup email"25dependencies: []26version: 1.0.127author: Pranav Nagrecha28updated: 2026-08-1429metadata:30 source: https://skillmd.com/skills/pranavnagrecha/agent-metric-dashboards31---3233# Agent Metric Dashboards3435The five agent KPIs: sessions and turns per session, deflection, latency36percentiles, reasoning intensity (the observable cost driver), and quality. This37skill wires each to a **real** source and lays out the single-pane dashboard an38executive reviewer can read without being misled.3940## The Prerequisite That Gates Everything4142Agentforce Session Tracing must be enabled — Setup → Einstein Audit, Analytics,43and Monitoring Setup → Agentforce Session Tracing plus the Session Tracing Data44Model — **before** the conversations you want to measure. Analytics appear only45for conversations occurring after setup; there is no backfill. Readers need the46Data Cloud User permission set or the dashboard is empty for them.4748This makes dashboard enablement a row on the go-live checklist49(`agentforce/agent-deployment-checklist`), not an analytics backlog item.5051## Where The Numbers Actually Come From5253| KPI | Source | Notes |54|---|---|---|55| Sessions | `AIAgentSession` (Data Cloud DMO) | Also reported directly by Agentforce Observability |56| Turns per session | `AIAgentInteraction` ÷ `AIAgentSession` | Turn-by-turn detail |57| Deflected sessions, average latency | Agentforce Observability | Reported as first-class insights |58| Escalation rate | Action data on `AIAgentInteraction` | Sessions whose action sequence includes the transfer action |59| Action failure rate | `AIAgentInteractionStep` | Group by action; sort by rate, not volume |60| Quality, feedback | Session-tracing metric scores and feedback signals | Calibrate against human labels quarterly |6162There is no standard `Conversation__c` and no queryable per-token cost. A spec63that names them is a data-engineering project, not a dashboard.6465## Adoption Signals6667Every production agent from activation onward; monthly executive review. Tracing68must be on from day zero, so this skill is consumed before launch rather than69after the first week.7071## Recommended Workflow72731. Enable Session Tracing and the Session Tracing Data Model before activation;74 assign Data Cloud User to every intended reader and verify with a real one.752. Build the volume and efficiency tiles from `AIAgentSession` and76 `AIAgentInteraction`: sessions, turns per session, LLM calls per turn, p50 and77 p95 latency. Split agent latency from action latency — different owners,78 different fixes.793. Make deflection causal or rename it. Randomise a holdout arm at the routing80 layer and report `(rate_control − rate_treatment) / rate_control` with both81 arm sizes; where no holdout is possible, label the tile "Escalation rate (no82 control arm)".834. Pair every proxy with its confound **on the same tile**: deflection with84 72-hour repeat-contact rate, CSAT with `n` and response rate, aggregate with85 the per-subagent breakdown sorted by rate (*subagent* is the April 202686 rename of *topic*; the metadata names did not change).875. Report cost as drivers (sessions × turns/session × LLM-calls/turn) and take88 absolute cost from Salesforce consumption reporting. Reconcile monthly; never89 compute a currency figure from org data alone.906. Annotate every trend with agent version activations, prompt template91 activations, and model version changes — model versions move without a92 deployment on your side.937. Derive alert thresholds from four weeks of observed distribution, give each94 one a named owner, and review at four weeks: did it fire, was every firing95 actionable.9697## Key Considerations9899- Deflection **rises when the agent gets worse** — a user who abandons looks100 identical to one who was helped. It is never a solo KPI.101- Mean latency hides the tail that drives abandonment. Report p50 and p95.102- A quality score whose agreement with human judgement is unmeasured is an103 unmonitored dependency of every decision made from it.104- The OTel export (`GET /services/data/v66.0/einstein/audit/otel/{session-id}`)105 is one session per call, limited to the previous 72 hours, and Beta. Sample106 rather than exporting everything, and alert on exporter *lag*.107108## Worked Examples (see `references/examples.md`)109110- *Deflection with baseline* — Service org with 40% pre-agent escalation rate.111- *Tokens/conversation trend* — Costs spike after a subagent-instruction rewrite.112113## Common Gotchas (see `references/gotchas.md`)114115- **CSAT response bias** — Only frustrated users answer the survey — CSAT looks terrible.116- **Deflection = 'user gave up'** — No escalation because user closed the browser in frustration.117- **Cost metric without model version** — Cost/conversation changes overnight due to model upgrade.118119## Top LLM Anti-Patterns (full list in `references/llm-anti-patterns.md`)120121- Single-number CSAT with no context.122- Deflection without a baseline — reports vanity metrics.123- LLM-as-judge never calibrated — grades itself.124125## Official Sources Used126127- About Agentforce Session Tracing — https://help.salesforce.com/s/articleView?id=ai.generative_ai_session_trace_about.htm&type=5128- Set Up Agentforce Session Tracing — https://help.salesforce.com/s/articleView?id=ai.generative_ai_session_trace_setup.htm&type=5129- Data Model for Agentforce Session Tracing — https://help.salesforce.com/s/articleView?id=ai.generative_ai_session_trace_data_model.htm&type=5130- Export Agentforce Session Tracing Data (OTel API) — https://developer.salesforce.com/docs/ai/agentforce/guide/otel-api.html131- Data Model and Calculated Fields for Agent Analytics — https://help.salesforce.com/s/articleView?id=ai.generative_ai_agent_analytics_data_model.htm&type=5