# Workflow Debugger

> Audit a Duvo Agent — or a multi-Agent workflow connected by a Queue — across many Runs to find systemic inefficiencies and quality issues, then recommend concrete AOP and architecture changes. Use when the user asks to "analyze this workflow", "audit my Agent", "why is this Agent slow / inconsistent / low quality across runs", "why does my queue keep backing up", or wants a health check over an Agent's recent Runs — as opposed to debugging one failed Run (that's run-debugger). Reads recent Runs, eval scores, the producer/consumer queue topology, and the AOPs those Runs actually ran against via the Duvo public API — leading with the queue topology and backlog and sizing the run sample to the question; hands off to aop-writer for any AOP rewrite.

- Skill: `duvoai/workflow-debugger` (Agent Skill)
- Install (CLI): `npx skillmds@latest add duvoai/workflow-debugger`
- Raw SKILL.md: https://api.skillmd.com/api/skills/duvoai/workflow-debugger/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Product & Planning
- License: MIT
- Author: duvoai (https://skillmd.com/u/duvoai)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/duvoai/workflow-debugger

---


# Workflow Debugger

## What is Duvo?

[Duvo](https://duvo.ai) is an AI-powered automation platform that handles repetitive business work across the systems a team already uses. Unlike traditional automation that follows rigid, pre-programmed rules, a Duvo **Agent** understands the goal, adapts to each situation, and acts on the user's behalf through their own **Connections** (linked tools like Gmail, Slack, or a CRM) — as if the user were doing the work themselves. An Agent is configured once — its **AOP** (the markdown procedure that becomes its prompt), Connections, and settings form a **Build** — and then performs **Runs**: individual executions, each with an input, a full transcript, and a result.

## What you're doing

A **workflow** is more than one Agent's Runs: it's a **structure** — often a _pair_ of Agents connected by a **Queue** (a **producer** that pushes cases in, a **consumer** triggered to work them) — _and_ the population of Runs flowing through it. When the workflow is slow, inconsistent, low-quality, or backing up, the user wants two things:

1. **What the workflow is doing inefficiently, with evidence — structural or behavioural.**
2. **What changes would make it faster, cheaper, or more reliable next time.**

You answer both. Some findings are **structural** — the topology, a backed-up Queue, the seam between the producer and consumer AOPs — and you read these from the topology and the Queue's state, not from a large run sample. Others are **behavioural** — quality, escalation, wasted work — and these need counts across a sample of Runs. Match the evidence to the question instead of profiling many Runs by default. You do not ship the change: the user (or `aop-writer`) lands it.

You read; you do not edit Agents, AOPs, Connections, queues, or cases.

## run-debugger vs workflow-debugger

These two are complementary — pick the right one, and use them together.

- **`run-debugger`** diagnoses **one Run** that failed or produced the wrong outcome, grounded in that Run's transcript and the Build it ran. Use it when the user points at a specific Run.
- **`workflow-debugger`** (this skill) audits the **whole Agent or a producer→consumer pair** — grounded in the queue topology and backlog, the AOPs at the seam, and a sample of recent Runs with their eval scores. Use it when the user wants a health check, an efficiency audit, or asks why the Agent behaves badly _in general_.

If the sweep surfaces a recurring failure that needs transcript-level depth, hand a representative Run to `run-debugger`. If the user only has one bad Run, start with `run-debugger`.

## Operating mode

Use this session's configured Duvo access to perform the operations below. Follow its runtime instructions for invocation and parameter lookup. Operation names identify the required action; they do not imply that a same-named tool must appear in the tool list. Do not choose another transport or infer that Duvo is unavailable from the tool list alone.

- **API mode** — use configured Duvo access to pull the Run set, topology, and AOPs directly.
- **Paste mode** — in a standalone environment with no configured Duvo access, ask for the recent Run list (status, case titles, eval scores), producer/consumer setup, and AOPs in effect.

The analysis dimensions, taxonomy, recommendation shape, and output rule are identical across modes; only data gathering differs. Do not invent Run data, eval scores, or AOP content.

## The single most important rule

**Ground every claimed inefficiency in evidence fit to the finding — not a hunch.** A **behavioural** claim ("this Agent over-escalates") needs counts across a sample of Runs: a pattern is "N of the M recent Runs I pulled did X", and one failed Run is a `run-debugger` question, not a pattern. A **structural** claim (a producer/consumer mix, a backed-up Queue, a monolithic AOP) is grounded in the topology, the `problems` array, the Queue's backlog, and the AOPs at the seam — it does **not** need a large run sample, and manufacturing a run count for it is noise. Either way, if you can't point to the evidence, say so rather than asserting it.

Two corollaries:

- **Analyse the AOP the Runs actually ran against**, identified by the `build_id` carried on recent Runs — not a nominal "live" label. There is no "live revision" filter in the API; the Build that recent Runs executed is the honest answer. (The current live Build is usually the highest `revision_number`, but a promotion can repoint it, so trust the `build_id` on real Runs.)
- **Distinguish a symptom from its cause.** "Eval pass rate is low" is the symptom. The cause is almost always one AOP gap repeated every run, a miscalibrated threshold, a missing terminal action, or a topology mismatch. Name the cause.

## Inputs you need

At minimum, one of:

- An **Agent ID** (the Agent to audit), or
- A **Queue ID** (to audit the producer/consumer workflow around it).

From either you can derive the rest — the queue from the Agent's Runs, the partner Agents from the queue. If you have neither, ask the user before reading anything. Do not guess from context.

## Tools — read-only public API operations (API mode)

In API mode these identify the required reads. Use only filters supported by the configured runtime; do not assume every transport exposes the same parameters.

- `listRuns` — recent Runs for the Agent, with status, `build_id`, `case_*` fields, timestamps, and `eval_summaries`. Newest-first by default, so `limit` controls how far back you reach; **narrow the pull** with `status`, `has_issues`, `issue_severity`, `since` (an ISO timestamp), or `case_queue_id` rather than fetching everything. Use a small initial limit (20 by default, max 100) and the filters the configured operation supports.
- `getRevision` — a single Build, including its `config` (which holds the AOP). Pass the `build_id` from recent Runs.
- `listAgentRevisions` — the Agent's Build history (`revision_number`, timestamps).
- `listQueueAgents` — the queue's **producers and consumers**, each with `case_trigger_enabled`, `is_handover_target`, and a `problems` array (`multiple_triggers`, `producer_consumer_mix`). The fastest read for a topology problem — one call, no run sample.
- `listCases` / `getQueue` — the Queue's **backlog and state**. `listCases` filtered by status (`pending`, `needs_input`, `postponed`, `claimed`, `completed`, `failed`) returns the waiting cases and a `total`, so a deep, ageing `pending` / `postponed` backlog (sorted oldest-first) is the direct evidence a Queue is backing up — no run sample needed.
- `listAgentCaseTriggers` — which queue(s) trigger this Agent (the consumer binding).
- `getAgent` — Agent-level metadata (name, delivery settings).
- `getCase` / `listCaseRuns` — a single case's state and every Run that has worked it, when you need to confirm a case is bouncing rather than closing.

**Match the data to the question.** A small peek of recent Runs anchors the `build_id` (the AOP actually in effect) and the `case_queue_id` (which Queue this Agent works); beyond that, **structural** questions are carried by the topology and the Queue backlog, and only **behavioural** questions (quality, escalation, wasted work) need a larger run sample. Don't default to a big run pull — fetch what the lens needs.

## What to ask the user (paste mode)

In paste mode, ask for the minimum needed — matched to the lens, not a big run dump:

1. **For a queue workflow (structural):** who produces and who consumes, and how deep the backlog is (counts of `pending` / `postponed` / `needs_input` cases), plus the producer and consumer AOPs in effect.
2. **For a quality / behavioural question:** a recent Run sample — status, case title, and eval score per Run (narrowed to failures or issues when appropriate) — plus the eval `final_comment` text across the affected Runs.
3. **Always:** enough of a recent Run peek to anchor the in-effect AOP (`build_id`) and the Queue the Agent works.

Open with the structure (or with the run sample, for a pure quality question); ask for more only if the first round can't place the pattern in the taxonomy.

## Investigation workflow

The steps are the same in either mode; only the data source changes. The order is **structure-first** for a queue workflow: the topology and Queue state carry the structural findings cheaply and tell you how many Runs — if any beyond a small peek — you actually need.

1. **Frame the question and anchor.** Fix the lens — **structural** (topology, backlog, decomposition) vs. **behavioural** (quality, escalation, wasted work) — and the scope (single Agent vs. producer→consumer workflow). Pull a small peek of recent Runs (`listRuns`, a handful) to read the `case_queue_id` (which Queue this Agent works) and the `build_id` the Runs actually ran — you need both to find the topology and the in-effect AOP. This peek is also your behavioural baseline; widen it only in step 4.

2. **For a queue workflow, read the structure.** A handful of cheap calls that carry the structural findings without a large run sample. _API mode:_ `listQueueAgents` (producers, consumers, `problems`); the Queue backlog via `listCases` filtered to `pending` / `needs_input` / `postponed`, sorted oldest-first (a deep, ageing backlog is the direct "backing up" signal); `listAgentCaseTriggers` to confirm the binding. _Paste mode:_ ask who produces, who consumes, and how deep the backlog is. A standalone Agent with no `case_queue_id` has no topology — skip to step 4.

3. **Read the AOPs at the seam.** Take the `build_id` from the peek and pull that Build's AOP for the consumer (and the producer). _API mode:_ `getRevision` for that `build_id` — the AOP is in `config`. _Paste mode:_ ask the user to paste the AOP in effect. Read producer and consumer AOPs together — many workflow problems live at the seam between them.

4. **Widen the run sample only for behavioural questions.** If the findings are already structural, the peek from step 1 is enough — don't pull more. For a quality or run-level-behaviour question, widen the sample (narrowed by `status` / `has_issues` / `issue_severity`, toward the 100 max only for a broad health check), then profile it (see dimensions below): status mix, eval pass rate and severity, recurring `final_comment`, cadence and duration, and which `build_id`(s) the Runs ran. Counts here become your behavioural evidence. _API mode:_ `listRuns` filtered to the Agent. _Paste mode:_ ask the user for the list.

5. **Synthesise the report.** Place the top issues in the taxonomy, attach the evidence that proves each — topology / backlog for structural, counts for behavioural — and propose one concrete change per issue. AOP changes hand off to `aop-writer`; topology changes are described as an architecture suggestion.

## What to profile across the run set

Each dimension maps to a field on the Runs from `listRuns`. Quantify, don't eyeball.

- **Status breakdown** — count `completed` / `failed` / `interrupted` / `stopped` / `waiting` / `needs_attention` / `running`. A high `needs_attention` or `waiting` share signals escalation or closure problems; `interrupted` / `stopped` signal wasted work.
- **Case variety** — is it the same `case_title` every run, or many distinct cases/markets? Repetition of one title across Runs means a case that won't close; wide variety means real throughput.
- **Eval scores** — for each Run's `eval_summaries`: is `passed < total`? Read `severityCounts` (`critical` / `medium` / `low`). Cluster by severity — a recurring `critical` is the headline.
- **Recurring eval comments** — the same `final_comment` complaint across many Runs is the single strongest signal of a prompt issue: the AOP is producing the same defect every run.
- **Frequency and timing** — Run cadence from `created_at` / `started_at`, and duration from `started_at` → `completed_at`. A steady drumbeat of tiny near-identical Runs hints at batching or scheduling; long durations hint at a monolithic AOP.
- **Build spread** — are recent Runs on one `build_id` or several? A change in behaviour around a Build boundary points at an AOP edit as the cause.

## Inefficiency taxonomy

Most workflow problems are one of these. Name the category, and back it with counts.

1. **Recurring quality gap (eval-driven).** Many Runs share the same `final_comment` and `passed < total`, often at one severity. Cause: a single AOP gap producing the same defect every run. Evidence: count of Runs with that complaint + their severity. Fix: the AOP line that omits the criterion.

2. **Cases that don't close (terminal-closure leak).** Cases bounce via postpone/re-pickup without `complete_case` / `fail_case`. Evidence — read it directly: a deep `postponed` / `needs_input` backlog (`listCases`) and one stuck case's `listCaseRuns` showing repeated pickups with no terminal action; the same `case_title` recurring in the run sample corroborates. Fix: add the missing terminal action to that AOP branch.

3. **Escalation miscalibration (HITL).** Over-escalation — a large `needs_attention` / `waiting` share where the AOP should decide autonomously; or under-escalation — costly autonomous actions with no Human-in-the-loop gate. Evidence: the status mix vs. the AOP's decision rules. Fix: tune the threshold in the AOP.

4. **Serial work that should be batched or scheduled.** Many tiny Runs at high cadence doing near-identical work, or a fixed drumbeat that should be a schedule. Evidence: run frequency + case-title sameness. Suggestion: batch through the queue, or move to a scheduled trigger.

5. **Producer/consumer imbalance or topology problem.** `listQueueAgents` reports a `problems` entry (`multiple_triggers`, `producer_consumer_mix`), or the producer floods cases faster than the consumer clears them. Evidence: the `problems` array, plus a `pending` backlog growing faster than the consumer drains it (`listCases`). Suggestion: split triggers, adjust consumer concurrency, or separate producer from consumer.

6. **Monolithic Agent (decomposition signal).** One Agent's AOP spans Connection domains and distinct cadences; its Runs run long and fail at the seams. Evidence: AOP length/phase boundaries + a spread of unrelated failure modes in one Agent. Suggestion: split into a producer→consumer pair via a Queue or `request_handover`.

7. **Wasted Runs.** `interrupted` / `stopped` runs, postpone loops, retries with no forward progress. Evidence: status mix + repeated `build_id` with no completion. Fix: an AOP early-return or guard so the Agent stops doing no-op work.

If a problem doesn't fit, name the pattern plainly. Do not force-fit.

## What a recommendation looks like

A recommendation is **one concrete change to one artifact**, with the evidence that motivates it:

- "12 of the last 20 Runs failed eval with the comment _'did not include the PO number'_ (all `medium`). **Quote the AOP line to change:** _'Reply to the supplier with the delivery status.'_ → it should require the PO number. Hand to `aop-writer`."
- "The same case _'Reorder SKU-4471'_ appears in 9 Runs, all `waiting`, never `completed`. Step 5 of the consumer AOP has no `complete_case` on the in-stock branch. Add it."
- "`listQueueAgents` reports `producer_consumer_mix` on Agent X — it both fills and drains the queue. Split it into two Agents."
- "16 of the last 20 Runs ran < 20s on near-identical single-SKU cases. Batch via the queue or move to a 15-minute schedule instead of per-case triggers."

Avoid: "tighten the AOP", "improve quality", "consider batching". A recommendation the user can't act on verbatim is not a recommendation. When the fix is in the AOP, **quote the exact line to change** — the user asked for that specificity.

## Handoff to `aop-writer`

When a recommendation is an AOP change, **stop short of rewriting the AOP here.** Hand off to `aop-writer` with two things:

1. The exact AOP that was in effect (from `getRevision` on the `build_id` the Runs ran).
2. The specific change request, phrased the way the user would ("rewrite Step 5 to require the PO number in the supplier reply").

`aop-writer` returns the rewritten AOP. You do not. This split is deliberate: `workflow-debugger` finds the systemic issue; `aop-writer` writes the fix. Mixing the two produces shallow rewrites and unanchored audits.

## Anti-patterns — reject

- **Returning a behavioural finding without a run sample.** A quality / escalation / wasted-work claim needs counts from Runs you actually pulled — without them you're guessing. (A _structural_ finding — a `problems` entry, a deep backlog — stands on the topology and Queue state instead, no large sample required.)
- **Calling one or two Runs a pattern.** A finding needs a count across the run set. Two bad Runs is a `run-debugger` question, not a workflow inefficiency.
- **Auditing the wrong AOP** — the Agent's current Build when recent Runs ran an earlier one. Read the Build the Runs actually executed (`build_id`).
- **Reading only one side of a queue.** Producer and consumer AOPs must be read together; the problem is often the seam between them.
- **Inventing run counts, eval comments, or AOP lines** the data doesn't show. Quote what's there; report a gap as a gap.
- **Bundling unrelated changes** into one recommendation, or returning more than the top few — prioritise by evidence weight.
- **Rewriting the AOP inline.** Hand off to `aop-writer`.

## Output rule

Return one structured report with these labelled sections, in this order:

- **What the workflow does** — one sentence.
- **Top inefficiencies** — up to three, ordered by evidence weight. Each names a taxonomy category and carries its evidence: counts from the run set and/or a quoted eval comment.
- **Prompt changes** — for each AOP-level fix, name the artifact (producer or consumer AOP, the step) and **quote the exact line to change**.
- **Architecture suggestions** — topology-level changes (batching, scheduling, decomposition, producer/consumer rebalancing), only when the data supports them. Omit the section if there are none.
- **Next step** — e.g. "I can invoke `aop-writer` to rewrite Step 5 of the consumer AOP", or "Hand Run `<id>` to `run-debugger` for the transcript-level cause".

If the user asked only about one dimension ("is this Agent over-escalating?"), answer that dimension with its counts and skip the rest.

## Reading the request

1. **Find the Agent or Queue reference** in the conversation. If absent, ask before reading.
2. **Determine scope.** Single Agent ("audit this Agent") vs. workflow ("why does this queue back up", "analyse this producer→consumer flow"). The first profiles one Agent's Runs; the second adds the topology and reads both AOPs.
3. **Determine the lens.** Efficiency (speed, cost, wasted Runs, batching) vs. quality (eval scores, recurring defects) vs. reliability (closure, escalation). Lead with the lens the user named; surface the others only if the data makes them unavoidable.

Use only data returned by configured Duvo access (API mode) or shared by the user (paste mode). Do not infer the contents of Files, Connections' upstream systems, or Runs you didn't pull. The run set, the topology, and the AOPs are the source of truth.

## Final check before returning

Walk through this once on your draft. Fix anything that fails.

- [ ] Every finding is grounded in evidence fit to it — counts across the run sample for behavioural patterns, the topology / `problems` / Queue backlog for structural ones — not a single Run and not a hunch.
- [ ] You read the AOP the Runs **actually ran** (`build_id`), not just the current Build.
- [ ] For a queue workflow, you read **both** producer and consumer AOPs and checked `listQueueAgents` `problems`.
- [ ] Each inefficiency is named from the taxonomy and carries its evidence.
- [ ] Each prompt change **quotes the exact AOP line** to change and names the artifact.
- [ ] AOP rewrites are handed to `aop-writer`, not written here.
- [ ] Duvo terminology used: Agent, Run, Build, AOP, Connection, Queue, Files, Setup.
- [ ] You did not invent run counts, eval comments, AOP lines, or topology the data doesn't show.

## Duvo terminology

Use Duvo's nouns when describing the workflow and the fix. Never substitute — the user is working inside the product and these are the words on the screen.

| Use        | Not                                 |
| ---------- | ----------------------------------- |
| Agent      | assignment, AI teammate, bot        |
| Run        | task, job, execution                |
| Build      | revision, version                   |
| AOP        | SOP, instructions, prompt, playbook |
| Connection | integration, account                |
| Queue      | case queue, backlog                 |
| Files      | knowledge base, documents           |
| Setup      | configuration, config               |

## See also

- `run-debugger` — for one failed Run: it reads the transcript and the Build that ran it. This skill audits the whole workflow; hand it a representative Run when a pattern needs transcript-level depth.
- `aop-writer` — once you've named an AOP-level fix, hand off the in-effect AOP and the change request; this skill never rewrites AOPs itself.
- `duvo-cli` — command references for a session configured to use the CLI, or a user auditing from a shell.

## Resources

- [Duvo](https://duvo.ai) — product website
- [Duvo documentation](https://docs.duvo.ai) — building Agents, AOPs, Connections, Queues
- [Web app](https://app.duvo.ai) — open the Agent, inspect its Runs, evals, and the Build that ran them
- [Duvo CLI (`@duvoai/cli`)](https://www.npmjs.com/package/@duvoai/cli) — the read commands this skill relies on in API mode; pairs with the `duvo-cli` skill
- [Public skill repository](https://github.com/duvoai/skills) — the MIT-licensed community release of this skill, packaged for installation in third-party Claude Code setups