Dogfood

Dogfood Skill

VCS Provider

This skill uses VCS operations through Exarchos MCP actions (create_issue, etc.). These actions automatically detect and route to the correct VCS provider (GitHub, GitLab, Azure DevOps). No gh/glab/az commands needed — the MCP server handles provider dispatch.

Overview

Retrospective analysis of Exarchos MCP tool usage. Uses the MCP server's own self-service capabilities as the primary diagnostic instrument — describe APIs, views, playbooks, and runbooks turned inward to diagnose failures.

This is the in-runtime, event-sourced counterpart to the older transcript-only triage: agents file friction reports in real time via exarchos_workflow feedback, which land on the shared meta/feedback stream. Dogfood reads that stream as a first-class input (Step 1h) — each feedback.recorded event is an agent-surfaced friction signal already captured at the moment it hurt, not reconstructed after the fact.

Three distinct failure modes require different fixes — code changes, documentation updates, or skill instruction improvements. Mixing them wastes effort.

Platform-Agnosticity

Per docs/designs/archive/2026-03-09-platform-agnosticity.md: the MCP server is the self-sufficient, platform-agnostic core. The debug trace relies entirely on MCP tools — not conversation introspection — so it works for any MCP client. Conversation scanning is supplementary.

Diagnostic self-service tools: describe(topology) for HSM verification, describe(playbook) for adherence checks, describe(eventTypes, emissionGuide) for event schema/catalog comparison, describe(actions) for schema/gate metadata, runbook(phase) for step conformance, pipeline/convergence/telemetry views for health metrics.

Triggers

Activate this skill when:

• User runs /dogfood or dogfood
• User asks "what went wrong this session" or "review the failures"
• User wants to triage errors from a workflow run
• End of a workflow session to capture learnings

Process

Step 1: Debug Trace via MCP Self-Service

Query the MCP server's own self-service capabilities to build a ground-truth diagnostic picture. This is the primary investigation method — it uses the same tools any MCP client has access to.

1a. Identify Active Workflows

Use exarchos_view with action: "pipeline" to get an aggregated view of active workflows with their phases and task counts. The view is repo-scoped by default (the caller's repo only); when the response reports unscopedTotal greater than page.total, workflows in other repos are hidden — re-query with scope: "all" to inspect the full cross-repo inventory.

If $ARGUMENTS specifies a workflow or feature ID, scope to that workflow. Otherwise, inspect all non-terminal workflows.

1b. Inspect Workflow State and Topology

For each relevant workflow:

1. Read state — exarchos_workflow get to retrieve current phase, tasks, reviews, gate results.
2. Read topology — exarchos_workflow describe(topology: "<workflowType>") to get the HSM definition. Compare the agent's phase transition attempts against valid transitions. Invalid transition attempts = documentation issue (skill prescribed wrong path) or user error.
3. Check guard prerequisites — For workflow.guard-failed events, look up the guard in the topology to understand unmet preconditions.

1c. Playbook Adherence Check

Use exarchos_workflow describe(playbook: "<workflowType>") to retrieve phase playbooks. For each phase executed, compare playbook's tools, events, transitionCriteria, guardPrerequisites, humanCheckpoint, and compactGuidance against what the agent actually did and what skill docs prescribe.

Playbook violations are diagnostic gold:

• Agent deviated and skill docs told it to → documentation issue (skill contradicts playbook)
• Agent deviated and skill docs agree with playbook → user error
• Playbook is wrong (prescribes invalid tools/events) → code bug

1d. Event Log Analysis

Use exarchos_event query(stream) on the workflow's event stream. Look for:

• Rejected events — absent from log despite agent attempts (corroborate with conversation errors)
• Missing events — compare against playbook events field and exarchos_event describe(emissionGuide: true). Missing model-emitted events = documentation gap or user error.
• Sequence anomalies — wrong order, duplicates, or timeline gaps
• Schema mismatches — use describe(eventTypes: [...]) to get authoritative JSON Schema. Compare actual payloads against schema for semantically wrong fields.

1e. Orchestrate Action and Gate Analysis

1. Schema verification — exarchos_orchestrate describe(actions: [...]) for authoritative schemas. Compare agent's parameters against schema to detect stale skill docs or improvisation.
2. Gate metadata — Describe output includes { blocking, dimension, autoEmits }. Check: did the agent treat blocking/non-blocking correctly? Did expected auto-emissions fire?
3. Gate convergence — exarchos_view convergence for per-dimension (D1-D5) pass rates. Low convergence suggests systemic gate issues.

1f. Runbook Conformance Check

Use exarchos_orchestrate runbook(phase) to retrieve relevant runbooks. Check: step ordering, decision branch correctness (steps with decide fields), onFail directive adherence (stop/continue/retry), and templateVars completeness.

1g. Telemetry Review

Use exarchos_view telemetry for per-tool performance. Flag: high error rates (systemic issues), high invocation counts (retry loops), and tools never invoked that the playbook prescribes.

1h. Friction Back-Channel (feedback.recorded)

Use exarchos_event query(stream: "meta/feedback") to read recent friction reports filed via exarchos_workflow feedback. These are agent-surfaced findings captured in real time — an agent hit a painful affordance and flagged it during the failing call instead of retrying silently. Treat each report as a pre-seeded finding:

• message — the agent's own account of the friction. Carry it into Step 3 as the "what went wrong" rather than reconstructing it from the transcript.
• sessionContext (workflow / action / errorCode) — correlate the report to the workflow + action it occurred in. Cross-reference with the event log (1d) and telemetry (1g) for corroborating server-side evidence.
• upstreamDelivered — false despite a configured configuredEndpoint is itself a finding (federation hop failing).

Each report still passes through Step 4 categorization (code / docs / user). A feedback.recorded event with no matching server-side defect is not automatically dismissed — the agent's friction is signal even when the call eventually succeeded (see the successful-after-retry anti-pattern). De-duplicate reports against findings already surfaced by the debug trace so a back-channel report and its trace evidence merge into one finding, not two.

Step 2: Scan Session for Failed Tool Calls

Supplement the debug trace with client-side context — review conversation for failed Exarchos tool calls.

Note: Platform-dependent step (requires conversation history). Skip on platforms without introspection; the debug trace is self-sufficient.

Target tools: exarchos_workflow, exarchos_event, exarchos_orchestrate, exarchos_view, exarchos_sync

Error signals: INVALID_INPUT, VALIDATION_ERROR, BATCH_APPEND_FAILED, Zod failures (invalid_type, invalid_enum_value, unrecognized_keys), ENOENT, CLAIM_FAILED, SEQUENCE_CONFLICT, CAS exhaustion, retry sequences, successful-after-retry calls.

Step 3: Diagnose Each Failure

Merge debug trace and conversation scan findings. For each failure document:

1. What was attempted — action, parameters, intent
2. What went wrong — error message and validation path
3. Server-side evidence — event log, state, describe output, views
4. Authoritative reference — the self-service query providing ground truth (playbook, topology, schema, runbook)
5. Root cause — per references/root-cause-patterns.md
6. Fix category — code, docs, or user behavior

Flag discrepancies only visible via server-side inspection as trace-only findings.

Step 4: Categorize into Buckets

Assign each failure to exactly one root cause bucket:

Bucket 1: Code Bug

The MCP server, event store, or workflow engine has a defect.

Signals: Schema rejects valid input (confirmed via describe), CAS failures with no concurrent writers, gate over-enforcement, identical-parameter retry succeeds (race condition), state corruption, topology/engine mismatch, auto-emission failure.

Action: File bug issue with reproduction steps, expected vs actual, and suggested fix.

Bucket 2: Documentation Issue

Skill docs are wrong, incomplete, or out of sync with the MCP server's self-service output.

Signals: Skill payload doesn't match describe schema, skill/playbook divergence, skill documents nonexistent topology paths, missing event types (compare emission guide), retry-based field discovery, runbook/skill contradictions, compactGuidance drift.

Action: File docs issue with file:line, the discrepancy, and correct information from describe output.

Bucket 3: User Error

The agent misused a tool in a way both docs and describe output correctly describe.

Signals: Format mismatch (confirmed by describe + docs agreement), invalid sequence (topology confirms), missing context both skill and playbook prescribe, runbook deviation without justification.

Action: Note for skill improvement if errors are frequent.

Step 5: Generate Report

Produce the report using the template from references/report-template.md. Include:

• Summary counts per bucket
• Debug trace summary (workflows inspected, events reviewed, describe queries issued, views consulted)
• Each failure with full diagnosis (including authoritative self-service references)
• Trace-only findings section (issues only visible via server-side inspection)
• Playbook/runbook adherence summary
• Actionable next steps (draft issue bodies for bugs/docs issues)

Step 6: Offer to File Issues

For findings in the Code Bug and Documentation Issue buckets, offer to create GitHub issues:

exarchos_orchestrate({ action: "create_issue", title: "<type>: <summary>", body: "<issue body>", labels: ["bug"] })

Only file issues with user confirmation — present the draft first.

Required Output Format

{
  "session_summary": {
    "total_tool_calls": 0,
    "failed_tool_calls": 0,
    "failure_rate": "0%",
    "debug_trace": {
      "workflows_inspected": 0,
      "events_reviewed": 0,
      "describe_queries": 0,
      "views_consulted": [],
      "feedback_reports_reviewed": 0,
      "trace_only_findings": 0
    }
  },
  "playbook_adherence": {
    "phases_checked": 0,
    "violations": [
      {
        "phase": "delegate",
        "field": "events",
        "expected": "team.spawned, team.task.assigned",
        "actual": "none emitted",
        "bucket": "documentation_issue"
      }
    ]
  },
  "runbook_conformance": {
    "runbooks_checked": 0,
    "deviations": []
  },
  "buckets": {
    "code_bug": [],
    "documentation_issue": [],
    "user_error": []
  },
  "findings": [
    {
      "id": 1,
      "bucket": "code_bug | documentation_issue | user_error",
      "tool": "exarchos_workflow",
      "action": "update",
      "error": "INVALID_INPUT: ...",
      "root_cause": "Schema rejects null branch on pending tasks",
      "trace_evidence": "describe(actions: ['update']) shows branch as required string; event log confirms no task.updated event",
      "authoritative_ref": "exarchos_workflow describe(actions: ['update']) → TaskSchema",
      "severity": "HIGH | MEDIUM | LOW",
      "suggested_fix": "Accept nullable branch in TaskSchema",
      "issue_draft": {
        "title": "bug: workflow task schema rejects null branch",
        "labels": ["bug"],
        "body": "..."
      }
    }
  ],
  "trace_only_findings": [
    {
      "id": "T1",
      "description": "State drift: agent assumed phase was 'delegate' but server shows 'plan'",
      "evidence": "exarchos_workflow get shows phase=plan; topology confirms plan→delegate requires planReviewComplete guard",
      "authoritative_ref": "exarchos_workflow describe(topology: 'feature') → guards",
      "bucket": "documentation_issue",
      "suggested_fix": "Skill should instruct agent to verify phase via get before proceeding"
    }
  ]
}

Anti-Patterns