# Audit Guide

> Comprehensive read-only audit of an existing Pathfinder guide. Combines structural (lint), semantic (check + best-practices decision trees + the 21 critical rules), and adversarial (attack) analysis into one phased workflow that produces a single prioritised report. Use when the user asks to audit, review, lint+check, or check the quality of an existing guide directory.

- Skill: `grafana/audit-guide` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add grafana/audit-guide`
- Raw SKILL.md: https://api.skillmd.com/api/skills/grafana/audit-guide/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Security
- Author: grafana (https://skillmd.com/u/grafana)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/grafana/audit-guide

---


# Audit Guide

Run a comprehensive read-only audit of an existing guide `content.json` and produce a single prioritised report. This skill orchestrates three independent sub-agent analyses — structural, semantic, adversarial — and synthesises them.

**Read-only on the guide.** This skill never writes to `content.json` or `manifest.json`. It only writes to `assets/`.

**Do NOT read external reference files upfront.** Each phase's sub-agent loads its own references. Everything the orchestrator needs is inline below.

This skill implements the **skill-memory convention**. See [../skill-memory.md](../skill-memory.md) for the shared `assets/`, manifest, and frontmatter standards.

---

## Workflow Overview

```
Input (guide directory path)
  │
  ├─ Phase 0: Check for Prior Run ── orchestrator
  │    Compares hash of content.json against assets/manifest.yaml
  │
  ├─ Phase 1: Acquire & Parse ───── orchestrator (no external reads)
  │    Reads: content.json, manifest.json (if present)
  │    Writes: assets/audit-input.json (flattened block tree with paths)
  │
  ├─ Phase 2: Structural Audit ──── sub-agent loads .cursor/commands/lint.md
  │    │                              + ../shared/critical-rules.md
  │    Writes: assets/structural-report.md
  │
  ├─ Phase 3: Semantic Audit ────── sub-agent loads .cursor/commands/check.md
  │    │                              + .cursor/authoring-guide.mdc
  │    │                              + .cursor/best-practices.mdc
  │    │                              + AGENTS.md (critical rules)
  │    Writes: assets/semantic-report.md
  │
  ├─ Phase 4: Adversarial Audit ─── sub-agent loads .cursor/commands/attack.md
  │    │                              + .cursor/edge-cases-and-troubleshooting.mdc
  │    Writes: assets/adversarial-report.md
  │
  └─ Phase 5: Synthesise ────────── orchestrator
       Merges the three raw reports → assets/audit-report.md
       Writes/updates assets/manifest.yaml
```

When `focus area` is provided, the orchestrator skips Phase 2/3/4 phases not in the focus and synthesises a focused report. See **Inputs** below.

---

## Inputs

- **Required**: `guide_dir` — path to a directory containing `content.json` (e.g., `alerting-101/`).
- **Optional**: `severity_floor` — `errors` | `warnings` | `all` (default `all`). Findings below the floor are omitted from the synthesised report.
- **Optional**: `focus` — `structural` | `selectors` | `lazy-render` | `requirements` | `editorial`. When set, skip phases that don't contribute to that focus. Mapping:
  - `structural` → Phase 2 only
  - `selectors` → Phase 2 + Phase 4
  - `lazy-render` → Phase 2 + Phase 4
  - `requirements` → Phase 3
  - `editorial` → Phase 3 + Phase 4
  - (unset) → all phases

If the user provides no arguments, ask them for `guide_dir` and offer the defaults for the other two.

---

## Critical Rules

Read [../shared/critical-rules.md](../shared/critical-rules.md) for the canonical 18 + 3 critical rules. Phase 2 and Phase 3 sub-agents must cite rule numbers when reporting findings. The shared file forwards to [../../../AGENTS.md](../../../AGENTS.md) — that is the authoritative list.

---

## Severity Rubric

Read [severity-rubric.md](severity-rubric.md) for the canonical severity definitions used by all three audit phases. The rubric defines what each phase classifies as **blocking**, **warning**, or **info** and how Phase 5 prioritises across the three reports.

---

## Generated Files

The skill's deliverable is `assets/audit-report.md`. Everything else in `assets/` is intermediate.

```
{guide_dir}/
  content.json                    ← INPUT (never modified)
  manifest.json                   ← INPUT (never modified, may be absent)
  assets/
    manifest.yaml                 ← skill memory (hash, focus, severity_floor)
    audit-input.json              ← flattened block tree (Phase 1)
    structural-report.md          ← Phase 2 raw output
    semantic-report.md            ← Phase 3 raw output
    adversarial-report.md         ← Phase 4 raw output
    audit-report.md               ← Phase 5 deliverable (prioritised findings)
```

### Frontmatter disclaimer

Every generated markdown file in `assets/` **must** start with:

```markdown
---
disclaimer: Auto-generated by audit-guide skill. Do not edit manually.
notice: To regenerate, re-run the skill against this guide.
input_sha256: {hex digest}
guide_id: {id from content.json}
---
```

### Manifest file

After Phase 5, write `{guide_dir}/assets/manifest.yaml`:

```yaml
# Auto-generated by audit-guide skill. Do not edit manually.
schema_version: 1
skill_name: "audit-guide"
generated_at: "{ISO 8601 timestamp}"
input_sha256: "{hex digest — see below}"

guide:
  id: "{guide id from content.json}"
  total_blocks: {N}
  total_interactive_steps: {N}

audit:
  focus: "{focus or 'all'}"
  severity_floor: "{errors|warnings|all}"
  finding_counts:
    blocking: {N}
    warning: {N}
    info: {N}

files:
  audit_input: "assets/audit-input.json"
  structural_report: "assets/structural-report.md"
  semantic_report: "assets/semantic-report.md"
  adversarial_report: "assets/adversarial-report.md"
  audit_report: "assets/audit-report.md"
```

Compute `input_sha256` by hashing the raw `content.json` content:

```bash
shasum -a 256 {guide_dir}/content.json | awk '{print $1}'
```

This means re-running the audit against the same `content.json` matches the prior run and triggers warm-start. Re-running after any edit (even cosmetic) triggers a fresh audit.

---

## Phase 0: Check for Prior Run (Orchestrator)

Before starting the audit pipeline, check whether this guide has been audited before.

1. Check whether `{guide_dir}/assets/manifest.yaml` exists.
2. If it does: read the manifest, extract `input_sha256` and `generated_at`.
3. Re-hash the current `content.json`: `shasum -a 256 {guide_dir}/content.json | awk '{print $1}'`.
4. Compare hashes:
   - **Match** → no drift since the prior audit. Tell the user the audit is up to date (cite `generated_at`) and offer three options: (a) view the existing `audit-report.md`, (b) re-run a specific phase (Phase 2/3/4) only, (c) proceed with a full fresh audit anyway.
   - **Mismatch** → drift detected; tell the user the guide has changed since `{generated_at}`. Move prior reports to `assets/history/{generated_at}/` (preserve audit trail), then proceed with a fresh full audit.
5. If `manifest.yaml` does not exist: fresh run — proceed to Phase 1.

If the user picks "view the existing report", read `assets/audit-report.md` and surface its top-level findings inline; do not re-run any phase.

---

## Phase 1: Acquire & Parse (Orchestrator)

1. Verify `{guide_dir}/content.json` exists and is valid JSON. If not, report the parse error and stop.
2. Read `{guide_dir}/manifest.json` if present (informational only — Phase 3 may reference it).
3. Walk the block tree and produce a flattened representation at `assets/audit-input.json`. Every block gets a stable path string:
   - Top-level blocks: `blocks[i]`
   - Nested in a section: `blocks[i].blocks[j]`
   - Nested in a conditional: `blocks[i].whenTrue[j]` or `blocks[i].whenFalse[j]`
   - Steps inside multistep/guided: `blocks[i].steps[k]`

   Shape of each entry:

   ```json
   {
     "path": "blocks[2].blocks[0]",
     "type": "interactive",
     "id": "open-dashboards",
     "action": "highlight",
     "reftarget": "a[data-testid='...']",
     "requirements": ["navmenu-open"],
     "objectives": [],
     "lazyRender": false,
     "skippable": false
   }
   ```

4. Create `{guide_dir}/assets/` if it doesn't exist.
5. Determine which phases to run based on `focus`. Default: run all three.

---

## Phase 2: Structural Audit (Sub-agent)

Skip if `focus` is set to a value that doesn't include structural (`requirements`, `editorial`).

Dispatch a sub-agent (Explore subagent type) with the following brief:

````
You are the **Structural Audit** phase of the audit-guide skill. Read the following files in order, then produce a structured report.

**Read (in order):**

1. [.cursor/commands/lint.md](../../../.cursor/commands/lint.md) — the lint checklist (this is your primary rule source)
2. [../shared/critical-rules.md](../shared/critical-rules.md) — the canonical 21 critical rules (cited by number in findings)
3. [./severity-rubric.md](./severity-rubric.md) — how to classify findings as blocking / warning / info
4. {guide_dir}/assets/audit-input.json — the flattened block tree you are auditing

**Apply every check in lint.md against every entry in audit-input.json.** Also apply:

- Critical rule 4 (no multistep singletons) — flag any `multistep` block with `len(steps) == 1`
- Critical rule 17 (no focus-before-formfill) — flag any `interactive` with `action: "highlight"` targeting an input/textarea selector with `doIt` true or unset
- Critical rule 18 (`schemaVersion` is optional but if set must be `"1.1.0"`)
- Critical rule 19 (`popout` requires `targetvalue` of exactly `"sidebar"` or `"floating"`)
- Critical rule 20 (`navigate` actions chaining to another guide should use `openGuide` not `?doc=`)
- Critical rule 21 (virtualised targets need `guided` + `lazyRender: true`) — heuristic: any step targeting a selector containing `[data-cy='wb-list-item']`, `tr[data-row-id`, table-row selectors, or `:contains(` text-match patterns in a list context is a candidate. Flag plain `interactive` blocks with such targets.

**Output: a structured markdown report at `{guide_dir}/assets/structural-report.md`** with the standard frontmatter disclaimer and these sections:

```markdown
# Structural Audit

## Summary

- Total findings: N (blocking: N, warning: N, info: N)
- Total blocks audited: N

## Blocking

For each finding:
- **{path}** — {short title}
- Rule: critical rule {number} OR lint.md item {number}
- Detail: {one-sentence explanation}
- Suggested fix: {concrete next step}

## Warning

(same shape)

## Info

(same shape)
```

Cite block paths exactly as they appear in `audit-input.json` (`blocks[i].blocks[j]` etc.). Cite rule numbers from the shared critical rules file. Do not invent rules.

If you find zero findings in a section, omit the section header (don't write empty sections).

**Stop after writing the report. Do not modify content.json. Do not run further phases — the orchestrator handles synthesis.**
````

The sub-agent writes the report. The orchestrator then proceeds to Phase 3.

---

## Phase 3: Semantic Audit (Sub-agent)

Skip if `focus` is set to a value that doesn't include semantic (`structural`, `selectors`, `lazy-render` without `requirements`).

Dispatch a sub-agent (Explore subagent type) with the following brief:

````
You are the **Semantic Audit** phase of the audit-guide skill. Read the following files in order, then produce a structured report.

**Read (in order):**

1. [.cursor/commands/check.md](../../../.cursor/commands/check.md) — the check checklist
2. [.cursor/authoring-guide.mdc](../../../.cursor/authoring-guide.mdc) — best practices and code smells
3. [.cursor/best-practices.mdc](../../../.cursor/best-practices.mdc) — decision trees, canonical code smells (§6), quality checklist (§7)
4. [../../../AGENTS.md](../../../AGENTS.md) — canonical 21 critical rules (cited by number in findings)
5. [.cursor/docs/requirements-reference.md](../../../docs/requirements-reference.md) — for chain validation
6. [./severity-rubric.md](./severity-rubric.md) — how to classify findings
7. {guide_dir}/assets/audit-input.json — the flattened block tree

**Apply (in order):**

- Every code smell in best-practices.mdc §6 (15 canonical items)
- Every item in best-practices.mdc §7 quality checklist
- Critical rules 1-21 from AGENTS.md — flag any violation, citing the rule number
- Requirements/objectives chain logic: if section N has an objective `has-datasource:X`, section N+1's steps that need X should require it. If section M has a `section-completed:S` requirement, verify section S exists and precedes M in the block tree.
- Variable references: every `var-<name>:<value>` requirement must correspond to an `input` block earlier in the guide with `variableName: <name>`. Every `{{<name>}}` in markdown content must correspond to an `input` block.
- First-action page anchor: the first interactive action should have `on-page:/path` or be a `navigate` action.
- Tooltip quality: tooltips must be ≤ 250 chars, single sentence, and not name the element being highlighted.
- Section bookends (rule 14): each `section` should have a 1-sentence intro markdown **immediately before** it and a 1-sentence summary **immediately after** it in the parent `blocks` array. Do **not** flag missing *in-section* intro when outside bookends exist. You may treat a pre-section "To …, complete the following steps:" line as satisfying the intro.
- Section intro markdown that may number as a step: if the first block inside a `section` is `markdown` that starts with "You'll " / "You will " or is a one-sentence action preview before the first interactive, flag as **Warning**. Cite the block path. Note that Block Editor may number this as step 1; LP review promotes to inline after live confirm. Fix guidance: move the intro outside the section (rule 14 placement).
- False noops: `action: "noop"` whose content tells the learner to click, type, open, or select UI without a `reftarget` — flag as **Warning** (prefer `markdown` or a real highlight).

**Output: a structured markdown report at `{guide_dir}/assets/semantic-report.md`** with the standard frontmatter disclaimer and the same Summary / Blocking / Warning / Info section shape as the structural report. Cite rule numbers and block paths.

**Stop after writing the report. Do not modify content.json.**
````

---

## Phase 4: Adversarial Audit (Sub-agent)

Skip if `focus` is `structural` or `requirements`.

Dispatch a sub-agent (Explore subagent type) with the following brief:

````
You are the **Adversarial Audit** phase of the audit-guide skill. Read the following files in order, then simulate a confused user.

**Read (in order):**

1. [.cursor/commands/attack.md](../../../.cursor/commands/attack.md) — the attack protocol
2. [.cursor/edge-cases-and-troubleshooting.mdc](../../../.cursor/edge-cases-and-troubleshooting.mdc) — known edge cases
3. [./severity-rubric.md](./severity-rubric.md) — how to classify findings
4. {guide_dir}/assets/audit-input.json — the flattened block tree
5. {guide_dir}/content.json — the original guide (for prose content, which audit-input.json strips)

**Simulate at least these user profiles:**

- An impatient user who clicks "Do it" before reading prose
- A user who navigates away mid-guide and comes back
- A user on a mobile/narrow viewport (long tooltips wrap, virtualised lists scroll)
- A user without admin rights attempting admin-only steps
- A user who has already completed the setup the guide describes
- A user who fills the wrong value into a form field

For each profile, walk through the guide step by step, noting what would go wrong. Categorise findings:

- **Blocking**: the guide hard-fails (e.g., admin step without `skippable`, popout with bad `targetvalue`)
- **Warning**: the guide is confusing or recoverable (e.g., tooltip names the element being highlighted, missing `verify` on a save action)
- **Info**: nit-level UX suggestions

**Output: a structured markdown report at `{guide_dir}/assets/adversarial-report.md`** with the standard frontmatter disclaimer and the same Summary / Blocking / Warning / Info section shape. Group findings by user profile within each severity.

**Stop after writing the report. Do not modify content.json.**
````

---

## Phase 5: Synthesise (Orchestrator)

After all dispatched phases complete:

1. Read the three raw reports (or fewer if `focus` skipped some).
2. Build a single prioritised report at `{guide_dir}/assets/audit-report.md` with frontmatter disclaimer and these sections:

```markdown
# Audit Report — {guide id}

## Summary

- Audit phases run: {structural, semantic, adversarial}
- Total findings: N (blocking: N, warning: N, info: N) — at severity floor `{floor}`
- Verdict: {ready to merge | needs revision | blocking issues}

## Blocking (must fix before merge)

Cross-phase merged findings sorted by block path. For each:
- **{path}** — {short title}
- Reported by: {phase(s)}
- Rule: {citation}
- Detail: {one-sentence}
- Suggested fix: {concrete next step}

(Deduplicate: if structural and semantic both flag the same block path for the same rule, list it once and credit both phases.)

## Structural

(remaining structural findings not in Blocking)

## Semantic

(remaining semantic findings not in Blocking)

## Adversarial

(remaining adversarial findings not in Blocking)

## Suggestions

Info-level items from all three phases, grouped by theme.

## Raw reports

- [Structural](structural-report.md)
- [Semantic](semantic-report.md)
- [Adversarial](adversarial-report.md)
```

3. Apply `severity_floor`: omit findings below the floor (e.g., if floor is `warnings`, drop all info-level findings; if `errors`, drop info and warning).
4. Write/update `{guide_dir}/assets/manifest.yaml` with the schema documented above.
5. Surface a one-paragraph summary to the user: total findings, blocking count, and the path to `audit-report.md`. Do not paste the full report inline unless the user asks.

---

## When to use vs. skip

Use audit-guide when:

- A guide is about to be merged and you want one comprehensive pre-merge review.
- You're returning to a guide after upstream Pathfinder changes (e.g., new critical rules added) and want to know what's stale.
- A guide is failing in production and you want a structured triage.

Skip audit-guide and use the lighter `/lint`, `/check`, or `/attack` commands directly when:

- You only need one of the three perspectives.
- You're iterating quickly and don't want the asset directory.
- The guide is new and unstable — run audit-guide once it has stabilised.

---

## See Also

- [autogen-guide/SKILL.md](../autogen-guide/SKILL.md) — generate a guide from source code (companion skill for *new* guides)
- [update-guide/SKILL.md](../update-guide/SKILL.md) — modify an existing guide (companion skill for *changes*)
- [migrate-guide/SKILL.md](../migrate-guide/SKILL.md) — migrate to the package format
- [../skill-memory.md](../skill-memory.md) — shared skill memory convention