Audit Guide

Audit Guide

Run a comprehensive read-only audit of an existing guide content.json and produce a single prioritised report. This skill orchestrates three independent sub-agent analyses — structural, semantic, adversarial — and synthesises them.

Read-only on the guide. This skill never writes to content.json or manifest.json. It only writes to assets/.

Do NOT read external reference files upfront. Each phase's sub-agent loads its own references. Everything the orchestrator needs is inline below.

This skill implements the skill-memory convention. See ../skill-memory.md for the shared assets/, manifest, and frontmatter standards.

Workflow Overview

Input (guide directory path)
  │
  ├─ Phase 0: Check for Prior Run ── orchestrator
  │    Compares hash of content.json against assets/manifest.yaml
  │
  ├─ Phase 1: Acquire & Parse ───── orchestrator (no external reads)
  │    Reads: content.json, manifest.json (if present)
  │    Writes: assets/audit-input.json (flattened block tree with paths)
  │
  ├─ Phase 2: Structural Audit ──── sub-agent loads .cursor/commands/lint.md
  │    │                              + ../shared/critical-rules.md
  │    Writes: assets/structural-report.md
  │
  ├─ Phase 3: Semantic Audit ────── sub-agent loads .cursor/commands/check.md
  │    │                              + .cursor/authoring-guide.mdc
  │    │                              + .cursor/best-practices.mdc
  │    │                              + AGENTS.md (critical rules)
  │    Writes: assets/semantic-report.md
  │
  ├─ Phase 4: Adversarial Audit ─── sub-agent loads .cursor/commands/attack.md
  │    │                              + .cursor/edge-cases-and-troubleshooting.mdc
  │    Writes: assets/adversarial-report.md
  │
  └─ Phase 5: Synthesise ────────── orchestrator
       Merges the three raw reports → assets/audit-report.md
       Writes/updates assets/manifest.yaml

When focus area is provided, the orchestrator skips Phase 2/3/4 phases not in the focus and synthesises a focused report. See Inputs below.

Inputs

• Required: guide_dir — path to a directory containing content.json (e.g., alerting-101/).
• Optional: severity_floor — errors | warnings | all (default all). Findings below the floor are omitted from the synthesised report.
• Optional: focus — structural | selectors | lazy-render | requirements | editorial. When set, skip phases that don't contribute to that focus. Mapping:
  • structural → Phase 2 only
  • selectors → Phase 2 + Phase 4
  • lazy-render → Phase 2 + Phase 4
  • requirements → Phase 3
  • editorial → Phase 3 + Phase 4
  • (unset) → all phases

If the user provides no arguments, ask them for guide_dir and offer the defaults for the other two.

Critical Rules

Read ../shared/critical-rules.md for the canonical 18 + 3 critical rules. Phase 2 and Phase 3 sub-agents must cite rule numbers when reporting findings. The shared file forwards to ../../../AGENTS.md — that is the authoritative list.

Severity Rubric

Read severity-rubric.md for the canonical severity definitions used by all three audit phases. The rubric defines what each phase classifies as blocking, warning, or info and how Phase 5 prioritises across the three reports.

Generated Files

The skill's deliverable is assets/audit-report.md. Everything else in assets/ is intermediate.

{guide_dir}/
  content.json                    ← INPUT (never modified)
  manifest.json                   ← INPUT (never modified, may be absent)
  assets/
    manifest.yaml                 ← skill memory (hash, focus, severity_floor)
    audit-input.json              ← flattened block tree (Phase 1)
    structural-report.md          ← Phase 2 raw output
    semantic-report.md            ← Phase 3 raw output
    adversarial-report.md         ← Phase 4 raw output
    audit-report.md               ← Phase 5 deliverable (prioritised findings)

Frontmatter disclaimer

Every generated markdown file in assets/ must start with:

---
disclaimer: Auto-generated by audit-guide skill. Do not edit manually.
notice: To regenerate, re-run the skill against this guide.
input_sha256: {hex digest}
guide_id: {id from content.json}
---

Manifest file

After Phase 5, write {guide_dir}/assets/manifest.yaml:

# Auto-generated by audit-guide skill. Do not edit manually.
schema_version: 1
skill_name: "audit-guide"
generated_at: "{ISO 8601 timestamp}"
input_sha256: "{hex digest — see below}"

guide:
  id: "{guide id from content.json}"
  total_blocks: {N}
  total_interactive_steps: {N}

audit:
  focus: "{focus or 'all'}"
  severity_floor: "{errors|warnings|all}"
  finding_counts:
    blocking: {N}
    warning: {N}
    info: {N}

files:
  audit_input: "assets/audit-input.json"
  structural_report: "assets/structural-report.md"
  semantic_report: "assets/semantic-report.md"
  adversarial_report: "assets/adversarial-report.md"
  audit_report: "assets/audit-report.md"

Compute input_sha256 by hashing the raw content.json content:

shasum -a 256 {guide_dir}/content.json | awk '{print $1}'

This means re-running the audit against the same content.json matches the prior run and triggers warm-start. Re-running after any edit (even cosmetic) triggers a fresh audit.

Phase 0: Check for Prior Run (Orchestrator)

Before starting the audit pipeline, check whether this guide has been audited before.

1. Check whether {guide_dir}/assets/manifest.yaml exists.
2. If it does: read the manifest, extract input_sha256 and generated_at.
3. Re-hash the current content.json: shasum -a 256 {guide_dir}/content.json | awk '{print $1}'.
4. Compare hashes:
   • Match → no drift since the prior audit. Tell the user the audit is up to date (cite generated_at) and offer three options: (a) view the existing audit-report.md, (b) re-run a specific phase (Phase 2/3/4) only, (c) proceed with a full fresh audit anyway.
   • Mismatch → drift detected; tell the user the guide has changed since {generated_at}. Move prior reports to assets/history/{generated_at}/ (preserve audit trail), then proceed with a fresh full audit.
5. If manifest.yaml does not exist: fresh run — proceed to Phase 1.

If the user picks "view the existing report", read assets/audit-report.md and surface its top-level findings inline; do not re-run any phase.

Phase 1: Acquire & Parse (Orchestrator)

1. Verify {guide_dir}/content.json exists and is valid JSON. If not, report the parse error and stop.

2. Read {guide_dir}/manifest.json if present (informational only — Phase 3 may reference it).

3. Walk the block tree and produce a flattened representation at assets/audit-input.json. Every block gets a stable path string:

   • Top-level blocks: blocks[i]
   • Nested in a section: blocks[i].blocks[j]
   • Nested in a conditional: blocks[i].whenTrue[j] or blocks[i].whenFalse[j]
   • Steps inside multistep/guided: blocks[i].steps[k]

   Shape of each entry:

   {
     "path": "blocks[2].blocks[0]",
     "type": "interactive",
     "id": "open-dashboards",
     "action": "highlight",
     "reftarget": "a[data-testid='...']",
     "requirements": ["navmenu-open"],
     "objectives": [],
     "lazyRender": false,
     "skippable": false
   }
   

4. Create {guide_dir}/assets/ if it doesn't exist.

5. Determine which phases to run based on focus. Default: run all three.

Phase 2: Structural Audit (Sub-agent)

Skip if focus is set to a value that doesn't include structural (requirements, editorial).

Dispatch a sub-agent (Explore subagent type) with the following brief:

You are the **Structural Audit** phase of the audit-guide skill. Read the following files in order, then produce a structured report.

**Read (in order):**

1. [.cursor/commands/lint.md](../../../.cursor/commands/lint.md) — the lint checklist (this is your primary rule source)
2. [../shared/critical-rules.md](../shared/critical-rules.md) — the canonical 21 critical rules (cited by number in findings)
3. [./severity-rubric.md](./severity-rubric.md) — how to classify findings as blocking / warning / info
4. {guide_dir}/assets/audit-input.json — the flattened block tree you are auditing

**Apply every check in lint.md against every entry in audit-input.json.** Also apply:

- Critical rule 4 (no multistep singletons) — flag any `multistep` block with `len(steps) == 1`
- Critical rule 17 (no focus-before-formfill) — flag any `interactive` with `action: "highlight"` targeting an input/textarea selector with `doIt` true or unset
- Critical rule 18 (`schemaVersion` is optional but if set must be `"1.1.0"`)
- Critical rule 19 (`popout` requires `targetvalue` of exactly `"sidebar"` or `"floating"`)
- Critical rule 20 (`navigate` actions chaining to another guide should use `openGuide` not `?doc=`)
- Critical rule 21 (virtualised targets need `guided` + `lazyRender: true`) — heuristic: any step targeting a selector containing `[data-cy='wb-list-item']`, `tr[data-row-id`, table-row selectors, or `:contains(` text-match patterns in a list context is a candidate. Flag plain `interactive` blocks with such targets.

**Output: a structured markdown report at `{guide_dir}/assets/structural-report.md`** with the standard frontmatter disclaimer and these sections:

```markdown
# Structural Audit

## Summary

- Total findings: N (blocking: N, warning: N, info: N)
- Total blocks audited: N

## Blocking

For each finding:
- **{path}** — {short title}
- Rule: critical rule {number} OR lint.md item {number}
- Detail: {one-sentence explanation}
- Suggested fix: {concrete next step}

## Warning

(same shape)

## Info

(same shape)
```

Cite block paths exactly as they appear in `audit-input.json` (`blocks[i].blocks[j]` etc.). Cite rule numbers from the shared critical rules file. Do not invent rules.

If you find zero findings in a section, omit the section header (don't write empty sections).

**Stop after writing the report. Do not modify content.json. Do not run further phases — the orchestrator handles synthesis.**

The sub-agent writes the report. The orchestrator then proceeds to Phase 3.

Phase 3: Semantic Audit (Sub-agent)

Skip if focus is set to a value that doesn't include semantic (structural, selectors, lazy-render without requirements).

Dispatch a sub-agent (Explore subagent type) with the following brief:

You are the **Semantic Audit** phase of the audit-guide skill. Read the following files in order, then produce a structured report.

**Read (in order):**

1. [.cursor/commands/check.md](../../../.cursor/commands/check.md) — the check checklist
2. [.cursor/authoring-guide.mdc](../../../.cursor/authoring-guide.mdc) — best practices and code smells
3. [.cursor/best-practices.mdc](../../../.cursor/best-practices.mdc) — decision trees, canonical code smells (§6), quality checklist (§7)
4. [../../../AGENTS.md](../../../AGENTS.md) — canonical 21 critical rules (cited by number in findings)
5. [.cursor/docs/requirements-reference.md](../../../docs/requirements-reference.md) — for chain validation
6. [./severity-rubric.md](./severity-rubric.md) — how to classify findings
7. {guide_dir}/assets/audit-input.json — the flattened block tree

**Apply (in order):**

- Every code smell in best-practices.mdc §6 (15 canonical items)
- Every item in best-practices.mdc §7 quality checklist
- Critical rules 1-21 from AGENTS.md — flag any violation, citing the rule number
- Requirements/objectives chain logic: if section N has an objective `has-datasource:X`, section N+1's steps that need X should require it. If section M has a `section-completed:S` requirement, verify section S exists and precedes M in the block tree.
- Variable references: every `var-<name>:<value>` requirement must correspond to an `input` block earlier in the guide with `variableName: <name>`. Every `{{<name>}}` in markdown content must correspond to an `input` block.
- First-action page anchor: the first interactive action should have `on-page:/path` or be a `navigate` action.
- Tooltip quality: tooltips must be ≤ 250 chars, single sentence, and not name the element being highlighted.
- Section bookends (rule 14): each `section` should have a 1-sentence intro markdown **immediately before** it and a 1-sentence summary **immediately after** it in the parent `blocks` array. Do **not** flag missing *in-section* intro when outside bookends exist. You may treat a pre-section "To …, complete the following steps:" line as satisfying the intro.
- Section intro markdown that may number as a step: if the first block inside a `section` is `markdown` that starts with "You'll " / "You will " or is a one-sentence action preview before the first interactive, flag as **Warning**. Cite the block path. Note that Block Editor may number this as step 1; LP review promotes to inline after live confirm. Fix guidance: move the intro outside the section (rule 14 placement).
- False noops: `action: "noop"` whose content tells the learner to click, type, open, or select UI without a `reftarget` — flag as **Warning** (prefer `markdown` or a real highlight).

**Output: a structured markdown report at `{guide_dir}/assets/semantic-report.md`** with the standard frontmatter disclaimer and the same Summary / Blocking / Warning / Info section shape as the structural report. Cite rule numbers and block paths.

**Stop after writing the report. Do not modify content.json.**

Phase 4: Adversarial Audit (Sub-agent)

Skip if focus is structural or requirements.

Dispatch a sub-agent (Explore subagent type) with the following brief:

You are the **Adversarial Audit** phase of the audit-guide skill. Read the following files in order, then simulate a confused user.

**Read (in order):**

1. [.cursor/commands/attack.md](../../../.cursor/commands/attack.md) — the attack protocol
2. [.cursor/edge-cases-and-troubleshooting.mdc](../../../.cursor/edge-cases-and-troubleshooting.mdc) — known edge cases
3. [./severity-rubric.md](./severity-rubric.md) — how to classify findings
4. {guide_dir}/assets/audit-input.json — the flattened block tree
5. {guide_dir}/content.json — the original guide (for prose content, which audit-input.json strips)

**Simulate at least these user profiles:**

- An impatient user who clicks "Do it" before reading prose
- A user who navigates away mid-guide and comes back
- A user on a mobile/narrow viewport (long tooltips wrap, virtualised lists scroll)
- A user without admin rights attempting admin-only steps
- A user who has already completed the setup the guide describes
- A user who fills the wrong value into a form field

For each profile, walk through the guide step by step, noting what would go wrong. Categorise findings:

- **Blocking**: the guide hard-fails (e.g., admin step without `skippable`, popout with bad `targetvalue`)
- **Warning**: the guide is confusing or recoverable (e.g., tooltip names the element being highlighted, missing `verify` on a save action)
- **Info**: nit-level UX suggestions

**Output: a structured markdown report at `{guide_dir}/assets/adversarial-report.md`** with the standard frontmatter disclaimer and the same Summary / Blocking / Warning / Info section shape. Group findings by user profile within each severity.

**Stop after writing the report. Do not modify content.json.**

Phase 5: Synthesise (Orchestrator)

After all dispatched phases complete:

1. Read the three raw reports (or fewer if focus skipped some).
2. Build a single prioritised report at {guide_dir}/assets/audit-report.md with frontmatter disclaimer and these sections:

# Audit Report — {guide id}

## Summary

- Audit phases run: {structural, semantic, adversarial}
- Total findings: N (blocking: N, warning: N, info: N) — at severity floor `{floor}`
- Verdict: {ready to merge | needs revision | blocking issues}

## Blocking (must fix before merge)

Cross-phase merged findings sorted by block path. For each:
- **{path}** — {short title}
- Reported by: {phase(s)}
- Rule: {citation}
- Detail: {one-sentence}
- Suggested fix: {concrete next step}

(Deduplicate: if structural and semantic both flag the same block path for the same rule, list it once and credit both phases.)

## Structural

(remaining structural findings not in Blocking)

## Semantic

(remaining semantic findings not in Blocking)

## Adversarial

(remaining adversarial findings not in Blocking)

## Suggestions

Info-level items from all three phases, grouped by theme.

## Raw reports

- [Structural](structural-report.md)
- [Semantic](semantic-report.md)
- [Adversarial](adversarial-report.md)

3. Apply severity_floor: omit findings below the floor (e.g., if floor is warnings, drop all info-level findings; if errors, drop info and warning).
4. Write/update {guide_dir}/assets/manifest.yaml with the schema documented above.
5. Surface a one-paragraph summary to the user: total findings, blocking count, and the path to audit-report.md. Do not paste the full report inline unless the user asks.

When to use vs. skip

Use audit-guide when:

• A guide is about to be merged and you want one comprehensive pre-merge review.
• You're returning to a guide after upstream Pathfinder changes (e.g., new critical rules added) and want to know what's stale.
• A guide is failing in production and you want a structured triage.

Skip audit-guide and use the lighter /lint, /check, or /attack commands directly when:

• You only need one of the three perspectives.
• You're iterating quickly and don't want the asset directory.
• The guide is new and unstable — run audit-guide once it has stabilised.

See Also

• autogen-guide/SKILL.md — generate a guide from source code (companion skill for new guides)
• update-guide/SKILL.md — modify an existing guide (companion skill for changes)
• migrate-guide/SKILL.md — migrate to the package format
• ../skill-memory.md — shared skill memory convention