Documentation Audit

/documentation-audit

Audit a TypeScript project's documentation against an opinionated baseline organised in four layers — project entry and onboarding, architectural and decision documentation, code-level documentation, operational documentation and drift — preceded by a diagnostic snapshot. Then offer to generate an implementation plan for the gaps.

The default mental model is a TypeScript and React application, but most checks apply equally to any TypeScript project (libraries, services, monorepos). The operational layer adapts to the project shape — library-only projects silently skip the deployment, rollback, and monitoring checks while still running drift detection.

How this differs from neighbouring audits

Static-first design with optional link-check enrichment

This skill is read-only. Two modes:

• Static (default). Read every Markdown file in the repository (.md, .mdx, excluding node_modules and ignored paths), package.json, source files (for TSDoc/JSDoc and TODO detection), and Git metadata for last-modified timestamps. Internal and relative links are always checked against the file system regardless of mode — no network needed for that.
• Static plus opt-in --with-link-check. Additionally HEAD-request external links found in documentation files to verify they resolve (HTTP 2xx or 3xx). The skill never POSTs, never authenticates, never follows redirects beyond the HEAD response. Timeouts and transient failures are tolerated; persistent failures across the run are reported.

The skill never modifies any documentation, source, or configuration file.

Usage

/documentation-audit                                # default: concise Top 5 + full report saved + ask about plan
/documentation-audit --worktree                          # create an isolated Git worktree, then run the audit there
/documentation-audit --learn                        # mid-level engineer teaching mode (detailed explanations + file/line examples)
/documentation-audit --teach                        # alias for --learn
/documentation-audit --with-link-check              # static plus HEAD-checks of external URLs
/documentation-audit --threshold-readme-min-lines=50        # override default 30
/documentation-audit --threshold-adr-staleness-days=365     # override default 180
/documentation-audit --threshold-feature-doc-coverage=70    # override default 50 (percent for partial)
/documentation-audit --threshold-public-api-doc-coverage=80 # override default 60 (percent for present)
/documentation-audit --threshold-todo-staleness-days=180    # override default 90
/documentation-audit --threshold-doc-staleness-days=365     # override default 180

💡 Pro tip: Add --worktree to run this audit in an isolated Git worktree.

The skill never accepts --apply. The implementation plan is descriptive Markdown.

The opinionated baseline

A check resolves to one of four statuses:

• present — the invariant holds.
• partial — most signals resolve, with a small number of exceptions, or the codebase shows mixed adherence to a soft check.
• missing — a structural prerequisite is absent (no README at all, for example).
• violation — the audit identified concrete documentation or drift that breaks the invariant.

Layer 0 is informational only and has no status. Layer 4's operational checks (deployment, rollback, monitoring, feature flags) are silently skipped when the project is library-only or no deployable target is detected.

Layer 0 — Diagnostic snapshot (always written, no pass/fail)

• Total documentation file count (.md and .mdx files, excluding node_modules and gitignored paths).
• Documentation-to-code line ratio (Markdown lines vs .ts plus .tsx lines).
• README presence and size in lines.
• LICENSE presence and detected SPDX identifier.
• ADR count and the most-recent ADR date.
• TODO/FIXME count, with breakdown by age bucket: less than 30 days, 30–90 days, more than 90 days.
• Detected Storybook (@storybook/* in devDependencies plus .storybook/ directory).
• Detected documentation-site framework: Docusaurus, Nextra, VitePress, MkDocs, Sphinx, custom, or none.
• Per-feature README coverage rate: number of feature folders with a README.md (or barrel-file documentation comment) divided by total feature folder count.
• External-link total count and, when --with-link-check is set, the count of unreachable external links.
• Project shape (informs the operational layer): deployable application, library, monorepo, or hybrid.

Layer 1 — Project entry and onboarding

Layer 2 — Architectural and decision documentation

Layer 3 — Code-level documentation

The audit treats TSDoc and JSDoc as equivalent. TSDoc is a TypeScript-specific superset; the project picks one convention or mixes them, and the audit grades coverage and quality without preferring one over the other.

Layer 4 — Operational documentation and drift

The operational checks (deployment, rollback, monitoring, feature flags) require the project to ship to a deployable target. The audit detects project shape from configuration: a next.config.*, vercel.json, netlify.toml, Dockerfile, server entry point, or framework-conventional deployable layout signals a deployable application; a package.json with main/module/exports and no application entry signals a library. When the project is library-only, the operational checks are silently skipped. The drift checks always run, regardless of project shape.

What this skill does

1. Reads the knowledge graph when present. Soft dependency: when graphify-out/graph.json exists, the audit cross-references feature folders against graph centrality so the per-feature documentation check (and the implementation plan) prioritises the most-consumed features. Without the graph, the audit falls back to per-folder coverage rates with no centrality weighting.

2. Confirms a Node.js project. Detects package.json. If absent, the skill stops and tells the user it currently supports Node.js projects.

3. Detects project shape (deployable application, library, monorepo, hybrid) for the operational-layer dispatch.

4. Detects observability and feature-flag stacks (for layer 4 conditional checks): Sentry, Datadog, LaunchDarkly, GrowthBook, etc., from dependencies.

5. When --with-link-check is set, collects every external URL from documentation files and HEAD-requests them with a short timeout. Records reachability status per URL. Internal links and anchors are always checked against the file system.

6. Walks every documentation file to collect counts, headings, code blocks, and link references. Cross-references README command blocks against package.json scripts.

7. Walks every source file to collect TODO/FIXME comments (with Git blame for age), TSDoc/JSDoc coverage on public exports, and commented-out code blocks.

8. Writes Layer 0 — the diagnostic snapshot to .architect-audits/documentation-audit/snapshot.md and prepends the same content to findings.md.

9. Walks each check in the active layer list, applying any --include, --exclude, and threshold overrides. Records a status, evidence, and (where relevant) sample file references per check.

10. Writes phase 1 outputs to .architect-audits/documentation-audit/:

    • findings.md — diagnostic snapshot followed by check results, grouped by layer.
    • findings.json — machine-readable.
    • snapshot.md — diagnostic snapshot on its own.
    • metadata.json — skill version, run timestamp, Graphify revision (when present), project shape, observability stack, feature-flag stack, applied thresholds, applied filters, link-check flag and counts.

11. Phase 2 — offers to plan the gaps. Summarises the findings in chat and asks the user a single yes-or-no question:

    "Generate an implementation plan for the documentation gaps? (yes/no)"

    On yes, writes .architect-audits/documentation-audit/implementation-plan.md describing exactly which documents to create, which sections to add to existing docs, which TODOs to address (or close), which links to fix, and which TSDoc/JSDoc blocks to write — ordered by audience: onboarding documentation first (newcomer experience matters most), then architectural and operational, then code-level, then drift cleanup. The plan does not modify any project files.

    On no, exits cleanly.

Implementation steps

Step 1 — Confirm the prerequisites

test -f package.json || { echo "documentation-audit: no package.json detected. This skill currently supports Node.js projects only."; exit 1; }

Step 2 — Detect project shape

Inspect the repository for shape signals:

• Deployable application: next.config.*, vercel.json, netlify.toml, Dockerfile, app/ or pages/ directory, src/main.tsx plus a build script, server entry points.
• Library: package.json with main/module/exports fields and no application entry; absence of deployment configuration.
• Monorepo: packages/ directory plus a workspace declaration in package.json or pnpm-workspace.yaml.
• Hybrid: signals from multiple categories.

Record the detected shape in metadata.json. The shape gates layer 4's operational checks.

Step 3 — Detect observability and feature-flag stacks

Scan dependencies for @sentry/*, @datadog/*, @grafana/* (where applicable), launchdarkly, @growthbook/*, statsig-js, configcat-js, unleash-client. Record matches in metadata.json. The detection seeds the feature-flag-documentation check and the monitoring-documentation check.

Step 4 — Walk documentation files

Enumerate .md and .mdx files outside node_modules and .gitignore. For each:

• Total lines.
• Last-modified timestamp from Git (the most recent commit touching the file).
• Heading structure.
• Fenced code blocks (capture commands inside).
• Link references: classify as internal (relative path or anchor) or external (absolute URL).

Special handling for README.md: extract command blocks for cross-referencing against package.json scripts. Special handling for ADRs: detect the ADR directory and parse each file for the template-section headers.

Step 5 — Walk source files

Enumerate .ts and .tsx files. For each:

• Identify exported public-API symbols (functions, hooks, components, types) and whether they carry a TSDoc/JSDoc block.
• Find TODO/FIXME comments. For each, run git blame to get the most recent modification date.
• Find blocks of commented-out code (multiple consecutive comment lines that parse as TypeScript when uncommented).
• Find empty /** */ blocks.

Use Graphify communities to sample broadly when present; otherwise sweep all source files.

Step 6 — Optionally HEAD-check external links

When --with-link-check is set, collect the unique set of external URLs from step 4 and issue HEAD requests:

• Timeout per URL: 5 seconds.
• Concurrency: at most 10 in flight.
• Acceptable: HTTP 2xx, 3xx (without following redirects).
• Tolerate transient failures: retry once, then mark as unreachable.

Record per-URL status in metadata.

Step 7 — Build the diagnostic snapshot

Aggregate the collected data into the items listed in Layer 0. Write snapshot.md and prepend the same content to findings.md.

Step 8 — Resolve each check

For each check in the active layer list, walk its detection logic. Layer 4's operational checks are skipped silently when the detected project shape is library-only with no deployable signal. Threshold-bearing checks compare aggregated values to the configured thresholds.

For each check, record evidence and up to ten representative samples plus a total count.

Step 9 — Write phase 1 outputs

Create .architect-audits/documentation-audit/ if needed. Write findings.md, findings.json, snapshot.md, metadata.json. Overwrite previous runs of these four; preserve implementation-plan.md unless the user agrees to regenerate it.

Step 10 — Print the concise chat summary and offer phase 2

Print a human-first, scannable summary in the chat. Do not print the full layered findings — those are written to disk in Step 9. The chat output has exactly this shape:

1. Short header — audit name, timestamp, and a one-line summary of the codebase state.
2. Top 5 Highest-Leverage Recommendations — ordered by architectural principles: test philosophy, maintainability, risk reduction, velocity, long-term health. For fewer than five findings, print what exists. For each recommendation (numbered 1–5):
   • Title (one clear line).
   • Why it matters (explain the principle in 1–2 sentences).
   • Real consequences if ignored (honest downside for the team or project).
   • Smallest high-leverage fix (exact next step, effort level, and which files to touch).
   • At the end, add a lettered sub-list of concrete actions if useful (e.g. 2a, 2b) so the user can reply with "2b" or "1 and 3" to trigger implementation.
3. Bottom line: Full detailed audit report (layered findings, snapshot, metadata, implementation plan) → .architect-audits/documentation-audit/findings.md

When --learn or --teach is set, expand each recommendation into mid-level engineer teaching mode:

• For every item, explain as if teaching a mid-level engineer, pointing to specific files and line numbers from the current codebase.
• Use educational language: "Here's why this pattern bites teams in the long run…", "This is the exact mistake I see in most codebases at your stage…", "The fix is small but pays off huge because…".
• Include a short "What you'll learn from fixing this" section for each recommendation.
• Keep the numbered/lettered structure so the user can still reply with "2b" or "1 and 3".
• End with the same bottom-line link to the full report.

After printing, ask the single yes-or-no question: "Generate an implementation plan for the gaps identified above? (yes/no)" Do not proceed to phase 2 without an explicit affirmative.

Step 11 — Phase 2: generate the implementation plan

When the user agrees, build implementation-plan.md, ordered by audience:

1. Header — repository name, baseline version, project shape, observability stack, feature-flag stack, timestamp, total counts per layer.
2. Onboarding fixes (highest priority) — README expansion or rewrite, missing essentials, LICENSE addition, .env.example scaffolding, tool-version pin file. Per finding, the file to add or section to write, with a starter template.
3. Architectural and decision-documentation fixes — architecture overview scaffold, ADR directory and starter template, per-feature READMEs prioritised by Graphify centrality when present.
4. Operational and drift fixes — deployment runbook scaffold, rollback procedure scaffold, monitoring documentation scaffold, feature-flag documentation index. Drift remediation: README script renames, broken-link fixes (internal first, external when --with-link-check provided data).
5. Code-level fixes — TSDoc blocks for the highest-impact public exports (graph-prioritised), TODO closeouts (oldest first), empty-doc-block removal, commented-out-code deletion.
6. Closing checklist — flat checkbox list mirroring the gaps, suitable for pasting into a pull-request description.

The plan is descriptive, not executable. It does not write documentation, edit source, or fix links.

Findings file shape

findings.json:

{
  "skillVersion": "1.0.0",
  "runStartedAt": "2026-04-26T13:47:00Z",
  "runFinishedAt": "2026-04-26T13:47:21Z",
  "projectShape": "deployable-application",
  "observabilityStack": ["sentry"],
  "featureFlagStack": ["growthbook"],
  "withLinkCheck": true,
  "thresholds": {
    "readmeMinLines": 30,
    "adrStalenessDays": 180,
    "featureDocCoverage": 50,
    "publicApiDocCoverage": 60,
    "todoStalenessDays": 90,
    "docStalenessDays": 180
  },
  "snapshot": {
    "documentationFileCount": 47,
    "docToCodeLineRatio": 0.18,
    "readme": { "present": true, "lines": 42 },
    "license": { "present": true, "spdx": "MIT" },
    "adrs": { "count": 9, "mostRecentDate": "2026-03-12" },
    "todos": { "total": 31, "lessThan30Days": 8, "thirtyToNinetyDays": 12, "moreThan90Days": 11 },
    "storybookDetected": true,
    "docSiteFramework": "nextra",
    "perFeatureReadmeCoverage": 0.62,
    "externalLinkCount": 84,
    "externalLinkUnreachableCount": 5
  },
  "summary": {
    "projectEntryAndOnboarding":           { "present": 5, "partial": 1, "missing": 1, "violation": 1 },
    "architecturalAndDecisionDocumentation":{ "present": 3, "partial": 2, "missing": 0, "violation": 1 },
    "codeLevelDocumentation":              { "present": 3, "partial": 3, "missing": 0, "violation": 1 },
    "operationalDocumentationAndDrift":    { "present": 4, "partial": 2, "missing": 1, "violation": 1 }
  },
  "checks": [
    {
      "layer": "operational-documentation-and-drift",
      "check": "readme-script-drift-detection",
      "status": "violation",
      "evidence": ["README.md"],
      "samples": [
        { "command": "pnpm dev:legacy", "expectedScript": "dev:legacy", "actualScripts": ["dev", "dev:standalone"] }
      ],
      "totalCount": 2,
      "expectation": "Commands shown in the README reference scripts that exist in package.json.",
      "gap": "2 README commands point at scripts that no longer exist (dev:legacy, test:integration:legacy).",
      "remediation": "Either restore the scripts in package.json (if they are still wanted) or update the README to reference the current script names. Prefer updating the README — script renames usually reflect intentional cleanup."
    }
  ]
}

findings.md mirrors the same content in human-readable form, with the diagnostic snapshot at the top and one section per check, grouped by layer. snapshot.md contains only the snapshot. metadata.json carries skill identity, timestamps, Graphify revision (when present), the project shape, the observability and feature-flag stacks, applied thresholds, applied filters, the withLinkCheck flag, and per-link reachability status when the flag is set.

Idempotency rules

• Re-running with no flags overwrites findings.md, findings.json, snapshot.md, and metadata.json in place.
• implementation-plan.md is preserved across runs unless the user agrees to regenerate it.
• Filter flags, threshold overrides, and the --with-link-check flag are recorded in metadata.json so a partial run can be reproduced.
• Link-reachability data is timestamp-tagged in metadata; staleness is the user's responsibility to manage by re-running.

Failure modes and remediation

What this skill explicitly does NOT do

• Modify any documentation, source, or configuration file.
• Generate documentation. The implementation plan describes what to write; it does not write it.
• Run any code, including npm run commands referenced in the README. Drift detection compares text against package.json; it does not execute.
• Authenticate to private documentation sites or wiki tools (Confluence, Notion, internal handbooks). The audit covers documentation that lives in the repository.
• Translate or audit translations of documentation.
• Audit the quality of writing beyond structural and drift checks. Whether a paragraph is clear is a human judgement call.
• Replace human review of operational documentation. The audit verifies presence; whether the runbook would actually work in a real incident is a tabletop-exercise concern.
• Audit @typescript-eslint documentation rules (require-jsdoc, valid-jsdoc). That is owned by /linting-audit.
• Audit non-Markdown documentation formats (ReStructuredText, AsciiDoc, plain text). Markdown and MDX only.

Source: CW-Codewalnut/ArchitectPlaybook — distributed by TomeVault.