Documentation Quality Assurance

Documentation Quality Assurance

Prevent documentation rot through systematic audits, automated validation, and pipeline-enforced quality gates.

Scope: Inheritable skill. Covers drift detection, preflight validation, semantic accuracy, link integrity, 5-pass quality pipeline, staleness detection, and large-project organization.

Complements: The Documentarian agent uses this skill as its knowledge foundation. This skill is the "what" — the agent is the "who".

Audit Priority: Semantics Over Syntax

The most damaging documentation errors are semantic, not syntactic. A wrong count is annoying. A false claim about functionality is dangerous.

Audit Priority Hierarchy

Semantic Audit Questions

Before any documentation audit, ask these questions in order:

1. Does this feature actually exist? — Check if documented functionality has corresponding implementation
2. Is this claim still true? — Validate assertions against current codebase state
3. Are there contradictions? — Cross-check related documents for conflicting information
4. Is the version current? — Compare documented versions against package.json/CHANGELOG
5. Do examples work? — Test code snippets against actual API/CLI behavior
6. Do links resolve? — Verify internal and external references (automate this)
7. Are counts accurate? — Check hardcoded numbers against canonical sources (automate this)

Rule: Questions 1-5 require human judgment. Questions 6-7 can be automated. Never spend human attention on automatable checks at the expense of semantic review.

Common Semantic Bugs

The Count Problem (P3)

Hardcoded counts in documentation drift silently:

Count Elimination Rules

Canonical Count Sources

When a count is needed, always derive it from the file system:

Docs-as-Architecture

Documentation is a load-bearing part of the system, not a separate artifact:

Document Header Pattern

Comprehensive Metadata Headers

Operational documentation (regression checklists, deployment guides, QA procedures, release workflows) should include comprehensive headers that provide complete context at-a-glance.

Minimal Header (4 lines) — Insufficient:

**Date**: 2026-02-14
**Status**: In Progress
**Purpose**: Verify v5.7.1 UI features
**Method**: Install and test

Enhanced Header (10+ lines) — Comprehensive:

**Version**: 5.7.1
**Date**: 2026-02-14
**Status**: ⚠️ PENDING UI VERIFICATION — WebP avatars regenerated, awaiting restart + testing
**Testing Mode**: CP2 Contingency (Local Install)
**VSIX Size**: 9.44 MB (426 files)
**Key Changes**: Enterprise auth removed, WebP avatars optimized (144×144, 92% reduction)

**Purpose**: Local install verification of all v5.7.1 visual identity + UI features
**Method**: Install VSIX locally, restart VS Code, test in current workspace (CP2 contingency)
**Expected Outcome**: All 9 test sections pass → DoD criterion #4 complete → Ready to publish

Header Field Guidelines

Rule: Include enough metadata that anyone can understand the document's context without reading the body. Operational docs reviewed during incidents need at-a-glance clarity.

6-Pass Quality Pipeline

Run these passes in sequence on any document suite:

Rule 1: Don't merge passes — each pass has a single focus. Rule 2: Complete all semantic passes (1-3) before mechanical passes (4-5). A perfectly formatted lie is still a lie. Rule 3: Never allow automated tooling to "pass" a doc suite until human semantic review is complete.

Preflight Validation

Automated Checks

Run before every release or documentation change:

# Example preflight validation script
function Test-DocQuality {
    $errors = @()
    
    # Check 1: All markdown links resolve
    Get-ChildItem -Recurse -Filter "*.md" | ForEach-Object {
        $content = Get-Content $_.FullName -Raw
        $links = [regex]::Matches($content, '\[([^\]]+)\]\(([^)]+)\)')
        foreach ($link in $links) {
            $target = $link.Groups[2].Value
            if ($target -notmatch '^https?://' -and $target -notmatch '^#') {
                $resolved = Join-Path (Split-Path $_.FullName) $target
                if (-not (Test-Path $resolved)) {
                    $errors += "Broken link in $($_.Name): $target"
                }
            }
        }
    }
    
    # Check 2: No orphan files in docs folder
    # Check 3: Required sections present in each doc type
    # Check 4: Version strings match package.json
    
    return $errors
}

Pre-Implementation Cross-Reference Sweep

Before adding any new file, check what already references the concept:

1. Grep for the concept name across all docs
2. Identify which files will need updating
3. Create the new file AND update all references in a single commit

Anti-pattern: Creating a new skill/agent and updating only one reference document. Every catalog, index, and count needs updating simultaneously.

Link Integrity

Link Audit Protocol

Orphan Detection

# Find markdown files not referenced by any other markdown file
$allMd = Get-ChildItem -Recurse -Filter "*.md" | Select-Object -ExpandProperty Name
$referenced = @()
Get-ChildItem -Recurse -Filter "*.md" | ForEach-Object {
    $content = Get-Content $_.FullName -Raw
    $links = [regex]::Matches($content, '\]\(([^)]+\.md)')
    foreach ($link in $links) {
        $referenced += Split-Path $link.Groups[1].Value -Leaf
    }
}
$orphans = $allMd | Where-Object { $_ -notin $referenced }

Staleness Detection

Last Validated Dates

Add validation dates to critical documents:

<!-- Last Validated: 2026-02-12 by Documentarian audit -->

Staleness Tiers

Drift Detection During Maintenance

During regular maintenance cycles (meditation, dream, release prep):

1. Compare document claims against current reality
2. Flag any numeric drift (counts, versions, dates)
3. Check if referenced files still exist
4. Verify code examples still compile/execute
5. Update Last Validated timestamp after review

Large Project Organization

15+ File Threshold

When a documentation folder exceeds 15 files, introduce numbered chapter folders:

docs/
├── 01-getting-started/
│   ├── installation.md
│   ├── quick-start.md
│   └── configuration.md
├── 02-architecture/
│   ├── overview.md
│   ├── components.md
│   └── data-flow.md
├── 03-operations/
│   ├── deployment.md
│   ├── monitoring.md
│   └── troubleshooting.md
└── README.md              # Index/TOC

Rules:

• Numbered prefixes ensure consistent ordering across all tools
• Each chapter folder has 3-7 files (not 1, not 20)
• Root README.md serves as table of contents with links to all sections
• Flat structure is fine for < 15 files

Multi-Audience Documentation

Audience Matrix

Every doc suite serves multiple readers:

Rule: Each document should declare its audience. A document trying to serve all audiences well serves none of them well.

Ship First, Document After (Threshold)

Anti-pattern: Blocking a release to write perfect docs. Ship with minimal docs (README + quick start), then iterate.

Doc Audit Checklist

Run this 10-item checklist for any documentation review. Semantic checks first.

Phase 1: Semantic Accuracy (🧠 Human Required)

Phase 2: Mechanical Accuracy (🤖 Automatable)

Rule: Never mark a doc suite "clean" based only on Phase 2 passing. Phase 1 semantic checks are non-negotiable.

TODO Files as Self-Models

A TODO list that contains completed work is worse than no TODO list. TODO.md is a self-model — when read at session start, it forms a mental picture of what exists and what doesn't. Completed tasks masquerading as pending create:

1. Rediscovery tax — work already done gets re-investigated
2. False urgency — energy directed at "building" something already built

The Fix: Done Section First

## Done — Audited [date]
- [x] secretScanner.ts ported to shared/utils/
- [x] All 15 extension.ts files implemented

## Next
- [ ] npm run compile — verify TypeScript
- [ ] F5 smoke test in Extension Development Host

Maintenance Rule: During every meditation or sprint transition, audit TODO.md first. Move completed items to Done. A stale self-model wastes more time than the audit costs.

CHANGELOG Best Practices