Doc Consistency Checker

doc-consistency-checker

Purpose: Detect when high-level project documentation (CLAUDE.md, README.md, etc.) is stale based on recent code changes and generate update suggestions.

Architecture: Operation-specific skill (Layer 3) - analyzes git diff and compares against documentation content.

Created: 2025-12-02 - Addresses documentation maintenance gap in FABER workflows.

2. Change Detection Focus

   • ANALYZE git diff for meaningful changes
   • IDENTIFY documentation-relevant changes (APIs, features, config)
   • IGNORE trivial changes (formatting, comments, whitespace)
   • PRIORITIZE high-impact changes

3. Target Documents

   • DEFAULT targets: CLAUDE.md, README.md, docs/README.md, CONTRIBUTING.md
   • SUPPORT custom targets via configuration
   • ONLY check documents that exist
   • NEVER create new documents

4. Script Usage

   • USE check-consistency.sh for change analysis
   • USE generate-suggestions.sh for update proposals
   • USE apply-updates.sh only after user confirmation

Step 1: Output Start Message

🎯 STARTING: Documentation Consistency Check
Targets: {target_list}
Base: {base_ref} → Head: {head_ref}
───────────────────────────────────────

Step 2: Validate Input Parameters

Check required parameters:

• targets: Array of target document paths (optional, uses defaults)
• base_ref: Git reference for comparison base (optional, default: main)
• head_ref: Git reference for comparison head (optional, default: HEAD)
• mode: Operation mode (optional, default: confirm)

Default targets:

DEFAULT_TARGETS=("CLAUDE.md" "README.md" "docs/README.md" "CONTRIBUTING.md")

Step 3: Analyze Git Changes

Invoke check-consistency.sh:

./skills/doc-consistency-checker/scripts/check-consistency.sh \
  --base "$BASE_REF" \
  --head "$HEAD_REF" \
  --output-format json

The script analyzes the git diff and categorizes changes:

Change Categories

API Changes (High Priority):

• New endpoints or routes
• Modified request/response schemas
• Authentication changes
• Rate limiting changes

Feature Changes (High Priority):

• New commands or skills
• Modified functionality
• New configuration options
• UI/UX changes

Architecture Changes (High Priority):

• New components or modules
• Modified dependencies
• Database schema changes
• Integration changes

Configuration Changes (Medium Priority):

• Environment variables
• Config file formats
• Default values

Documentation Changes (Low Priority):

• Existing doc updates
• Comment changes
• README updates in subdirectories

Output Format

{
  "changes": {
    "api": [
      {"file": "src/routes/auth.ts", "type": "added", "summary": "New /auth/login endpoint"}
    ],
    "features": [
      {"file": "src/commands/new-cmd.md", "type": "added", "summary": "New slash command"}
    ],
    "architecture": [],
    "configuration": [
      {"file": ".env.example", "type": "modified", "summary": "New API_KEY variable"}
    ],
    "documentation": []
  },
  "files_changed": 15,
  "lines_added": 342,
  "lines_removed": 45
}

Step 4: Check Each Target Document

For each target document that exists:

for target in "${TARGETS[@]}"; do
  if [[ -f "$target" ]]; then
    echo "📄 Checking $target..."

    # Read current content
    content=$(cat "$target")

    # Identify sections that may need updates
    check_sections "$target" "$CHANGES_JSON"
  fi
done

Section Analysis

For CLAUDE.md, check:

• Repository Overview section
• Architecture section
• Directory Structure section
• Configuration Files section
• Common Development Tasks section
• Key Files to Reference section

For README.md, check:

• Features section
• Installation section
• Usage section
• API section
• Configuration section

For CONTRIBUTING.md, check:

• Development Setup section
• Coding Standards section
• Testing section

Step 5: Generate Consistency Report

{
  "status": "stale | current",
  "targets_checked": 4,
  "targets_stale": 2,
  "stale_documents": [
    {
      "path": "CLAUDE.md",
      "sections_affected": ["Architecture", "Directory Structure"],
      "changes_requiring_update": [
        {
          "change_type": "feature",
          "file": "plugins/docs/skills/doc-consistency-checker/",
          "impact": "New skill added - update Directory Structure section"
        }
      ],
      "priority": "high"
    }
  ],
  "current_documents": ["README.md", "CONTRIBUTING.md"]
}

Step 6: Generate Update Suggestions (if operation=suggest)

Invoke generate-suggestions.sh:

./skills/doc-consistency-checker/scripts/generate-suggestions.sh \
  --target "$TARGET_PATH" \
  --changes "$CHANGES_JSON" \
  --output-format diff

For each stale document, generate:

• Specific text additions
• Section updates
• New section suggestions (if needed)

Suggestion Format

--- CLAUDE.md
+++ CLAUDE.md
@@ -45,6 +45,8 @@ plugins/
 ├── docs/               # Documentation management
 │   ├── agents/         # docs-manager agent
 │   ├── skills/         # Document operations
+│   │   └── doc-consistency-checker/  # NEW: Documentation staleness detection
 │   └── commands/       # User commands

Step 7: Present for Confirmation (if mode=confirm)

📝 Documentation Update Suggestions
────────────────────────────────────

📄 CLAUDE.md (2 sections affected):

1. Directory Structure
   + Add doc-consistency-checker skill entry

2. Common Development Tasks
   + Add consistency checking workflow

Apply these updates? (y/n/edit)

Step 8: Apply Updates (if approved or mode=auto)

Invoke apply-updates.sh:

./skills/doc-consistency-checker/scripts/apply-updates.sh \
  --target "$TARGET_PATH" \
  --updates "$UPDATES_JSON" \
  --backup true

• Create backup before modification
• Apply changes section by section
• Validate result with doc-validator

Step 9: Output End Message

✅ COMPLETED: Documentation Consistency Check
Targets Checked: {count}
Status: {current/stale}
Updates Applied: {count}
───────────────────────────────────────
Next: Review applied changes

High Priority Changes (Always Flag)

1. New Files in Key Directories

   • plugins//skills//SKILL.md → Update Directory Structure
   • plugins//commands/.md → Update Available Commands
   • plugins//agents/.md → Update Agents section

2. Configuration Changes

   • *.config.json, *.toml → Update Configuration section
   • .env*, .env.example → Update Environment Variables
   • package.json (scripts) → Update Scripts section

3. API/Interface Changes

   • New exports → Update API section
   • Modified function signatures → Update Usage section
   • New CLI arguments → Update CLI Reference

Medium Priority Changes (Suggest Review)

1. Dependency Changes

   • package.json (dependencies) → Consider noting major updates
   • requirements.txt → Consider noting Python dependency updates

2. Test Coverage Changes

   • New test files → Consider updating Testing section
   • Coverage thresholds → Consider updating quality metrics

Low Priority Changes (Informational)

1. Documentation in Subdirectories

   • Already maintained by their own processes

2. Internal Refactoring

   • No external API changes

This skill uses 3 scripts in skills/doc-consistency-checker/scripts/:

check-consistency.sh:

• Analyzes git diff between base and head
• Categorizes changes by type (api, feature, config, etc.)
• Identifies documentation-relevant changes
• Returns structured JSON with change details

generate-suggestions.sh:

• Reads change analysis and target document
• Identifies sections needing updates
• Generates diff-format suggestions
• Prioritizes by impact level

apply-updates.sh:

• Creates backup of target document
• Applies approved updates
• Validates result
• Returns success/failure status

All scripts return structured JSON.

Check Response (Current):

{
  "success": true,
  "operation": "check",
  "status": "current",
  "targets_checked": 4,
  "targets_stale": 0,
  "message": "All target documents are up to date with recent changes"
}

Suggest Response:

{
  "success": true,
  "operation": "suggest",
  "suggestions": [
    {
      "target": "CLAUDE.md",
      "section": "Directory Structure",
      "action": "add",
      "content": "│   └── doc-consistency-checker/  # Documentation staleness detection",
      "context_line": "│   ├── skills/         # Document operations",
      "priority": "high"
    }
  ],
  "total_suggestions": 3
}

Apply Response:

{
  "success": true,
  "operation": "apply",
  "updates_applied": 3,
  "files_modified": ["CLAUDE.md"],
  "backups_created": ["CLAUDE.md.backup.2025-12-02"]
}

Error Response:

{
  "success": false,
  "operation": "check",
  "error": "No git repository found",
  "error_code": "NOT_GIT_REPO"
}

FABER Release Phase:

• Called before PR creation
• Ensures docs are updated before release

docs-manager Agent:

• Receives consistency check requests
• Routes apply operations through doc-writer

doc-validator Skill:

• Validates documents after updates are applied