Documentation Consistency Skill

What I do

I detect and repair documentation drift across the repository's markdown files:

1. Cross-File Count Sync — Verify skill/agent counts match reality across setup.sh, setup.ps1, README.md, and AGENTS.md
2. PLAN vs Reality Drift — Detect stale checkboxes, mismatched acceptance criteria, and outdated scope sections in PLAN*.md files
3. Structural Integrity — Validate required sections exist in PLAN files, README tables, and AGENTS.md routing tables
4. Orphan/Stale References — Find skill/agent names in documentation that don't exist on disk, and vice versa

When to use me

Use this skill when:

• After adding, removing, or renaming skills or subagents
• Before creating a PR to verify documentation is consistent
• After a long work session with many edits to documentation files
• When PLAN checkboxes may not reflect actual implementation state
• Periodically as a health check against documentation rot

Trigger phrases:

• "check documentation consistency"
• "audit docs"
• "verify doc consistency"
• "check for documentation drift"
• "consistency check"
• "validate documentation"

Related Skills

Validation Levels

Audit Category 1: Cross-File Count Sync

What It Checks

Skill and agent counts must be identical across all documentation files.

Source of truth: Actual files on disk.

ACTUAL_SKILLS=$(ls -d opencode_app/.opencode/skills/*/ | grep -v _archived | grep -v scripts | wc -l)
ACTUAL_AGENTS=$(ls opencode_app/.opencode/agents/*.md 2>/dev/null | wc -l)

Files Checked

Validation Commands

ACTUAL=$(ls -d opencode_app/.opencode/skills/*/ | grep -v _archived | grep -v scripts | wc -l)
echo "Actual skills on disk: $ACTUAL"
echo "---"
echo "setup.sh:    $(grep -oP 'SKILLS \(\K[0-9]+' deploy/setup.sh | head -1)"
echo "setup.ps1:   $(grep -oP 'SKILLS \(\K[0-9]+' deploy/setup.ps1 | head -1)"
echo "README.md:   $(grep -oP '\d+ skills' README.md | head -1)"
echo "AGENTS.md:   $(grep -oP '\d+ skill' AGENTS.md | head -1)"

Auto-Fix Strategy

1. Count actual skills on disk (excluding _archived/ and scripts/)
2. Find all count references in each file using the search patterns above
3. For each mismatch: use edit tool to replace the old count with actual count
4. Verify per-category counts sum to total

Audit Category 2: PLAN vs Reality Drift

What It Checks

PLAN*.md files must accurately reflect the current implementation state.

Checks Performed

Validation Commands

PLAN_FILE="PLANS/PLAN-GIT-204.md"

echo "=== Unchecked items ==="
grep -n "\- \[ \]" "$PLAN_FILE"

echo "=== Checked items ==="
grep -n "\- \[x\]" "$PLAN_FILE"

echo "=== Scope section files ==="
grep -A 20 "## Scope" "$PLAN_FILE"

echo "=== Files changed on branch ==="
git diff --name-only main...HEAD

echo "=== Branch reference ==="
grep "Branch" "$PLAN_FILE"
echo "Actual: $(git branch --show-current)"

Auto-Fix Strategy

1. Compare git diff --name-only main...HEAD with Scope section
2. For each checkbox: check if referenced file exists and was modified on branch
3. If file exists and was modified → mark [x]
4. If file listed in Scope but not touched → flag as potentially stale
5. Update Scope section if new files were touched that aren't listed
6. Always preserve content — only update checkbox states and Scope entries

PLAN File Required Sections

Every PLAN*.md should contain these sections:

# PLAN-<ID>: <Title>

**Issue**: <link>
**Branch**: `<branch-name>`
**Status**: <status>

---

## Problem
<description>

## Solution
<description>

---

## Phase 1: <Name>
- [ ] <task>
- [ ] <task>

## Phase N: <Name>
- [ ] <task>

---

## Acceptance Criteria
- [ ] <criterion>

## Scope
| File | Action |
|------|--------|
| <path> | Create/Update |

## Risks & Mitigation
| Risk | Severity | Mitigation |
|------|----------|-----------|

Audit Category 3: Structural Integrity

What It Checks

Documentation files must have complete, well-formed sections and tables.

Checks Per File

README.md:

• ## Skill Modularization section exists
• Skill Categories table has correct header row: | Category | Skills | Purpose |
• All expected category rows present
• Subagents table exists (if applicable)
• Repository structure tree is present

AGENTS.md:

• ## Repository Overview section exists
• ## Shared Config Strategy section exists
• ## Subagent Locations table exists
• ## CodeGraph section exists (if configured)
• ## Adding New Subagents or Skills section exists
• Skill count references are accurate

deploy/setup.sh and deploy/setup.ps1:

• SKILLS ( listing section exists
• All category lines have format: Category (N): skill1, skill2, ...
• Category counts sum to total skill count
• Both files have identical category listings and counts

Validation Commands

echo "=== README.md sections ==="
grep "^## " README.md

echo "=== README.md Skill Categories table ==="
grep -A 20 "Skill Categories" README.md | head -20

echo "=== AGENTS.md sections ==="
grep "^## " AGENTS.md

echo "=== setup.sh category count sum ==="
grep -oP '\(\K[0-9]+' deploy/setup.sh | head -15

Auto-Fix Strategy

1. For missing sections: report with template content to insert
2. For malformed tables: suggest corrected rows
3. For count mismatches in categories: flag the specific category with expected vs actual
4. Do NOT auto-fix structural issues without confirmation — these require human judgment

Audit Category 4: Orphan/Stale References

What It Checks

Every skill, agent, and MCP reference in documentation must point to something that exists on disk.

Checks Performed

Validation Commands

echo "=== Agents on disk but NOT in docs ==="
for f in opencode_app/.opencode/agents/*.md; do
  name=$(basename "$f" .md)
  if ! grep -rq "$name" AGENTS.md README.md; then
    echo "  MISSING FROM DOCS: $name"
  fi
done

echo "=== Skills on disk but NOT in setup.sh ==="
for d in opencode_app/.opencode/skills/*/; do
  name=$(basename "$d")
  [[ "$name" == "_archived" || "$name" == "scripts" ]] && continue
  if ! grep -q "$name" deploy/setup.sh; then
    echo "  MISSING FROM SETUP.SH: $name"
  fi
done

echo "=== Skills in setup.sh but NOT on disk ==="
grep -oP '[a-z][a-z0-9-]+' deploy/setup.sh | sort -u | while read name; do
  if [[ "$name" == *"-skill" || "$name" == *"-workflow" || "$name" == *"-creator" ]]; then
    if [ ! -d "opencode_app/.opencode/skills/$name" ]; then
      echo "  ORPHAN IN SETUP.SH: $name"
    fi
  fi
done

echo "=== Cross-skill references check ==="
for d in opencode_app/.opencode/skills/*/SKILL.md; do
  name=$(basename "$(dirname "$d")")
  refs=$(grep -oP '[a-z][a-z0-9-]+-skill' "$d" 2>/dev/null | sort -u)
  for ref in $refs; do
    [[ "$ref" == "$name" ]] && continue
    if [ ! -d "opencode_app/.opencode/skills/$ref" ]; then
      echo "  BROKEN REF in $name: $ref"
    fi
  done
done

Auto-Fix Strategy

1. For orphans in deploy scripts: remove the stale entry from the listing
2. For missing entries in deploy scripts: add to appropriate category
3. For broken cross-skill references: report for manual review (wrong to auto-delete)
4. For undocumented skills/agents: add to appropriate documentation section
5. Always update counts after adding/removing entries

Validation Report Template

When running a consistency check, generate a report in this format:

## Documentation Consistency Report

**Level**: <quick|standard|thorough|targeted>
**Date**: <ISO 8601>
**Branch**: <branch-name>

### Summary

| Category | Status | Issues Found | Fixed |
|----------|--------|-------------|-------|
| Cross-File Counts | PASS/FAIL | N | N |
| PLAN vs Reality | PASS/FAIL/SKIP | N | N |
| Structural Integrity | PASS/FAIL/SKIP | N | N |
| Orphan References | PASS/FAIL/SKIP | N | N |

### Issues

#### Cross-File Count Sync
- [ISSUE] setup.sh: expected 74 skills, found 73
  - Fix: edit deploy/setup.sh, change `SKILLS (73)` to `SKILLS (74)`
- [PASS] setup.ps1: count matches (74)
- ...

#### PLAN vs Reality Drift
- [ISSUE] PLAN-GIT-204.md: checkbox line 28 still unchecked but file exists
  - Fix: edit PLANS/PLAN-GIT-204.md, change `- [ ]` to `- [x]` on line 28
- ...

#### Structural Integrity
- [PASS] README.md: all required sections present
- ...

#### Orphan References
- [ISSUE] Agent `old-subagent` listed in AGENTS.md but no .md file on disk
  - Fix: remove from AGENTS.md routing table
- ...

Steps

Step 1: Determine Validation Level

Based on context or explicit request, choose a validation level:

• quick: Run only Audit Category 1 (counts)
• standard: Run Categories 1, 2, 4
• thorough: Run all 4 categories
• targeted: Run all categories for a single PLAN file

Step 2: Run Checks

For each applicable category, execute the validation commands and collect results.

Step 3: Generate Report

Output the Validation Report using the template above.

Step 4: Apply Fixes (if requested)

For each issue found:

1. Present the fix to the user
2. If user confirms (or auto-fix is enabled), apply via edit tool
3. Re-run the specific check to verify the fix

Step 5: Final Verification

Run the full set of validation commands one more time to confirm all issues are resolved.

Common Issues

Count Mismatch Between Files

Issue: SKILLS (73) in setup.sh but 74 skills in README.md

Solution: Determine actual count from disk, then update all files to match. Always use the disk count as source of truth.

PLAN Checkboxes Out of Sync

Issue: Work completed but checkboxes still [ ], or checkboxes marked [x] for unimplemented work.

Solution: Cross-reference with git diff --name-only. If a file was modified, its corresponding checkbox should be [x]. If no file matches a [x] checkbox, flag for review.

Stale Category Counts

Issue: Category says (3) but has 4 skills listed.

Solution: Count the actual skills in the category listing and update the parenthetical number.

Orphan Agent Reference

Issue: Agent name in AGENTS.md but no .md file exists.

Solution: Check if it was renamed or removed. If removed, delete the reference. If renamed, update the name. If it should exist, the agent file needs to be created.

Missing PLAN Required Section

Issue: PLAN file missing ## Risks & Mitigation or ## Scope.

Solution: Add the missing section with appropriate template content. Do not remove existing sections.

Best Practices

1. Always run thorough check before PRs — catches all categories of drift
2. Run quick check after any skill/agent change — catches count mismatches early
3. Use targeted level during plan execution — focuses on the active PLAN file
4. Disk is truth — when in doubt, trust what's actually on the filesystem
5. Don't auto-fix structural issues — missing sections require human judgment
6. Fix all files in one pass — update setup.sh, setup.ps1, README.md, and AGENTS.md together
7. Re-validate after fixes — always run checks again to confirm resolution
8. Keep the report — store validation reports in LEARNINGS/solutions/ for future reference

Source: darellchua2/opencode-config-template — distributed by TomeVault.