# Doc Audit

> Expert in auditing Sphinx/RST documentation for Drift project. Use when validating code examples and removing subjective language before release. Use when this capability is needed.

- Skill: `tomevault-io/doc-audit` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add tomevault-io/doc-audit`
- Raw SKILL.md: https://api.skillmd.com/api/skills/tomevault-io/doc-audit/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Security
- Author: tomevault-io (https://skillmd.com/u/tomevault-io)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/tomevault-io/doc-audit

---


# Documentation Audit Skill

Learn how to audit documentation for Drift project - validate code examples and ensure objective language.

## When to Use This Skill

- Reviewing documentation before release
- Validating code examples against implementation
- Ensuring objective, factual documentation language
- Checking for outdated or incorrect examples
- Maintaining documentation quality standards

## How to Audit Documentation

### Overview of Audit Process

A complete documentation audit includes three phases:
1. **Code Example Validation** - Verify all code examples match actual implementation
2. **Subjective Language Detection** - Flag subjective or marketing language
3. **Cross-Reference Implementation** - Check against actual code using serena MCP

Work through each phase systematically for each documentation file.

## How to Validate Code Examples

### Step 1: Extract Code Examples

Look through documentation files for code blocks:

**RST files:**
```rst
.. code-block:: python

   from drift.cli import cli
   drift.analyze("log.json")
```

**Markdown files:**
```markdown
```python
from drift.cli import cli
drift.analyze("log.json")
```
```

Categorize as:
- Complete examples (should run as-is)
- Snippets (partial code for illustration)

### Step 2: Validate Imports with Serena MCP

Use `mcp__serena__find_symbol` to verify imports:

**Example validation:**
```python
# Documentation says:
from drift.cli import cli

# Verify with serena:
mcp__serena__find_symbol(
    name_path_pattern="cli",
    relative_path="src/drift/cli.py"
)
```

**Common import patterns to check:**
- `from drift.cli import ...`
- `from drift.core.analyzer import ...`
- `from drift.config.loader import ...`
- `from drift.providers import ...`
- `from drift.validation import ...`

If symbol not found, import path is incorrect.

### Step 3: Verify CLI Commands

Check all CLI examples in documentation:

**Commands to verify:**
```bash
drift                    # Basic usage
drift --no-llm           # Programmatic validation only
drift --days 7           # Analyze last 7 days
drift --format json      # JSON output
drift --cwd PATH         # Working directory
drift --rules NAMES      # Specific rules
```

**How to verify:**
```bash
# Run command to check it exists and works
drift --help

# Check for specific flags
drift --help | grep -- --no-llm
drift --help | grep -- --format
```

### Step 4: Validate Configuration Examples

Check YAML configuration examples:

**Example documentation config:**
```yaml
# From docs/configuration.rst
drift_rules:
  provider: anthropic
  model: claude-3-sonnet-20240229
```

**Validate with serena:**
```python
# Check config structure
mcp__serena__find_symbol(
    name_path_pattern="Config",
    relative_path="src/drift/config"
)

# Read actual config loading code
mcp__serena__find_symbol(
    name_path_pattern="load_config",
    relative_path="src/drift/config",
    include_body=True
)
```

Look for:
- Correct parameter names
- Valid provider values (anthropic, bedrock, claude-code)
- Correct structure and nesting

### Step 5: Verify API Methods

Use serena to check that documented APIs exist:

**Documentation example:**
```python
analyzer = Analyzer(config_path=".drift.yaml")
results = analyzer.analyze()
```

**Verify with serena:**
```python
# Find Analyzer class
mcp__serena__find_symbol(
    name_path_pattern="Analyzer",
    relative_path="src/drift",
    depth=1  # Include methods
)

# Check analyze method specifically
mcp__serena__find_symbol(
    name_path_pattern="Analyzer/analyze",
    relative_path="src/drift",
    include_body=True
)
```

Compare:
- Method exists
- Parameters match
- Return type matches
- No deprecated methods

### Step 6: Check Example Completeness

For each code example, verify:

**Complete example checklist:**
- [ ] All imports included
- [ ] No undefined variables
- [ ] No missing context
- [ ] Could theoretically run (or marked as snippet)
- [ ] Error handling shown (where relevant)

**Example of incomplete documentation:**
```python
# BAD - missing imports and context
result = analyzer.analyze()
print(result)
```

**Should be:**
```python
# GOOD - complete example
from drift.core.analyzer import Analyzer

analyzer = Analyzer(config_path=".drift.yaml")
result = analyzer.analyze()
print(result)
```

## How to Detect Subjective Language

### Step 1: Scan for Subjective Adjectives/Adverbs

Search documentation for these patterns:

**Common subjective words:**
- "easily", "simply", "just"
- "obviously", "clearly"
- "powerful", "elegant", "beautiful"
- "amazing", "best", "better"
- "great", "excellent"
- "effortlessly", "seamlessly"
- "straightforward"

**How to search:**
```bash
grep -n "easily\|simply\|just\|powerful" docs/*.rst
```

Or use serena:
```python
mcp__serena__search_for_pattern(
    substring_pattern="easily|simply|just|powerful",
    relative_path="docs/"
)
```

### Step 2: Flag Marketing Language

**Bad - marketing language:**
```
Drift provides a powerful validation system that makes
it easy to ensure code quality.
```

**Good - objective technical description:**
```
Drift validates project structure using rule-based checks.
Validation runs programmatically without LLM calls.
```

**What to look for:**
- Superlatives without data ("best", "most powerful")
- Emotional appeals
- Vague benefit claims
- Comparisons without specifics

### Step 3: Identify False User Behavior Claims

**CRITICAL - never claim user behavior without data:**

**Bad examples:**
```
Most users run drift --no-llm in CI/CD
Many teams use this for code review
Typically developers check this daily
Users commonly integrate with GitHub Actions
```

**Good alternatives:**
```
drift --no-llm runs without API calls, suitable for CI/CD
Integrates with GitHub Actions via workflow files
Can be scheduled to run periodically
```

**Prohibited phrases:**
- "most users", "most teams", "most workflows"
- "many users", "many developers"
- "typically", "commonly", "usually"
- Any claim about what users do without hard data

### Step 4: Suggest Replacements

**Pattern:** Subjective claim → Objective fact

**Example 1:**
```
Bad:  "You can easily analyze AI conversations..."
Good: "You can analyze AI conversations..."
```

**Example 2:**
```
Bad:  "Drift's powerful validation system..."
Good: "Drift's validation system..."
```

**Example 3:**
```
Bad:  "Most users run validation before commits"
Good: "Validation can run before commits via pre-commit hooks"
```

## How to Use Serena MCP for Validation

### Find Symbols

```python
# Find a class
mcp__serena__find_symbol(
    name_path_pattern="Analyzer",
    relative_path="src/drift",
    depth=1  # Include methods
)

# Find specific method
mcp__serena__find_symbol(
    name_path_pattern="Analyzer/analyze",
    relative_path="src/drift",
    include_body=True  # See implementation
)
```

### Check Module Structure

```python
# Get overview of a module
mcp__serena__get_symbols_overview(
    relative_path="src/drift/cli.py",
    depth=1
)
```

Shows all functions, classes, methods in the file.

### Search for Usage Patterns

```python
# Find how something is used
mcp__serena__search_for_pattern(
    substring_pattern="Analyzer\\(",
    relative_path="tests/",
    restrict_search_to_code_files=True
)
```

Look at test files to see correct usage patterns.

### Verify Import Paths

```python
# Check if import exists
mcp__serena__find_symbol(
    name_path_pattern="cli",
    relative_path="src/drift/cli.py"
)

# If not found, import path in docs is wrong
```

## How to Generate Audit Reports

See [Report Structure](resources/report-structure.md) for:
- Complete report template
- Issue severity guidelines
- Example issue documentation

## Drift-Specific Validation Areas

### Validation Rules Documentation

Check that documented rules exist:

```python
# Find validation rules
mcp__serena__list_dir(
    relative_path="src/drift/validators",
    recursive=True
)

# Verify rule names match documentation
mcp__serena__search_for_pattern(
    substring_pattern="class.*Validator",
    relative_path="src/drift/validators"
)
```

### Provider Configuration Examples

Validate provider examples in docs:

**Anthropic configuration:**
```yaml
provider: anthropic
model: claude-3-sonnet-20240229
```

**Bedrock configuration:**
```yaml
provider: bedrock
model: anthropic.claude-v2
region: us-east-1
```

**Verify:**
```python
# Check provider loading code
mcp__serena__find_symbol(
    name_path_pattern="load_provider",
    relative_path="src/drift"
)
```

### CLI Documentation

Validate all CLI examples:

```bash
# Documented commands to check:
drift                    # Basic usage
drift --no-llm           # No LLM mode
drift --days 7           # Time range
drift --format json      # Output format
drift --rules NAMES      # Specific rules
drift --cwd PATH         # Working directory
```

**Verify each flag exists:**
```bash
drift --help | grep "flag-name"
```

## Example Workflows

See [Workflows](resources/workflows.md) for detailed audit workflows:
- Auditing a single documentation file
- Validating all CLI examples
- Complete audit workflow (3 phases)

## Resources

### 📖 [Code Validation](resources/code-validation.md)
Detailed guide for validating code examples in documentation.

**Use when:** Checking code examples, imports, and API usage in docs.

### 📖 [Workflows](resources/workflows.md)
Step-by-step workflows for different types of documentation audits.

**Use when:** Performing a documentation audit before release.

### 📖 [Report Structure](resources/report-structure.md)
Documentation audit report template with issue severity guidelines.

**Use when:** Generating an audit report with findings and recommendations.

---
> Converted and distributed by [TomeVault](https://tomevault.io/claim/jarosser06) — claim your Tome and manage your conversions.
<!-- tomevault:4.0:skill_md:2026-04-13 -->