Quality Scan

quality-scan

Scan Types Available:

1. critical - Crashes, security vulnerabilities, resource leaks, data corruption
2. logic - Algorithm errors, edge cases, type guards, off-by-one errors
3. cache - Cache staleness, race conditions, invalidation bugs
4. workflow - Build scripts, CI issues, cross-platform compatibility
5. workflow-optimization - CI optimization checks (build-required conditions on cached builds)
6. security - GitHub Actions workflow security (zizmor scanner)
7. documentation - README accuracy, outdated docs, missing documentation, junior developer friendliness

Why Quality Scanning Matters:

• Catches bugs before they reach production
• Identifies security vulnerabilities early
• Improves code quality systematically
• Provides actionable fixes with file:line references
• Prioritizes issues by severity for efficient remediation
• Cleans up junk files for a well-organized repository

Agent Prompts: All agent prompts are embedded in reference.md with structured , , , and tags following Claude best practices.

Do NOT:

• Fix issues during scan (analysis only - report findings)
• Skip critical scan types without user permission
• Report findings without file/line references
• Proceed if codebase has uncommitted changes (warn but continue)

Do ONLY:

• Run enabled scan types in priority order (critical → logic → cache → workflow)
• Generate structured findings with severity levels
• Provide actionable improvement tasks with specific code changes
• Report statistics and coverage metrics
• Deduplicate findings across scans

Process

Execute the following phases sequentially to perform comprehensive quality analysis.

Phase 1: Validate Environment

Verify the environment before starting scans:

git status

If working directory dirty:

• Warn user: "Working directory has uncommitted changes - continuing with scan"
• Continue with scans (quality scanning is read-only)

Phase 2: Update Dependencies

Update Process:

pnpm run update

Track for reporting:

• Packages updated: N
• Update status: Success/Failed (with warning)

Important: Only update dependencies in the current repository. Do NOT attempt to update sibling repositories as this is out of scope and could have unintended side effects.

Phase 2b: Install External Tools (zizmor)

1. Read minimumReleaseAge from .pnpmrc:

   grep 'minimumReleaseAge' .pnpmrc | cut -d'=' -f2
   

   This returns minutes (e.g., 10080 = 7 days). Default to 10080 if not found.

2. Query zizmor releases (using curl or gh):

   # Option A: curl (universally available)
   curl -s "https://api.github.com/repos/zizmorcore/zizmor/releases" | \
     jq '[.[] | select(.prerelease == false) | {tag: .tag_name, date: .published_at}] | .[0:10]'
   
   # Option B: gh (if available)
   gh api repos/zizmorcore/zizmor/releases --jq \
     '[.[] | select(.prerelease == false) | {tag: .tag_name, date: .published_at}] | .[0:10]'
   

3. Calculate age and select version:

   • Convert minimumReleaseAge from minutes to days: minutes / 1440
   • Find latest stable release older than that threshold
   • Example: If minimumReleaseAge=10080 (7 days) and today is March 24, select releases from March 17 or earlier

4. Install selected version (choose based on available tools):

   # macOS with Homebrew (latest only, version pinning limited)
   brew install zizmor
   
   # Python environments (version pinning supported)
   pipx install zizmor==VERSION
   uv tool install zizmor==VERSION
   uvx zizmor@VERSION --help
   

   Recommended priority: pipx/uvx > brew

Phase 3: Repository Cleanup

Cleanup Tasks:

1. Remove SCREAMING_TEXT.md files (all-caps .md files) that are NOT:

   • Inside .claude/ directory
   • Inside docs/ directory
   • Named README.md, LICENSE, or SECURITY.md

2. Remove temporary test files in wrong locations:

   • .test.mjs or .test.mts files outside test/ or __tests__/ directories
   • Temp files: *.tmp, *.temp, .DS_Store, Thumbs.db
   • Editor backups: *~, *.swp, *.swo, *.bak
   • Test artifacts: *.log files in root or package directories (not logs/)

# Find SCREAMING_TEXT.md files (all caps with .md extension)
find . -type f -name '*.md' \
  ! -path './.claude/*' \
  ! -path './docs/*' \
  ! -name 'README.md' \
  ! -name 'LICENSE' \
  ! -name 'SECURITY.md' \
  | grep -E '/[A-Z_]+\.md$'

# Find test files in wrong locations
find . -type f \( -name '*.test.mjs' -o -name '*.test.mts' \) \
  ! -path '*/test/*' \
  ! -path '*/__tests__/*' \
  ! -path '*/node_modules/*'

# Find temp files
find . -type f \( \
  -name '*.tmp' -o \
  -name '*.temp' -o \
  -name '.DS_Store' -o \
  -name 'Thumbs.db' -o \
  -name '*~' -o \
  -name '*.swp' -o \
  -name '*.swo' -o \
  -name '*.bak' \
\) ! -path '*/node_modules/*'

# Find log files in wrong places (not in logs/ or build/ directories)
find . -type f -name '*.log' \
  ! -path '*/logs/*' \
  ! -path '*/build/*' \
  ! -path '*/node_modules/*' \
  ! -path '*/.git/*'

If no junk files found:

• Report: "✓ Repository is clean - no junk files found"

Important:

• Always get user confirmation before deleting
• Show file contents if user is unsure
• Track deleted files for reporting

Phase 4: Structural Validation

Validation Tasks:

Run the consistency checker to validate monorepo structure:

node scripts/check-consistency.mjs

The consistency checker validates:

1. Required files - README.md, package.json existence
2. Vitest configurations - Proper mergeConfig usage
3. Test scripts - Correct test patterns per package type
4. Coverage scripts - Coverage setup where appropriate
5. External tools - external-tools.json format validation
6. Build output structure - Standard build/{mode}/out/Final/ layout
7. Package.json structure - Standard fields and structure
8. Workspace dependencies - Proper workspace:* and catalog: usage

If errors found:

1. Report as Critical findings in the final report
2. Include file:line references from checker output
3. Suggest fixes based on checker recommendations
4. Continue with remaining scans

If warnings found:

• Report as Low findings (these are expected deviations)
• Document in final report under "Structural Validation"

Track for reporting:

• Number of packages validated
• Number of errors/warnings/info messages
• Any architectural pattern violations

Phase 5: Determine Scan Scope

Default Scan Types (run all unless user specifies):

1. critical - Critical bugs (crashes, security, resource leaks)
2. logic - Logic errors (algorithms, edge cases, type guards)
3. cache - Caching issues (staleness, races, invalidation)
4. workflow - Workflow problems (scripts, CI, git hooks)
5. workflow-optimization - CI optimization (build-required checks for cached builds)
6. security - GitHub Actions security (template injection, cache poisoning, etc.)
7. documentation - Documentation accuracy (README errors, outdated docs)

User Interaction: Use AskUserQuestion tool:

• Question: "Which quality scans would you like to run?"
• Header: "Scan Types"
• multiSelect: true
• Options:
  • "All scans (recommended)" → Run all 4 scan types
  • "Critical only" → Run critical scan only
  • "Critical + Logic" → Run critical and logic scans
  • "Custom selection" → Ask user to specify which scans

Default: If user doesn't specify, run all scans.

If user requests non-existent scan type, report error and suggest valid types.

Phase 6: Execute Scans

// Example: Critical scan
Task({
  subagent_type: "general-purpose",
  description: "Critical bugs scan",
  prompt: `${CRITICAL_SCAN_PROMPT_FROM_REFERENCE_MD}

[IF monorepo] Focus on packages/ directories and root-level scripts/.
[IF single package] Focus on src/, lib/, and scripts/ directories.

Report findings in this format:
- File: path/to/file.mts:lineNumber
- Issue: Brief description
- Severity: Critical/High/Medium/Low
- Pattern: Code snippet
- Trigger: What input triggers this
- Fix: Suggested fix
- Impact: What happens if triggered

Scan systematically and report all findings. If no issues found, state that explicitly.`
})

For each scan:

1. Load agent prompt template from reference.md
2. Customize for repository context (determine monorepo vs single package structure)
3. Spawn agent with Task tool using "general-purpose" subagent_type
4. Capture findings from agent response
5. Parse and categorize results

Execution Order: Run scans sequentially in priority order:

• critical (highest priority)
• logic
• cache
• workflow (lowest priority)

Agent Prompt Sources:

• Critical scan: reference.md starting at line ~12
• Logic scan: reference.md starting at line ~100
• Cache scan: reference.md starting at line ~200
• Workflow scan: reference.md starting at line ~300
• Security scan: reference.md starting at line ~400
• Workflow-optimization scan: reference.md starting at line ~860
• Documentation scan: reference.md starting at line ~1040

Phase 7: Aggregate Findings

interface Finding {
  file: string           // "src/path/to/file.mts:89" or "packages/pkg/src/file.mts:89"
  issue: string          // "Potential null pointer access"
  severity: "Critical" | "High" | "Medium" | "Low"
  scanType: string       // "critical"
  pattern: string        // Code snippet showing the issue
  trigger: string        // What causes this issue
  fix: string            // Suggested code change
  impact: string         // What happens if triggered
}

Deduplication:

• Remove duplicate findings across scans (same file:line, same issue)
• Keep the finding from the highest priority scan
• Track which scans found the same issue

Prioritization:

• Sort by severity: Critical → High → Medium → Low
• Within same severity, sort by scanType priority
• Within same severity+scanType, sort alphabetically by file path

Phase 8: Generate Report

# Quality Scan Report

**Date:** YYYY-MM-DD
**Repository:** [repository name]
**Scans:** [list of scan types run]
**Files Scanned:** N
**Findings:** N critical, N high, N medium, N low

## Dependency Updates

**Status:** N packages updated
**Result:** Success/Failed

## Structural Validation

**Consistency Checker Results:**
- Packages validated: 12
- Errors: N (reported as Critical below)
- Warnings: N (reported as Low below)
- Info: N observations

**Validation Categories:**
✓ Required files
✓ Vitest configurations
✓ Test scripts
✓ Coverage scripts
✓ External tools
✓ Build output structure
✓ Package.json structure
✓ Workspace dependencies

## Critical Issues (Priority 1) - N found

### src/path/to/file.mts:89
- **Issue**: [Description of critical issue]
- **Pattern**: [Problematic code snippet]
- **Trigger**: [What triggers this issue]
- **Fix**: [Suggested fix]
- **Impact**: [Impact description]
- **Scan**: critical

## High Issues (Priority 2) - N found

[Similar format for high severity issues]

## Medium Issues (Priority 3) - N found

[Similar format for medium severity issues]

## Low Issues (Priority 4) - N found

[Similar format for low severity issues]

## Scan Coverage

- **Dependency updates**: N packages updated
- **Structural validation**: [IF consistency checker exists] N packages validated, N architectural patterns checked
- **Critical scan**: N files analyzed in [src/ or packages/]
- **Logic scan**: N files analyzed
- **Cache scan**: N files analyzed (if applicable)
- **Workflow scan**: N files analyzed (package.json, scripts/, .github/)

## Recommendations

1. Address N critical issues immediately before next release
2. Review N high-severity logic errors in patch application
3. Schedule N medium issues for next sprint
4. Low-priority items can be addressed during refactoring

## No Findings

[If a scan found no issues, list it here:]
- Critical scan: ✓ Clean
- Logic scan: ✓ Clean

Output Report:

1. Display report to console (user sees it)
2. Offer to save to file (optional): reports/quality-scan-YYYY-MM-DD.md

Phase 9: Complete

Quality Scan Complete

✓ Dependency updates: N packages updated ✓ Structural validation: [IF applicable] N packages validated (N errors, N warnings) ✓ Repository cleanup: N junk files removed ✓ Scans completed: [list of scan types] ✓ Total findings: N (N critical, N high, N medium, N low) ✓ Files scanned: N ✓ Report generated: Yes ✓ Scan duration: [calculated from start to end]

Dependency Update Summary:

• Packages updated: N
• Update status: Success/Failed

Structural Validation Summary: [IF consistency checker exists]

• Packages validated: N
• Consistency errors: N (included in critical findings)
• Consistency warnings: N (included in low findings)
• Architectural patterns checked: N

Repository Cleanup Summary:

• SCREAMING_TEXT.md files removed: N
• Temporary test files removed: N
• Temp/backup files removed: N
• Log files cleaned up: N

Critical Issues Requiring Immediate Attention:

• N critical issues found
• Review report above for details and fixes

Next Steps:

1. Address critical issues immediately
2. Review high-severity findings
3. Schedule medium/low issues appropriately
4. Re-run scans after fixes to verify

All findings include file:line references and suggested fixes.

Success Criteria

• ✅ <promise>QUALITY_SCAN_COMPLETE</promise> output
• ✅ All enabled scans completed without errors
• ✅ Findings prioritized by severity (Critical → Low)
• ✅ All findings include file:line references
• ✅ Actionable suggestions provided for all findings
• ✅ Report generated with statistics and coverage metrics
• ✅ Duplicate findings removed

Scan Types

See reference.md for detailed agent prompts with structured tags:

• critical-scan - Null access, promise rejections, race conditions, resource leaks
• logic-scan - Off-by-one errors, type guards, edge cases, algorithm correctness
• cache-scan - Invalidation, key generation, memory management, concurrency
• workflow-scan - Scripts, package.json, git hooks, CI configuration
• workflow-optimization-scan - CI optimization checks (build-required on installation steps with checkpoint caching)
• security-scan - GitHub Actions workflow security (runs zizmor scanner)
• documentation-scan - README accuracy, outdated examples, incorrect package names, missing documentation, junior developer friendliness (beginner-friendly explanations, troubleshooting, getting started guides)

All agent prompts follow Claude best practices with , , , , and tags.

Commands

This skill is self-contained. No external commands needed.

Context

This skill provides systematic code quality analysis by:

• Spawning specialized agents for targeted analysis
• Using Task tool to run agents autonomously
• Embedding agent prompts in reference.md following best practices
• Generating prioritized, actionable reports
• Supporting partial scans (user can select specific scan types)

For detailed agent prompts with best practices structure, see reference.md.

Converted and distributed by TomeVault — claim your Tome and manage your conversions.