Pskoett Measuring AI Proficiency Customize Measurement

Customize Measurement

Generate a customized .ai-proficiency.yaml configuration through a guided interview process.

When to Use

• First time setting up measure-ai-proficiency for a repository
• Current scoring doesn't match your team's structure
• Want to hide irrelevant recommendations (e.g., no MCP, no Gas Town)
• Your team uses different file names (e.g., SYSTEM_DESIGN.md instead of ARCHITECTURE.md)

Workflow

Phase 1: Interview

Ask questions in thematic batches of 2-3 questions using AskUserQuestion.

Batch 1: AI Tools

1. Which AI coding assistants does your team use?
   - Claude Code
   - GitHub Copilot
   - Cursor
   - OpenAI Codex
   - Multiple (specify)

2. Is one tool primary, or do you use them equally?

Batch 2: Documentation Conventions

1. Do you use different file names for documentation?
   Examples:
   - SYSTEM_DESIGN.md instead of ARCHITECTURE.md
   - CODING_STANDARDS.md instead of CONVENTIONS.md
   - docs/api/README.md instead of API.md

2. Where does your team store documentation?
   - Root level (ARCHITECTURE.md)
   - docs/ folder
   - documentation/ folder
   - Other location

Batch 3: Focus & Scope

1. Which capabilities are NOT relevant to your team? (Select all that apply)
   - hooks (Claude hooks)
   - commands (Claude slash commands)
   - skills (Agent skills)
   - memory (Memory files like LEARNINGS.md)
   - agents (Multi-agent setup)
   - mcp (MCP server configs)
   - beads (Beads memory system)
   - gastown (Gas Town orchestration)

2. What's your priority focus?
   - documentation (ARCHITECTURE.md, CONVENTIONS.md)
   - skills (Agent skills)
   - testing (Test documentation)
   - architecture (System design docs)
   - All equally

Batch 4: Thresholds & Industry

1. Is the default scoring appropriate for your repo?
   - Too strict (small team/startup - lower thresholds)
   - About right (default thresholds)
   - Too lenient (enterprise - higher thresholds)

2. Any industry-specific patterns to include?
   - FinTech (COMPLIANCE.md, PCI_DSS.md, SECURITY_STANDARDS.md)
   - Healthcare (HIPAA.md, PHI_HANDLING.md)
   - Open Source (GOVERNANCE.md, MAINTAINERS.md)
   - Enterprise (SOC2.md, SECURITY_AUDIT.md)
   - None / General

Batch 5: Validation (Optional)

1. Does your documentation mention example file names that don't exist?
   - Yes (meta-tools, template repos, documentation showing patterns)
   - No (all file references point to real files)

2. If yes, which patterns should be skipped during validation?
   - Best practices not yet adopted (.mcp.json, .claude/settings.json)
   - Industry examples (HIPAA.md, COMPLIANCE.md)
   - Custom examples (ask user to specify)

Phase 2: Generate Configuration

Based on interview responses, generate .ai-proficiency.yaml:

# .ai-proficiency.yaml
# Generated by customize-measurement skill
# Customized for: [repo name]

# AI tools in use
tools:
  - claude-code  # or detected tools

# Custom file locations (if different from defaults)
documentation:
  # architecture: "docs/SYSTEM_DESIGN.md"
  # conventions: "CODING_STANDARDS.md"
  # api: "docs/api/README.md"
  # testing: "docs/TESTING_GUIDE.md"

# Level thresholds (adjusted for team size)
thresholds:
  level_3: 15   # Comprehensive Context
  level_4: 12   # Skills & Automation
  level_5: 10   # Multi-Agent Ready
  level_6: 8    # Fleet Infrastructure
  level_7: 6    # Agent Fleet
  level_8: 5    # Custom Orchestration

# Skip irrelevant recommendations
skip_recommendations:
  # - hooks
  # - commands
  # - skills
  # - memory
  # - agents
  # - mcp
  # - beads
  # - gastown

# Focus areas (optional - show only these)
# focus_areas:
#   - documentation
#   - skills
#   - testing

# Skip validation warnings for documentation examples
# skip_validation_patterns:
#   - "COMPLIANCE.md"      # Example mentioned in docs
#   - ".mcp.json"          # Best practice not yet adopted

Phase 3: Validate

After generating the config:

measure-ai-proficiency

Verify the output matches expectations. Adjust config if needed.

Threshold Guidelines

Industry Patterns

FinTech

documentation:
  compliance: "COMPLIANCE.md"
  security: "SECURITY_STANDARDS.md"
  pci: "PCI_DSS.md"

Healthcare

documentation:
  hipaa: "HIPAA.md"
  phi: "PHI_HANDLING.md"
  security: "SECURITY_COMPLIANCE.md"

Open Source

documentation:
  governance: "GOVERNANCE.md"
  maintainers: "MAINTAINERS.md"
  contributing: "CONTRIBUTING.md"

Example Session

User: /customize-measurement

Claude: I'll help you create a customized .ai-proficiency.yaml for your repository.
        Let me ask a few questions to understand your setup.

[AskUserQuestion - AI Tools batch]
1. Which AI coding assistants does your team use?
2. Is one the primary tool?

User: We use Claude Code primarily, with some GitHub Copilot usage.

[AskUserQuestion - Documentation batch]
1. Do you use different file names for documentation?
2. Where do you store docs?

User: We use docs/DESIGN.md instead of ARCHITECTURE.md. Everything is in docs/.

[AskUserQuestion - Focus batch]
1. Which capabilities aren't relevant?
2. What's your priority focus?

User: We don't use MCP or Gas Town. Focus on documentation and testing.

[AskUserQuestion - Thresholds batch]
1. Is default scoring appropriate?
2. Any industry patterns?

User: We're a small startup, defaults are too strict. No special industry.

Claude: Based on your responses, here's your customized config:

[Generates .ai-proficiency.yaml with:
 - tools: claude-code, github-copilot
 - documentation.architecture: docs/DESIGN.md
 - thresholds lowered for startup
 - skip_recommendations: mcp, gastown
 - focus_areas: documentation, testing]

Run `measure-ai-proficiency` to validate.

Quick Reference

Triggers:

• "Customize AI proficiency measurement"
• "Set up .ai-proficiency.yaml"
• "Configure measurement for my repo"
• "The scoring doesn't match my setup"

Converted and distributed by TomeVault — claim your Tome and manage your conversions.