AI Tool Assessment

AI Tool Usage Assessment

Assess and report on AI tool subscriptions, usage patterns, and cost-effectiveness.

Overview

This skill provides a structured framework for evaluating AI tool subscriptions, identifying underutilized services, and optimizing AI-related spending. It generates comprehensive reports with actionable recommendations.

Quick Start

1. Trigger assessment - Ask to "assess AI tools" or "review AI subscriptions"
2. Answer questions - Provide usage patterns when asked
3. Review report - Generated at reports/ai-tool-assessment/assessment-YYYYMMDD.md
4. Take action - Follow recommendations for optimization

# View latest assessment
cat reports/ai-tool-assessment/assessment-$(date +%Y%m%d).md

# List all assessments
ls -la reports/ai-tool-assessment/

When to Use

• Monthly subscription reviews
• Quarterly budget planning
• When evaluating new AI tools
• After significant workflow changes
• Cost optimization initiatives
• Annual AI strategy reviews

Trigger

User asks to assess AI tools, review AI subscriptions, or analyze AI tool usage.

Instructions

When triggered, perform the following assessment:

1. Subscription Inventory

Review current paid memberships from docs/AI_development_tools.md:

2. Tool Usage Analysis

Assess each tool category:

Primary AI Assistants:

• Claude Max: Code generation, complex reasoning, long-context tasks
• OpenAI Plus: Alternative perspective, GPT-4o access, DALL-E
• Google AI Pro: Gemini access, Google ecosystem integration

Development Tools:

• GitHub Copilot: Inline code completion, IDE integration
• Claude-flow: Multi-agent orchestration
• Factory.ai: Automated droids for CI/CD
• Google Antigravity: Agent-first IDE (in evaluation)

3. Generate Assessment Report

Create report at reports/ai-tool-assessment/assessment-YYYYMMDD.md with:

# AI Tool Usage Assessment - [DATE]

## Executive Summary
- Total monthly spend: $X
- Primary tools in active use: [list]
- Tools underutilized: [list]
- Recommended actions: [list]

## Subscription Status
[Table of all subscriptions with renewal dates if known]

## Usage Patterns
[Analysis of which tools are used for what purposes]

## Cost-Effectiveness Analysis
| Tool | Cost/Month | Usage Level | Value Rating |
|------|------------|-------------|--------------|
| ... | ... | High/Medium/Low | 1-5 stars |

## Overlap Analysis
[Identify redundant capabilities across tools]

## Recommendations
1. [Keep/Cancel/Downgrade recommendations]
2. [Usage optimization suggestions]
3. [New tools to consider]

## Next Review Date
[Set quarterly review schedule]

4. Questions to Ask User

Before generating report, ask:

1. Which tools have you used most this month?
2. Are there specific tasks where one tool excels?
3. Any tools you haven't used in 30+ days?
4. New capabilities you need that current tools lack?

Execution Checklist

Before assessment:

• Access to subscription documentation
• User available for questions
• Previous assessments reviewed (if any)

During assessment:

• All subscriptions inventoried
• Usage patterns documented
• Cost-effectiveness analyzed
• Overlap identified
• User questions asked

After assessment:

• Report generated in correct location
• Recommendations prioritized
• Next review date set
• Source docs updated if needed

Error Handling

Common Issues

Error: Missing subscription data

• Cause: docs/AI_development_tools.md not found or outdated
• Solution: Ask user for current subscription list, create/update doc

Error: Incomplete usage data

• Cause: User unsure about usage patterns
• Solution: Suggest tracking usage for 2 weeks before full assessment

Error: Can't determine value

• Cause: Tool usage overlaps with others
• Solution: Run overlap analysis first, identify unique value per tool

Error: Outdated pricing

• Cause: Subscription costs changed
• Solution: Verify current pricing on vendor websites

Output

• Assessment report in reports/ai-tool-assessment/
• Updated docs/AI_development_tools.md if status changes
• Summary printed to console

Metrics

Report Templates

Executive Summary Template

## Executive Summary

**Assessment Date:** YYYY-MM-DD
**Review Period:** [Month/Quarter]
**Total Monthly Spend:** $XXX.XX

### Key Findings
1. [Primary finding]
2. [Secondary finding]
3. [Tertiary finding]

### Immediate Actions Required
- [ ] [Action 1]
- [ ] [Action 2]

### Long-term Recommendations
1. [Recommendation with timeline]
2. [Recommendation with timeline]

Tool Comparison Template

## Tool Comparison Matrix

| Capability | Claude | OpenAI | Google | Copilot |
|------------|--------|--------|--------|---------|
| Code Generation | ★★★★★ | ★★★★☆ | ★★★☆☆ | ★★★★☆ |
| Long Context | ★★★★★ | ★★★☆☆ | ★★★★☆ | N/A |
| IDE Integration | ★★★★☆ | ★★☆☆☆ | ★★★☆☆ | ★★★★★ |
| Cost Value | ★★★☆☆ | ★★★★☆ | ★★★★★ | ★★★★★ |

Best Practices

1. Regular cadence - Assess quarterly at minimum
2. Track trends - Compare assessments over time
3. User feedback - Include qualitative input
4. Trial before commit - Use free tiers to evaluate
5. Document decisions - Record why tools kept/cancelled
6. Consider switching costs - Factor in workflow disruption

Related Skills

• background-service-manager - For running assessment scripts
• session-start-routine - Includes tool health checks

Version History

• 2.0.0 (2026-01-02): Upgraded to v2 template - added Quick Start, When to Use, Execution Checklist, Error Handling, Metrics sections; enhanced frontmatter with version, category, related_skills
• 1.0.0 (2024-10-15): Initial release with subscription inventory, usage analysis, cost-effectiveness reporting, recommendation framework