Kimmo Agent Friendly Score

Agent-Friendly Score - DevTool Evaluation

Evaluate developer tools and SaaS products for compatibility with AI coding assistants (Cursor, Claude, GitHub Copilot).

When to Use

• User asks "How agent-friendly is [tool]?"
• User wants to evaluate a devtool for AI compatibility
• User is building a devtool and wants to optimize for AI assistants
• User is comparing tools and AI compatibility matters

The Agent Era Context

85% of developers now use AI tools regularly. When a developer asks Cursor to "add email functionality," the AI picks the service, writes the integration, and runs the install command.

The New Funnel:

• Traditional: Marketing → Landing → Docs → Trial → Conversion
• Agent: Problem → AI suggestion → npm install → Subscription

Tools that AI can easily work with get recommended. Tools it can't work with become invisible.

Scoring Framework

Category 1: SDK & API Design (30 points)

Scoring guide:

• 25-30: Excellent - AI can generate correct code first try
• 15-24: Good - AI mostly succeeds, occasional fixes needed
• 0-14: Poor - AI struggles to generate working code

Category 2: Documentation Quality (25 points)

Scoring guide:

• 20-25: AI can extract and apply correctly
• 10-19: AI needs some interpretation
• 0-9: AI will likely hallucinate or fail

Category 3: Training Data Presence (20 points)

Scoring guide:

• 15-20: Strong training data signal
• 8-14: Moderate presence
• 0-7: AI may not know this tool well

Category 4: MCP Integration (15 points)

Scoring guide:

• 12-15: Full agent workflow integration
• 5-11: Partial integration
• 0-4: Not in agent workflow

Category 5: Time to Working (10 points)

Scoring guide:

• 8-10: Instant productivity
• 4-7: Reasonable setup
• 0-3: Significant friction

Evaluation Workflow

Step 1: Identify the Tool

Get from user:

• Tool name and URL
• Category (email, auth, database, etc.)
• Main competitor to compare against

Step 2: SDK Evaluation

Check official SDK:

# Check npm for TypeScript types
npm info [package] types

# Check for SDK in multiple languages
# Visit: github.com/[org] and look for SDK repos

Test API consistency:

• Are endpoints predictable? (e.g., /users, /users/:id)
• Are responses consistent?
• Are errors structured?

Step 3: Documentation Audit

Visit docs and check:

• First code example is within scroll view
• Examples include all necessary imports
• Examples actually work when copied
• Structure uses semantic headings
• No authentication required to view

Step 4: Training Data Check

Search GitHub:

"import { X } from '[package]'" language:JavaScript
"from [package] import" language:Python

Search Stack Overflow:

[tool] is:question

Step 5: MCP Check

Search for MCP server:

• https://mcp.so - search for tool name
• Official docs - search for "MCP" or "Model Context Protocol"
• GitHub - search "[tool] mcp server"

Step 6: Time Test

Follow quickstart:

1. Start timer
2. Follow official quickstart exactly
3. Stop when first API call succeeds
4. Record time and friction points

Output Template

# Agent-Friendly Score: [Tool Name]

## Overall Score: [X]/100

| Category         | Score | Max |
| ---------------- | ----- | --- |
| SDK & API Design | X     | 30  |
| Documentation    | X     | 25  |
| Training Data    | X     | 20  |
| MCP Integration  | X     | 15  |
| Time to Working  | X     | 10  |

## Grade: [A/B/C/D/F]

- A (85-100): AI will recommend and integrate correctly
- B (70-84): AI will usually succeed
- C (55-69): AI needs help, may hallucinate
- D (40-54): Significant AI compatibility issues
- F (<40): AI will struggle or avoid

## Breakdown

### SDK & API Design ([X]/30)

**Strengths:**

- [What works well]

**Gaps:**

- [What's missing]

### Documentation ([X]/25)

**Strengths:**

- [What works well]

**Gaps:**

- [What's missing]

### Training Data Presence ([X]/20)

- GitHub repos found: [X]
- Stack Overflow questions: [X]
- Tutorial coverage: [High/Medium/Low]

### MCP Integration ([X]/15)

- MCP server: [Official/Community/None]
- Status: [Active/Stale/N/A]

### Time to Working ([X]/10)

- Quickstart time: [X minutes]
- Friction points: [list]

## Recommendations

### Quick Wins (High Impact, Low Effort)

1. [Recommendation]
2. [Recommendation]

### Strategic Improvements

1. [Recommendation]
2. [Recommendation]

## Competitor Comparison

| Metric          | [Tool] | [Competitor] |
| --------------- | ------ | ------------ |
| Agent Score     | X/100  | Y/100        |
| MCP Server      | Yes/No | Yes/No       |
| Time to Working | X min  | Y min        |

## Verdict

[One paragraph summary of whether this tool is positioned for the agent era]

Benchmarks by Category

Email APIs

Authentication

Databases

Use these as calibration when scoring.

Key Insight

The best technical product doesn't always win anymore. The most AI-accessible product wins. When an AI assistant can:

1. Understand your docs
2. Generate working code
3. Integrate via MCP

...you're in the conversation. When it can't, you're invisible to the fastest-growing developer segment.

By Kimmo Ihanus | kimmoihanus.com