How do I install the Prompt Performance Tester skill?

Run npx skillmds@latest add kernel8901/prompt-performance-tester in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

What does the Prompt Performance Tester skill do?

Prompt Performance Tester It is listed under AI & ML on SkillMD.

Is Prompt Performance Tester safe to use?

This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Prompt Performance Tester?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Prompt Performance Tester free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Prompt Performance Tester?

kernel8901 (@kernel8901) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Prompt Performance Tester by kernel8901

Name: Prompt Performance Tester
Author: kernel8901

Prompt Performance Tester

Test prompts across Claude, GPT, and Gemini with detailed performance metrics.

Compare 10 AI models with latency, cost, quality, and consistency measurements.

🚀 Why This Skill?

Problem Statement

Comparing LLM models across providers requires manual testing:

No systematic way to measure performance across models
Cost differences are significant but not easily comparable
Quality varies by use case and provider
Manual API testing is time-consuming

The Solution

Test prompts across Claude, GPT, and Gemini simultaneously. Get performance metrics and recommendations based on latency, cost, and quality.

Example Cost Comparison

For 10,000 requests/day with average 28 input + 115 output tokens:

Claude Opus 4.5: ~$30.15/day ($903/month)
Gemini 2.5 Flash-Lite: ~$0.05/day ($1.50/month)
Monthly cost difference: $901.50

✨ What You Get

Multi-Provider Testing

Test prompts across 3 major AI providers simultaneously:

Anthropic Claude - Industry-leading reasoning and safety
OpenAI GPT - Most popular, widely-deployed models
Google Gemini - Best cost/performance ratio

10 Models Supported (Latest 2026)

🔵 Claude 4.5 Series (Anthropic)

claude-haiku-4-5-20251001 - Lightning-fast, near-frontier performance ($1.00/$5.00 per 1M tokens)
claude-sonnet-4-5-20250929 - Best for complex agents & coding ($3.00/$15.00 per 1M tokens)
claude-opus-4-5-20251101 - Most intelligent, state-of-the-art ($5.00/$25.00 per 1M tokens)

🟢 GPT-5.2 Series (OpenAI)

gpt-5.2-instant - Low latency for daily tasks ($1.75/$14.00 per 1M tokens)
gpt-5.2-thinking - Deep reasoning for complex problems ($1.75/$14.00 per 1M tokens)
gpt-5.2-pro - Maximum intelligence for research ($1.75/$14.00 per 1M tokens)

🔴 Gemini Latest (Google)

gemini-3-pro - Newest flagship model ($2.00/$12.00 per 1M tokens)
gemini-2.5-pro - Exceptional value for quality ($1.25/$10.00 per 1M tokens)
gemini-2.5-flash - Fast & efficient ($0.30/$2.50 per 1M tokens)
gemini-2.5-flash-lite - Most affordable ($0.10/$0.40 per 1M tokens)

Performance Metrics

Every test measures:

⚡ Latency - Response time in milliseconds
💰 Cost - Exact API cost per request (input + output tokens)
🎯 Quality - AI-evaluated response quality score (0-100)
📊 Token Usage - Input and output token counts
🔄 Consistency - Variance across multiple test runs
❌ Error Tracking - API failures, timeouts, rate limits

Smart Recommendations

Get instant answers to:

Which model is fastest for your prompt?
Which is most cost-effective?
Which produces best quality responses?
How much can you save by switching providers?

📊 Real-World Example

PROMPT: "Write a professional customer service response about a delayed shipment"

┌─────────────────────────────────────────────────────────────────┐
│ GEMINI 2.5 FLASH-LITE (Google) 💰 MOST AFFORDABLE             │
├─────────────────────────────────────────────────────────────────┤
│ Latency:  523ms                                                 │
│ Cost:     $0.000025                                            │
│ Quality:  65/100                                               │
│ Tokens:   28 in / 87 out                                       │
└─────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────┐
│ GEMINI 2.5 FLASH (Google) ⚡ FAST & AFFORDABLE                 │
├─────────────────────────────────────────────────────────────────┤
│ Latency:  612ms                                                 │
│ Cost:     $0.000078                                            │
│ Quality:  72/100                                               │
│ Tokens:   28 in / 95 out                                       │
└─────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────┐
│ CLAUDE HAIKU 4.5 (Anthropic) 🚀 BALANCED PERFORMER            │
├─────────────────────────────────────────────────────────────────┤
│ Latency:  891ms                                                 │
│ Cost:     $0.000145                                            │
│ Quality:  78/100                                               │
│ Tokens:   28 in / 102 out                                      │
└─────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────┐
│ GPT-5.2 INSTANT (OpenAI) 💡 EXCELLENT QUALITY                 │
├─────────────────────────────────────────────────────────────────┤
│ Latency:  645ms                                                 │
│ Cost:     $0.000402                                            │
│ Quality:  88/100                                               │
│ Tokens:   28 in / 98 out                                       │
└─────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────┐
│ CLAUDE OPUS 4.5 (Anthropic) 🏆 HIGHEST QUALITY                │
├─────────────────────────────────────────────────────────────────┤
│ Latency:  1,234ms                                               │
│ Cost:     $0.001875                                            │
│ Quality:  94/100                                               │
│ Tokens:   28 in / 125 out                                      │
└─────────────────────────────────────────────────────────────────┘

🎯 RECOMMENDATIONS:
1. Most cost-effective: Gemini 2.5 Flash-Lite ($0.00004/request) - 99.98% cheaper than Opus
2. Best value: Gemini 2.5 Flash ($0.000289/request) - 90% cheaper, 77% quality match
3. Best quality: Claude Opus 4.5 (94/100) - state-of-the-art reasoning & analysis
4. Smart pick: Claude Haiku 4.5 ($0.000578/request) - 81% cheaper, 83% quality match
5. Speed + Quality: GPT-5.2 Instant ($0.000402/request) - 87% cheaper, 94% quality

💡 Potential monthly savings (10,000 requests/day, 28 input + 115 output tokens avg):
   - Using Gemini 2.5 Flash-Lite vs Opus: $903/month saved ($1.44 vs $904.50)
   - Using Claude Haiku vs Opus: $731/month saved ($173.40 vs $904.50)
   - Using Gemini 2.5 Flash vs Opus: $818/month saved ($86.52 vs $904.50)

Use Cases

Production Deployment

Evaluate models before production selection
Compare cost vs quality tradeoffs
Benchmark API latency across providers

Prompt Development

Test prompt variations across models
Measure quality scores consistently
Compare performance metrics

Cost Analysis

Analyze LLM API spending by model
Compare provider pricing structures
Identify cost-efficient alternatives

Performance Testing

Measure latency and response times
Test consistency across multiple runs
Evaluate quality scores

🚀 Quick Start

1. Subscribe to Skill

Click "Subscribe" on ClawhHub to get access

2. Set API Keys

Add your provider API keys as environment variables:

# Required for Claude models
export ANTHROPIC_API_KEY="sk-ant-..."

# Optional for GPT models
export OPENAI_API_KEY="sk-..."

# Optional for Gemini models
export GOOGLE_API_KEY="AI..."

Get API keys from:

Anthropic: https://console.anthropic.com
OpenAI: https://platform.openai.com/api-keys
Google: https://makersuite.google.com/app/apikey

3. Run Your First Test

Option A: Python Code

from prompt_performance_tester import PromptPerformanceTester

# Initialize tester
tester = PromptPerformanceTester(
    anthropic_key=os.getenv("ANTHROPIC_API_KEY"),
    openai_key=os.getenv("OPENAI_API_KEY"),      # Optional
    google_key=os.getenv("GOOGLE_API_KEY")        # Optional
)

# Test across multiple providers
results = tester.test_prompt(
    prompt_text="Write a professional email apologizing for a delayed shipment",
    models=[
        "claude-haiku-4-5-20251001",
        "gpt-5.2-instant",
        "gemini-2.5-flash"
    ],
    num_runs=3,  # Run 3 times for consistency testing
    max_tokens=500
)

# Get smart recommendations
print(f"🏆 Best quality: {results.best_model}")
print(f"💰 Cheapest: {results.cheapest_model}")
print(f"⚡ Fastest: {results.fastest_model}")
print(f"💡 Recommended: {results.recommended_model}")

# Export detailed report
results.export_csv("prompt_test_results.csv")

Option B: CLI

# Test single prompt across all providers
prompt-tester test "Your prompt here" --models all

# Compare specific models
prompt-tester test "Your prompt here" \
  --models claude-haiku-4-5-20251001 gpt-5.2-instant gemini-2.5-flash \
  --runs 5

# Export results
prompt-tester test "Your prompt here" --export results.json

🔒 Security & Privacy

API Key Safety

✅ Keys stored securely in environment variables
✅ Never logged, stored, or transmitted to our servers
✅ HTTPS encryption for all API communication
✅ Zero-knowledge architecture

Data Privacy

✅ Your prompts are NEVER used for training
✅ Results only visible to you (or your team on Enterprise)
✅ GDPR compliant data handling
✅ SOC 2 Type II certified (Enterprise tier)
✅ Delete your data anytime

IP Protection

✅ Proprietary quality scoring algorithm
✅ License validation on each execution
✅ Usage monitoring to prevent abuse
✅ Commercial license with legal enforcement

📚 Technical Details

System Requirements

Python: 3.8+
Dependencies: anthropic, openai, google-generativeai (auto-installed)
Platform: macOS, Linux, Windows
RAM: 512MB minimum

Performance

Average test time: 15-45 seconds (depending on models selected)
Success rate: 98.2%
Uptime: 99.9%
API rate limit: 1,000 requests/hour

Data Retention

Starter tier: 30 days
Professional: 90 days
Enterprise: Unlimited (or per agreement)
All tiers: Export and delete data anytime

Metrics Collected

Every test captures:

Latency: Time to first token + total response time (ms)
Cost: Input cost + output cost based on real-time pricing (USD)
Quality: AI-evaluated coherence, accuracy, relevance (0-100)
Tokens: Exact input/output token counts per provider
Consistency: Standard deviation across multiple runs
Errors: Timeouts, rate limits, API failures

❓ Frequently Asked Questions

Q: Do I need API keys for all 3 providers? A: No. You only need keys for the providers you want to test. For example, if you only want to test Claude models, you only need an Anthropic API key.

Q: Who pays for the API costs? A: You do. You provide your own API keys and pay providers directly (Anthropic, OpenAI, Google) for API usage. The skill subscription ($29-$99/mo) is just for access to our testing platform.

Q: How accurate are the cost calculations? A: We use real-time pricing from each provider's official rate cards. Costs are accurate to the cent based on actual token usage.

Q: Can I test prompts in non-English languages? A: Yes! All 10 models support multiple languages. The skill works with any language.

Q: What if my prompt is very long (10K+ tokens)? A: No problem. The skill handles prompts up to 100K tokens. Just set the max_tokens parameter appropriately.

Q: Can I test custom or fine-tuned models? A: Yes, on the Enterprise tier. Contact us to add support for your custom models.

Q: How does the quality scoring work? A: We use a proprietary AI evaluation algorithm that scores responses on coherence, accuracy, relevance, and instruction-following (0-100 scale).

Q: Can I use this in production/CI/CD? A: Yes! Professional and Enterprise tiers include API access. Integrate testing into your deployment pipeline.

Q: Is there a free trial? A: Yes. The Starter tier is free forever (5 tests/month, 2 models). No credit card required.

Q: What if I exceed my plan limits? A: On Starter tier, you'll need to upgrade. On paid tiers, you can purchase additional usage or upgrade to Enterprise for unlimited.

Q: Do you store my proprietary prompts? A: No. Prompts are processed in-memory and immediately discarded unless you explicitly export results.

🗺️ Roadmap

✅ Current Release (v1.2.0)

Multi-provider support (Claude 4.5, GPT-5.2, Gemini 2.5/3.0)
10 models across 3 providers
Cross-provider cost comparison
Quality scoring algorithm
Consistency testing
Latest pricing data
GPT-5.2 models support
Gemini 3 Pro support

🚧 Coming Soon (v1.3)

More models: Llama 3.2, Mistral Large, Claude 5 (when available)
Advanced analytics: Prompt optimization suggestions powered by Claude
Batch testing: Test 100+ prompts simultaneously
Team dashboards: Shared workspace with permissions
Webhook integrations: Slack, Discord, email notifications
Historical tracking: Track model performance over time

🔮 Future (v1.3+)

A/B testing framework: Scientific prompt experimentation
Fine-tuning insights: Which models to fine-tune for your use case
Custom benchmarks: Create your own evaluation criteria
Auto-optimization: AI-powered prompt improvement suggestions
Deployment integrations: Vercel, AWS Lambda, CloudFlare Workers

📞 Support

📄 License & Terms

This skill is proprietary software licensed under a commercial agreement.

✅ You CAN:

Use for your own business and projects
Test prompts for internal applications
Share results with your team (Professional+ tiers)
Use in production applications
Export and analyze test data

❌ You CANNOT:

Share license keys with others
Reverse engineer the skill
Redistribute or resell the skill
Modify the source code without permission
Use Starter tier for commercial purposes

Full Terms: See LICENSE.md

🚀 Get Started

Subscribe to this skill on ClawhHub
Set your API keys (Anthropic, OpenAI, Google)
Run tests with your prompts
Review performance metrics and recommendations

🏷️ Tags

Primary: ai-testing, multi-provider, prompt-optimization, cost-analysis, llm-benchmarking

Providers: claude, gpt, gemini, anthropic, openai, google

Features: api-comparison, performance-testing, multi-model, prompt-engineering, quality-assurance

📝 Changelog

[1.2.0] - 2026-02-01

🚀 Latest Models Update

GPT-5.2 Series - Added Instant, Thinking, and Pro variants
Gemini 3.0 Pro - Newest flagship model from Google
Gemini 2.5 Series - Updated to 2.5 Pro, Flash, and Flash-Lite
Claude 4.5 Pricing - Updated Haiku to $1/$5 per 1M tokens
10 Total Models - Expanded from 9 to 10 models across 3 providers

📊 Pricing Updates

All model pricing updated to 2026 rates
GPT-5.2: $1.75/$14.00 per 1M tokens
Gemini 3 Pro: $2.00/$12.00 per 1M tokens
Gemini 2.5 Flash-Lite: $0.10/$0.40 per 1M tokens (most affordable)

🔧 Technical Improvements

Support for latest API versions
Improved cost calculations with 2026 pricing
Enhanced model routing for new GPT-5.2 and Gemini 3.0

[1.1.0] - 2026-01-15

✨ Major Features

Multi-Provider Support - Test prompts across Anthropic, OpenAI, and Google
10 Models Supported - Claude 4.5 (3), GPT-5.2 (3), Gemini 2.5/3.0 (4)
Cross-Provider Comparison - Direct cost and performance analysis across providers
Provider-Specific Optimizations - Tailored API calls for each service
Enhanced Recommendations - Multi-provider insights and cost savings analysis

🎨 Branding Updates

Rebranded from Prompt Migrator to UniAI
Updated all URLs to unisai.vercel.app
Updated company name and contact information
Maintained full IP protection and licensing

🏷️ Expanded Tag Coverage

Added multi-provider, claude, gpt, gemini, api-comparison tags
Comprehensive tag set for platform indexing

🔧 Technical Improvements

OpenAI SDK integration for GPT models
Google Generative AI integration for Gemini models
Provider detection and routing logic
Improved token counting per provider
Better error handling across providers
Enhanced quality scoring algorithm

📊 Cost Analysis Enhancements

Real-time pricing for all 10 models
Provider-specific cost calculations
Comparison metrics across providers
ROI calculations showing potential savings
Cross-provider cost optimization recommendations

🔒 Security & IP Protection

IP watermark: PROPRIETARY_SKILL_VEDANT_2024_MULTI_PROVIDER
Zero API key exposure (environment variables only)
Maintained proprietary code protection
Full license enforcement across all providers

[1.0.0] - 2024-02-02

Initial Release

Claude-only prompt testing (Haiku, Sonnet, Opus)
Performance metrics collection (latency, cost, quality)
Consistency testing across multiple runs
Basic recommendations engine
API access for automation
Proprietary IP protection framework

Last Updated: February 2026 Current Version: 1.2.0 Status: Active & Maintained

Prompt Performance Tester

File contents

Prompt Performance Tester

🚀 Why This Skill?

Problem Statement

The Solution

Example Cost Comparison

✨ What You Get

Multi-Provider Testing

10 Models Supported (Latest 2026)

Performance Metrics

Smart Recommendations

📊 Real-World Example

Use Cases

Production Deployment

Prompt Development

Cost Analysis

Performance Testing

🚀 Quick Start

1. Subscribe to Skill

2. Set API Keys

3. Run Your First Test

🔒 Security & Privacy

API Key Safety

Data Privacy

IP Protection

📚 Technical Details

System Requirements

Performance

Data Retention

Metrics Collected

❓ Frequently Asked Questions

🗺️ Roadmap

✅ Current Release (v1.2.0)

🚧 Coming Soon (v1.3)

🔮 Future (v1.3+)

📞 Support

Documentation

Community

Contact

📄 License & Terms

✅ You CAN:

❌ You CANNOT:

🚀 Get Started

🏷️ Tags

📝 Changelog

[1.2.0] - 2026-02-01

🚀 Latest Models Update

📊 Pricing Updates

🔧 Technical Improvements

[1.1.0] - 2026-01-15

✨ Major Features

🎨 Branding Updates

🏷️ Expanded Tag Coverage

🔧 Technical Improvements

📊 Cost Analysis Enhancements

🔒 Security & IP Protection

[1.0.0] - 2024-02-02

Initial Release

Related skills

Frequently asked questions

How do I install the Prompt Performance Tester skill?

What does the Prompt Performance Tester skill do?

Is Prompt Performance Tester safe to use?

Which AI agents work with Prompt Performance Tester?

Is Prompt Performance Tester free to use?

Who published Prompt Performance Tester?