Plugins

3 plugins

Results for “skill-evaluation”

9 skills
More results
lionelndong
Skill Eval
Test a pipeline stage's skill file by running the stage WITH and WITHOUT the skill on the same input, comparing outputs, and proposing skill edits. Ryan Law principle 3 — recursive self-improvement. Run after any board complaint about a stage, and monthly per core stage.
0
tinh2
Evolve
Analyzes development cycle and quality metrics to identify recurring issues, then generates additive patches to skill instructions to prevent them.
13
alirezarezvani
Skill Tester
Validates, tests, and scores the quality of skills within a claude-skills ecosystem using Python scripts that check structure, syntax, runtime, and multi-dimensional quality metrics.
20.4k · bundle
tradermonty
Dual Axis Skill Reviewer
Review AI agent skills using a dual-axis method: deterministic code-based checks and LLM deep review, with weighted scoring and improvement recommendations.
2.3k · bundle
tinh2
Skill Scout
Scans a project for stack signals and pain points, searches the skills-hub registry and local skills, scores candidates, and presents a ranked shortlist with install commands.
13
dvy1987
Eval Pipeline
Design automated evaluation pipelines for LLM and agent systems — combining deterministic checks, statistical metrics, and LLM-as-judge scoring into repeatable, CI-integrated eval suites. Load when the user asks to set up automated evals, design an eval pipeline, integrate evals into CI/CD, create an eval suite, do eval-driven development, or says "automate my evals", "CI eval integration", "evaluation pipeline", "continuous evaluation", "monitoring eval quality", "set up regression testing for my agent". Sub-skill of eval-output orchestrator.
3 · bundle
akillness
Opik
Run Comet's Opik — open-source LLM observability, evaluation, and optimization — from one routing-first skill: install the Python/TypeScript SDK, stand up a server (Comet.com cloud, Docker Compose via `./opik.sh`, or Kubernetes/Helm), wire tracing through `@opik.track` or one of 50+ framework integrations (OpenAI, Anthropic, LangChain, LangGraph, LlamaIndex, CrewAI, DSPy, Ollama, Bedrock, Vercel AI SDK, …), score outputs with LLM-as-a-judge metrics (Hallucination, Moderation, Answer Relevance, Context Precision), and run Datasets/Experiments evaluations including PyTest CI gates. Use when the user wants LLM tracing, prompt evaluation, production LLM monitoring, agent optimization, or guardrails with Opik. Triggers on: opik, comet opik, opik configure, opik.sh, llm observability, llm tracing, llm as a judge, hallucination metric, prompt evaluation, opik dashboard, opik guardrails, agent optimizer.
42 · bundle