Results for “skill-evaluation”

For analysis/task/review agents using a Professional Skill on models, RAG, agents, evaluation, or safety; not for work without AI decision impact.

Design structured evaluation rubrics for scoring LLM and agent outputs — defining quality dimensions, scoring scales, hard gates, score descriptions, and edge cases. Load when the user asks to create an eval rubric, define evaluation criteria, design scoring dimensions, write an eval spec, or says "what should I evaluate", "design a rubric", "create eval criteria", "define quality dimensions", "evaluation rubric for", "how do I measure quality of". Sub-skill of eval-output orchestrator.

Evaluate Agent Skill design quality against official specifications and best practices. Use when reviewing, auditing, or improving SKILL.md files and skill packages. Provides multi-dimensional scoring and actionable improvement suggestions.

Internal skill. Called by setup-evaluation after a PASS. Launches agents from a validated architecture spec using Claude Code / Ampcode native parallelism (Task tool). Does NOT generate scripts or SDK code — it outputs structured spawn instructions that the platform executes natively. Never invoked directly by the user. Never launches without a setup-evaluation PASS.

Create, synthesize, and iteratively improve agent skills following the Agent Skills specification. Handles source capture, precision passes, authoring, registration, and validation.

Iteratively reviews and fixes Claude Code skill quality issues by running automated fix-review cycles using the skill-reviewer agent until they meet standards.

Analyze the available skill set and recommend candidate groups that could be consolidated into sub-skill bundles. Read-only — proposes a plan, does not move files.

Audit an existing processkit skill against the 5 Skill Killers, the Agent Skills standard, and Anthropic's troubleshooting playbook, producing a categorized findings report and draft Gotchas.

Use when creating test scenarios for a skill, setting up a tests/ directory for a plugin or package, running skill tests against a subagent, or verifying whether a SKILL.md changes agent behavior

Improves AI skills for activation, clarity, and cross-model reliability through benchmarking, salience tuning, and regression triage.

Use when creating, editing, evaluating, testing, or verifying ANY skill or skill-related file (SKILL.md, skill resources, skill scripts, or skill assets). If you're asked to evaluate or test a skill's effectiveness, use this skill.

当用户想测试或复盘某个 skill 的实际执行效果时使用——如“测一下这个 skill”“看看 agent 是否按 skill 执行”“生成 skill 覆盖率或溯源报告”。在隔离沙箱中发起一次 headless 盲测，逐步骤溯源并产出中文复盘报告。不要用于编写新 skill、调试业务代码、代码审查，或复盘当前会话已经发生的执行。

Review AI agent skills using a dual-axis method: deterministic code-based checks and LLM deep review, with weighted scoring and improvement recommendations.

Provides production-grade techniques for evaluating LLM outputs using LLMs as judges, covering direct scoring, pairwise comparison, bias mitigation, rubric generation, and confidence calibration.

Evaluate model quality with task-appropriate metrics and systematic error analysis. Use when: (1) comparing models, (2) analyzing failures, (3) setting go/no-go thresholds. NOT for: production monitoring implementation.

Find out why a coding-agent skill won't fire — grade each SKILL.md A–F on activation, simulate which skill a prompt triggers, and flag collisions where one silently shadows another.

Evaluates Claude Agent Skills on 10 quality axes with letter grades (A+ through F) and specific improvement recommendations. Use when auditing a skill, comparing skills, prioritizing improvements, or performing quality control on a skill library. Activate on "grade skill", "evaluate skill", "skill quality", "skill audit", "skill review", "rate skill". NOT for creating skills (use skill-architect), grading code quality, or evaluating non-skill documents.

Use this skill for search indexing, embeddings, RAG chunking, freshness, retrieval evaluation, source citations. Trigger when the task involves ai engineering work related to Search Indexing RAG, implementation, audits, debugging, strategy, or validation.

Expert skill for building LLM applications with the LlamaIndex framework — RAG pipelines, multi-agent orchestration, event-driven workflows, knowledge graph construction, production deployment, and evaluation. Use when working with LlamaIndex or comparing RAG and agent orchestration frameworks.

Provides guidance for automatically evolving and optimizing AI agents across any domain using LLM-driven evolution algorithms. Use when building self-improving agents, optimizing agent prompts and skills against benchmarks, or implementing automated agent evaluation loops.

Route AI agent engineering prompts to architecture, orchestration, evaluation, safety, debugging, context, prompt, MCP, persona, local AI, and Compound Engineering skills. Use when prompts mention agents, agent harnesses, agentic workflows, orchestration, evals, context management, MCP servers, or compound engineering.

Evaluates text-to-image generation models by decomposing assessment into five specialized skills (object presence, count, spatial relations, scale, and text rendering) and open-ended prompts, producing interpretable binary scores with visual and textual explanations.

Evaluate everything the PA agent manages — tasks, skills, PA network health, billing, calendar connections, and memory quality. Use when: owner asks for an evaluation, wants to know what's working and what isn't, or requests a performance report. Combines supervisor status with quality scoring.

Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when: - Implementing self-critique and reflection loops - Building evaluator-optimizer pipelines for quality-critical generation - Creating test-driven code refinement workflows - Designing rubric-based or LLM-as-judge evaluation systems - Adding iterative improvement to agent outputs (code, reports, analysis) - Measuring and improving agent response quality

Orchestrator for the eval-output skill suite — evaluate LLM and agent outputs for quality, accuracy, helpfulness, and safety using structured rubrics and LLM-as-judge techniques. Load when the user says "evaluate this output", "score this response", "run an eval", "LLM as judge", "evaluate agent output", "how good is this response", "rate this answer", "eval this", or provides an LLM output that should be assessed for quality. Single entry point for all output evaluation workflows.

Use this skill to classify tabular data end-to-end in MATLAB — load a dataset, prepare and clean it, select promising classifiers, train them, and compare accuracies with cross-validation, holdout, or hyperparameter optimization plus statistical tests. TRIGGER when: user asks to classify tabular data, pick classifiers for a dataset, compare classifier accuracy, run cross-validation or a holdout evaluation, or find the best model with statistical uncertainty. DO NOT TRIGGER when: user has non-tabular inputs (images, sequences, time series), wants a regression model, is training a specific neural network architecture (use matlab-train-network), or wants cost-sensitive learning or an arbitrary class-prior vector (this skill only supports the built-in uniform-prior toggle for imbalanced data).