Results for “evaluation”

Audits Claude skills and commands for quality, offering Quick Scan for changed skills and Full Stocktake modes with sequential subagent batch evaluation.

Evaluates NVIDIA Cosmos Policy on LIBERO and RoboCasa simulation environments. Use when setting up cosmos-policy for robot manipulation evaluation, running headless GPU evaluations with EGL rendering, or profiling inference latency on cluster or local GPU machines.

Run a heuristic evaluation of interactive UIs against Nielsen's 10 usability heuristics and interaction add-ons, scoring the rendered artifact and producing a prioritized fix list.

Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.

Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.

DeepEval — LLM evaluation framework, RAG metrics, hallucination detection, red-teaming, CI/CD integration

Formal evaluation framework for Codex sessions implementing eval-driven development (EDD) principles

Create a new AgentHub collaboration session with task, agent count, and evaluation criteria.

Provides production-grade techniques for evaluating LLM outputs using LLMs as judges, covering direct scoring, pairwise comparison, bias mitigation, rubric generation, and confidence calibration.

Coordinates quality assurance workflows by routing testing tasks to specialized sub-skills for API testing, performance benchmarking, test analysis, tool evaluation, and process optimization.

Holistic evaluation framework for language models measuring accuracy, calibration, robustness, and fairness

Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles

Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles

Provides canonical MSBuild property definition patterns including conditional defaults, composition, path normalization, target framework detection, and evaluation order for diagnosing and fixing property issues in .props and .csproj files.

Use when designing or auditing experiments for an ACL paper, covering tuned LLM baselines, multi-dataset and multilingual evaluation, statistical significance and variance, human evaluation with agreement reporting, contamination and prompt-sensitivity controls, ablations, and error-analysis expectations in NLP reviewing.

Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when: langfuse, llm observability, llm tracing, prompt management, llm evaluation.

Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when: langfuse, llm observability, llm tracing, prompt management, llm evaluation.

Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when: langfuse, llm observability, llm tracing, prompt management, llm evaluation.

Giskard RAGET (RAG Evaluation Toolkit): automatic testset generation (simple / complex / distracting / conversational), component-level scoring (retriever / generator / rewriter), hallucination and bias tests, CI integration. Compared to RAGAS and DeepEval. USE WHEN: user mentions "Giskard", "RAGET", "Giskard RAG toolkit", "automatic testset generation", "component-level RAG scoring", "hallucination test Giskard" DO NOT USE FOR: general RAGAS usage - use `rag-evaluation`; Stanford ARES - use `ares-framework`; CI/CD wiring - use `continuous-evaluation`

Test coverage quality analysis — behavioral coverage, critical gap identification, test resilience evaluation.

Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles

Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles

Design structured evaluation rubrics for scoring LLM and agent outputs — defining quality dimensions, scoring scales, hard gates, score descriptions, and edge cases. Load when the user asks to create an eval rubric, define evaluation criteria, design scoring dimensions, write an eval spec, or says "what should I evaluate", "design a rubric", "create eval criteria", "define quality dimensions", "evaluation rubric for", "how do I measure quality of". Sub-skill of eval-output orchestrator.

LLM output evaluation — automated metrics, LLM-as-judge, A/B testing, regression testing. Use when measuring LLM output quality, comparing prompt or model versions, building an automated eval pipeline, setting up regression tests for prompt changes, or evaluating RAG systems and bias/safety.

编写评估FastText文本分类模型的Python函数，计算accuracy、F1、recall和precision指标，并处理特定格式的标签文本分割。

DSPy declarative framework for automatic prompt optimization treating prompts as code with systematic evaluation and compilers

Screens resumes with skill matching, experience evaluation, and bias-free candidate ranking

Design reproducible evaluations for AI agents with representative task sets, explicit rubrics, appropriate graders, baselines, regression gates, and failure analysis. Use when defining agent quality, comparing prompts or models, validating a release, measuring tool-use reliability, investigating regressions, or deciding whether an agent is ready for production.

Design automated evaluation pipelines for LLM and agent systems — combining deterministic checks, statistical metrics, and LLM-as-judge scoring into repeatable, CI-integrated eval suites. Load when the user asks to set up automated evals, design an eval pipeline, integrate evals into CI/CD, create an eval suite, do eval-driven development, or says "automate my evals", "CI eval integration", "evaluation pipeline", "continuous evaluation", "monitoring eval quality", "set up regression testing for my agent". Sub-skill of eval-output orchestrator.

Explain and apply core ML/AI concepts — model types, training pipelines, evaluation metrics, and neural architectures.

Create a role interview kit with role outcomes, interview plan, scorecard, evidence prompts, and evaluation criteria.

Machine learning in Python with scikit-learn. Use for classification, regression, clustering, model evaluation, and ML pipelines.

Write RFPs with requirements, evaluation, and timeline. TRIGGERS - Use when user needs help with rfp-writer related tasks.

Write RFPs with requirements, evaluation, and timeline. TRIGGERS - Use when user needs help with rfp-writer related tasks.

Derive owner/operator dogfood scenarios from product evidence and active-use cadence, then produce adoption instructions and manual evaluation checks

Architectural decision-making framework. Requirements analysis, trade-off evaluation, ADR documentation. Use when making architecture decisions or analyzing system design.