Results for “evaluation”

Rates an agent's own output on five axes — accuracy, completeness, clarity, actionability, conciseness — producing a structured scorecard with evidence and improvement suggestions.

Fine-tune models on Azure AI Foundry using SFT, DPO, or RFT, covering dataset preparation, training job submission, deployment, and evaluation.

Implement iterative evaluation and refinement loops for AI agent outputs, using self-critique, evaluator-optimizer patterns, and rubric-based scoring to improve quality.

Provides structured guidance for building production-grade AI agent systems through context engineering, covering fundamentals, architectural patterns, operational excellence, and evaluation.

Simulates meiotic recombination to produce offspring genomes from parent pairs, modeling Mendelian segregation, de novo mutation, sex determination, trait inference, and clinical evaluation against a disease registry.

Every metric encodes an opinion about which mistake hurts.

Build AI applications on Microsoft Foundry using the Azure AI Projects Python SDK, including agent creation, tool integration, thread management, and evaluation.

Guides the creation of high-quality MCP servers that let LLMs interact with external services through well-designed tools, covering planning, implementation, testing, and evaluation.

Evaluates the robustness and alignment with human judgment of reference-based and reference-free evaluation metrics for machine translation and summarization, particularly under adversarial conditions.

Scores generated image captions against reference captions and source images using the Polos metric, which is trained to align with human judgments and probes hallucination robustness and open-vocabulary evaluation.

Evaluates NVIDIA Cosmos Policy on LIBERO and RoboCasa simulation environments. Use when setting up cosmos-policy for robot manipulation evaluation, running headless GPU evaluations with EGL rendering, or profiling inference latency on cluster or local GPU machines.

Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.

Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.

Formal evaluation framework for Codex sessions implementing eval-driven development (EDD) principles

Create a new AgentHub collaboration session with task, agent count, and evaluation criteria.

Provides production-grade techniques for evaluating LLM outputs using LLMs as judges, covering direct scoring, pairwise comparison, bias mitigation, rubric generation, and confidence calibration.

Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles

Use when designing or auditing experiments for an ACL paper, covering tuned LLM baselines, multi-dataset and multilingual evaluation, statistical significance and variance, human evaluation with agreement reporting, contamination and prompt-sensitivity controls, ablations, and error-analysis expectations in NLP reviewing.

Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when: langfuse, llm observability, llm tracing, prompt management, llm evaluation.

Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when: langfuse, llm observability, llm tracing, prompt management, llm evaluation.

Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when: langfuse, llm observability, llm tracing, prompt management, llm evaluation.

Design structured evaluation rubrics for scoring LLM and agent outputs — defining quality dimensions, scoring scales, hard gates, score descriptions, and edge cases. Load when the user asks to create an eval rubric, define evaluation criteria, design scoring dimensions, write an eval spec, or says "what should I evaluate", "design a rubric", "create eval criteria", "define quality dimensions", "evaluation rubric for", "how do I measure quality of". Sub-skill of eval-output orchestrator.

编写评估FastText文本分类模型的Python函数，计算accuracy、F1、recall和precision指标，并处理特定格式的标签文本分割。

DSPy declarative framework for automatic prompt optimization treating prompts as code with systematic evaluation and compilers

Screens resumes with skill matching, experience evaluation, and bias-free candidate ranking

Design reproducible evaluations for AI agents with representative task sets, explicit rubrics, appropriate graders, baselines, regression gates, and failure analysis. Use when defining agent quality, comparing prompts or models, validating a release, measuring tool-use reliability, investigating regressions, or deciding whether an agent is ready for production.

Explain and apply core ML/AI concepts — model types, training pipelines, evaluation metrics, and neural architectures.

Machine learning in Python with scikit-learn. Use for classification, regression, clustering, model evaluation, and ML pipelines.

MCP (Model Context Protocol) server build and evaluation guide, including local conventions for tool surfaces, config, and testing

Internal skill. Called by setup-evaluation after a PASS. Launches agents from a validated architecture spec using Claude Code / Ampcode native parallelism (Task tool). Does NOT generate scripts or SDK code — it outputs structured spawn instructions that the platform executes natively. Never invoked directly by the user. Never launches without a setup-evaluation PASS.

Machine learning development agent for end-to-end ML workflows: data preprocessing, model training, evaluation, hyperparameter tuning, and deployment

Trace, evaluate, and monitor LLM applications with an open-source observability platform.

Debug, evaluate, and monitor LLM applications with tracing, datasets, and built-in evaluators.

For analysis/task/review agents using a Professional Skill on models, RAG, agents, evaluation, or safety; not for work without AI decision impact.

Use when auditing Codex skills and commands for quality. Supports Quick Scan (changed skills only) and Full Stocktake modes with sequential subagent batch evaluation.

Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debug...