Results for “evaluation”

Create new skills, modify existing ones, and measure their performance through iterative evaluation and benchmarking.

Creates, runs, and analyzes Arize experiments for evaluating and comparing model performance using the ax CLI.

Manage Azure AI Foundry projects, connections, datasets, indexes, and evaluations using the Java SDK.

Build automated evaluation pipelines for Python LLM applications using real LLM calls and structured test datasets.

Instruments LLM applications with Langfuse for tracing, observability, and evaluation, covering setup, OpenAI and LangChain integrations, and best practices.

Instrument LLM applications with Langfuse for tracing, prompt management, evaluation, and cost tracking, including integrations with OpenAI, LangChain, and LlamaIndex.

Create, run, and analyze evaluation suites for Microsoft 365 Copilot declarative agents using the @microsoft/m365-copilot-eval CLI.

Evaluate NVIDIA Cosmos Policy on LIBERO and RoboCasa simulation environments with headless GPU evaluation and inference profiling.

Rates an agent's own output on five axes — accuracy, completeness, clarity, actionability, conciseness — producing a structured scorecard with evidence and improvement suggestions.

Implement iterative evaluation and refinement loops for AI agent outputs, using self-critique, evaluator-optimizer patterns, and rubric-based scoring to improve quality.

Provides structured guidance for building production-grade AI agent systems through context engineering, covering fundamentals, architectural patterns, operational excellence, and evaluation.

Build AI applications on Microsoft Foundry using the Azure AI Projects Python SDK, including agent creation, tool integration, thread management, and evaluation.

Creates and manages AI agent teams for complex engineering tasks, with model routing, cost optimization, and performance evaluation.

Evaluates execution transcripts and output files against a list of expectations, assigning pass/fail verdicts with cited evidence and critiquing the assertions themselves.

Compare coding agents head-to-head on reproducible tasks with pass rate, cost, time, and consistency metrics.

Instrument LLM applications with Langfuse to trace, score, and monitor cost, quality, and latency across OpenAI and LangChain integrations.

Spawns multiple parallel AI agents that compete on the same task using isolated git worktrees, evaluates results, and merges the best solution.

Build AI applications using the Azure AI Projects SDK for TypeScript, managing agents, connections, deployments, datasets, indexes, and evaluations.

Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles.

Designs autonomous agent harnesses with locked evaluators, editable surfaces, durable logging, novelty gates, pruning, rollback, and human approval boundaries.

Manage Azure AI Foundry projects with .NET SDK: create and run agents, manage connections, datasets, deployments, evaluations, and indexes.

Design multi-agent system architectures, generate tool schemas for Anthropic and OpenAI formats, and evaluate execution logs for cost, latency, and failure bottlenecks.

Turn model work into a production ML system with data contracts, repeatable training, measurable quality gates, deployable artifacts, and operational monitoring.

Deploy, evaluate, fine-tune, and manage Microsoft Foundry agents end-to-end using Azure Developer CLI and MCP tools.

Measures whether coding agents actually follow skills, rules, or agent definitions by generating test scenarios, running agents, and classifying tool calls to report compliance rates.

Tracks bidirectional confidence scores across five domains (TECH, OPS, JUDGMENT, COMMS, ORCH) for agents and users, with triggers, anti-patterns, and pool scoring per runtime.

Guides the creation of high-quality MCP servers that enable LLMs to interact with external services through well-designed tools, covering planning, implementation, testing, and evaluation.

Runs an autonomous optimization loop that iteratively improves an artifact against an objective and evaluator using Hypothesis Tree Refinement, with subagent executors in isolated git worktrees.

Guides the creation of high-quality MCP servers that enable LLMs to interact with external services through well-designed tools, covering planning, implementation, testing, and evaluation across multiple programming languages.

Provides a formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles to define pass/fail criteria, measure reliability with pass@k metrics, and create regression test suites.