Results for “agent-evaluation”

Guides the creation, iterative improvement, and evaluation of agent skills, including drafting, testing, benchmarking, and optimizing descriptions.

Build evaluation frameworks for agent systems with deterministic checks, regression suites, multi-dimensional rubrics, quality gates, production monitoring, and outcome measurement.

Implement iterative evaluation and refinement loops for AI agent outputs, using self-critique, evaluator-optimizer patterns, and rubric-based scoring to improve quality.

Provides structured guidance for building production-grade AI agent systems through context engineering, covering fundamentals, architectural patterns, operational excellence, and evaluation.

Compare coding agents head-to-head on reproducible tasks with pass rate, cost, time, and consistency metrics.

Measures whether coding agents actually follow skills, rules, or agent definitions by generating test scenarios, running agents, and classifying tool calls to report compliance rates.

Compares coding agents head-to-head on reproducible tasks, measuring pass rate, cost, time, and consistency.

Evaluates execution transcripts and output files against a list of expectations, assigning pass/fail verdicts with cited evidence and critiquing the assertions themselves.

Spawns multiple parallel AI agents that compete on the same task using isolated git worktrees, evaluates results, and merges the best solution.

Creates and manages AI agent teams for complex engineering tasks, with model routing, cost optimization, and performance evaluation.

Build AI applications on Microsoft Foundry using the Azure AI Projects Python SDK, including agent creation, tool integration, thread management, and evaluation.

Design multi-agent system architectures, generate tool schemas for Anthropic and OpenAI formats, and evaluate execution logs for cost, latency, and failure bottlenecks.

Designs autonomous agent harnesses with locked evaluators, editable surfaces, durable logging, novelty gates, pruning, rollback, and human approval boundaries.

Manage Azure AI Foundry projects with .NET SDK: create and run agents, manage connections, datasets, deployments, evaluations, and indexes.

Deploy, evaluate, fine-tune, and manage Microsoft Foundry agents end-to-end using Azure Developer CLI and MCP tools.

Tracks bidirectional confidence scores across five domains (TECH, OPS, JUDGMENT, COMMS, ORCH) for agents and users, with triggers, anti-patterns, and pool scoring per runtime.

Build AI applications using the Azure AI Projects SDK for TypeScript, managing agents, connections, deployments, datasets, indexes, and evaluations.

Provides a formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles to define pass/fail criteria, measure reliability with pass@k metrics, and create regression test suites.