Results for “pass-at-k”

Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality.

Use `analysis-agent` to map acceptance to validation, `task-agent` to add or run bounded tests, and `review-agent` to assess proof coverage. Skip work with no material change or already-fresh complete validation.

Canonical kip knowledge-store patterns for any process or agent: recall prior facts before work, assert decisions/gate outcomes/rejections as structured facts after work, resolve entities with explicit --model sonnet, and invoke the CLI Windows-safely via node packages/kip-sdk/dist/cli/kip.js when kip is not on PATH.

Orchestrates multi-agent workflows defined in YAML piece files, delegating tasks to team members and managing state transitions.

Append structured execution traces across operational, cognitive, and contextual surfaces with minimal overhead. Load when inspecting agent runs, logging tool calls and observations, enabling post-run debugging, or pairing with structured-planning step IDs. Also triggers on "trace this run", "log execution", "agent observability", "run log", or when fault-localize needs evidence. Default-on during multi-step plans. Traces live at .agent-loom/traces/ — git-ignored by default.

Run an agent loop until an exit condition is met. Use when the user says "loop", "babysit", "keep trying until", "check every X", "watch", or wants iterative autonomous execution.

Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles.