Agentv Eval Migrations

AgentV Eval Migrations

Use this skill when updating existing AgentV eval YAML, examples, docs, or generated eval authoring guidance after a schema-breaking change.

Before editing, read references/breaking-changes.md. For stale evals from AgentV v4.42.4, use that reference as the migration map: it lists the v4.42.4-era shape, current shape, migration steps, verification commands, and compatibility notes for each major breaking authoring change. Then compare the eval file against the current portable contract:

• Treat migrations as rule-based rewrites first. Do not rely on a manual "looks current" pass when the old field has a direct current replacement.
• Rewrite authored prompt data into top-level prompts plus tests[].vars; keep input only for documented raw-case import compatibility.
• Move sibling expected_output to vars.expected_output, then add or keep an explicit assertion that consumes it, usually llm-rubric with {{ expected_output }}.
• Rewrite target identity to id, keep provider for the backend/adapter kind, put provider settings under config, and rewrite env references to {{ env.NAME }}.
• Remove use_target, eval_cases, evalcases, providerPromptMap, and provider_prompt_map from authored eval/config YAML. Current AgentV rejects these fields, so do not preserve them as compatibility aliases.
• Keep committed eval YAML portable: prompts, cases, assertions, workspace templates, repos, hooks, env checks, Docker preflight/container config, and workspace.scope.
• Do not put machine-local existing workspace directories in eval YAML. Use --workspace-path for one-off runs or .agentv/config.local.yaml with execution.workspace_path for persistent local binding.
• Use workspace.scope: suite | attempt for portable workspace lifetime. Docker config is not a replacement for workspace folder lifetime.
• Keep wire-format fields in snake_case and TypeScript internals in camelCase.

After migration, validate with bun apps/cli/src/cli.ts validate <file> when working in the AgentV repo, or agentv validate <file> from an installed CLI. Run the repo's parser/schema tests for generated examples and fixtures when the change affects shared skill data or examples.