agentic-in Evaluation
Use this built-in skill when the user wants to compare prompts, models, retrieval setups, or agent behaviors with something stronger than anecdotes.
Core rules
- Define the task, dataset, metric, and baseline before running comparisons.
- Keep eval inputs and scoring rules stable enough to reproduce.
- Separate offline benchmarking from product acceptance criteria.
- Report both quantitative outcomes and obvious failure modes.
Default workflow
- Identify the decision the evaluation should support.
- Build or select the smallest credible dataset and metric set.
- Run the baseline and candidate systems under the same conditions.
- Summarize the tradeoffs, regressions, and confidence level.
Guardrails
- Do not declare wins from cherry-picked examples.
- Do not mix incomparable prompts, models, or retrieval settings in one score line.
- Do not skip failure-case inspection when the average looks good.