How do I install the Analyze Evals skill?

Run npx skillmds@latest add microsoft/analyze-evals in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Analyze Evals safe to use?

This skill has not completed SkillMD's automated safety review yet. Independent scanners report: SkillSpector: PASS, Skill Scanner: PASS. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Analyze Evals?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Analyze Evals free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Analyze Evals?

Microsoft (@microsoft) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Analyze Evals by microsoft — Agent Skill

Name: Analyze Evals
Author: Microsoft

Analyze Evals

Analyze exported evaluation results from Copilot Studio's Evaluate tab. The user provides a CSV file exported from the Copilot Studio UI; this skill parses it, identifies failures, and proposes YAML fixes. No API access or published agent required — just the exported CSV.

Microsoft Updated Sep 21, 2026 2.7k repo stars

File contents

Analyze Copilot Studio Evaluation Results

Analyze evaluation results exported from the Copilot Studio UI as CSV.

Phase 1: Get Results

Ask the user for the CSV file path if not already provided. The file is typically exported from Copilot Studio's Evaluate tab and named Evaluate <agent name> <date>.csv in their Downloads folder.

Read the CSV file. The in-product evaluation CSV has these columns:

Column	Meaning
`question`	The test utterance
`expectedResponse`	Expected response (may be empty)
`actualResponse`	What the agent responded
`testMethodType_1`	Eval method (e.g., `GeneralQuality`)
`result_1`	`Pass` or `Fail`
`passingScore_1`	Score threshold (may be empty)
`explanation_1`	Why it passed/failed (e.g., "Seems relevant; Seems incomplete; Knowledge sources not cited")

The _1 suffix indicates the first eval method. There may be additional methods (_2, _3, etc.) with the same column pattern.

Phase 2: Analyze Results

Focus on failed evaluations (result_1 = Fail, or any result_N = Fail).
For each failure, use the explanation column to understand the issue:
- "Question not answered" — The agent couldn't handle the question. Check if there's a matching topic or knowledge source.
- "Knowledge sources not cited" — The agent responded but didn't cite sources. Check knowledge source configuration and SearchAndSummarizeContent nodes.
- "Seems incomplete" — The response was partial. Check topic flow for early exits, missing branches, or incomplete SendActivity messages.
- Error messages in actualResponse (e.g., GenAIToolPlannerRateLimitReached) — These are runtime errors, not authoring issues. Flag them to the user as transient failures to retry.

Phase 3: Propose Fixes

For each failure, identify the relevant YAML file(s):
- Auto-discover the agent: Glob: **/agent.mcs.yml
- Find the relevant topic by matching the test utterance against trigger phrases and model descriptions
- Read the topic file to understand the current flow
Propose specific YAML changes to fix each failure. Present them to the user as a summary:
- Which test(s) failed and why
- Which file(s) need changes
- What the proposed change is (show the diff)
Wait for user decision. The user can:
- Accept all — apply all proposed changes
- Accept partially — apply only some changes (ask which ones)
- Reject — discard proposed changes and discuss alternative approaches
Apply accepted changes using the Edit tool. After applying, remind the user to push and publish again before re-running evaluations.

---name: analyze-evalsdescription: Analyze exported evaluation results from Copilot Studio's Evaluate tab. The user provides a CSV file exported from the Copilot Studio UI; this skill parses it, identifies failures, and proposes YAML fixes. No API access or published agent required — just the exported CSV.---# Analyze Copilot Studio Evaluation ResultsAnalyze evaluation results exported from the Copilot Studio UI as CSV.## Phase 1: Get Results1. **Ask the user for the CSV file path** if not already provided. The file is typically exported from Copilot Studio's Evaluate tab and named `Evaluate <agent name> <date>.csv` in their Downloads folder.2. **Read the CSV file**. The in-product evaluation CSV has these columns: | Column | Meaning | |--------|---------| | `question` | The test utterance | | `expectedResponse` | Expected response (may be empty) | | `actualResponse` | What the agent responded | | `testMethodType_1` | Eval method (e.g., `GeneralQuality`) | | `result_1` | `Pass` or `Fail` | | `passingScore_1` | Score threshold (may be empty) | | `explanation_1` | Why it passed/failed (e.g., "Seems relevant; Seems incomplete; Knowledge sources not cited") | The `_1` suffix indicates the first eval method. There may be additional methods (`_2`, `_3`, etc.) with the same column pattern.## Phase 2: Analyze Results1. **Focus on failed evaluations** (`result_1` = `Fail`, or any `result_N` = `Fail`).2. For each failure, use the `explanation` column to understand the issue: - **"Question not answered"** — The agent couldn't handle the question. Check if there's a matching topic or knowledge source. - **"Knowledge sources not cited"** — The agent responded but didn't cite sources. Check knowledge source configuration and `SearchAndSummarizeContent` nodes. - **"Seems incomplete"** — The response was partial. Check topic flow for early exits, missing branches, or incomplete `SendActivity` messages. - Error messages in `actualResponse` (e.g., `GenAIToolPlannerRateLimitReached`) — These are runtime errors, not authoring issues. Flag them to the user as transient failures to retry.## Phase 3: Propose Fixes1. **For each failure, identify the relevant YAML file(s)**: - Auto-discover the agent: `Glob: **/agent.mcs.yml` - Find the relevant topic by matching the test utterance against trigger phrases and model descriptions - Read the topic file to understand the current flow2. **Propose specific YAML changes** to fix each failure. Present them to the user as a summary: - Which test(s) failed and why - Which file(s) need changes - What the proposed change is (show the diff)3. **Wait for user decision**. The user can: - **Accept all** — apply all proposed changes - **Accept partially** — apply only some changes (ask which ones) - **Reject** — discard proposed changes and discuss alternative approaches4. **Apply accepted changes** using the Edit tool. After applying, remind the user to push and publish again before re-running evaluations.

Frequently asked questions