# Evaluate agent and model workflows with EvalScope

> Run repeatable EvalScope benchmark suites for LLM, VLM, RAG, and agent workflows, then inspect traces and reports before changing models or prompts.

- Skill: `agentskillexchange/evaluate-agent-and-model-workflows-with-evalscope` (Agent Skill)
- Install (CLI): `npx skillmds@latest add agentskillexchange/evaluate-agent-and-model-workflows-with-evalscope`
- Raw SKILL.md: https://api.skillmd.com/api/skills/agentskillexchange/evaluate-agent-and-model-workflows-with-evalscope/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: agentskillexchange (https://skillmd.com/u/agentskillexchange)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/agentskillexchange/evaluate-agent-and-model-workflows-with-evalscope

---


# Evaluate agent and model workflows with EvalScope

Run repeatable EvalScope benchmark suites for LLM, VLM, RAG, and agent workflows, then inspect traces and reports before changing models or prompts.

## Prerequisites

Python 3.10+, pip, evalscope package, model endpoint or local model backend, API credentials when evaluating hosted models, selected benchmark datasets

## Installation

Use the upstream install or setup path that matches your environment:
- pip install evalscope
- pip install 'evalscope[service]'

Requirements and caveats from upstream:
- <img src="https://img.shields.io/badge/python-%E2%89%A53.10-5be.svg">
- **🤖 Agent Evaluation Mode**: Drives benchmarks (e.g. GSM8K, AIME, SWE-bench Agentic) inside a controlled multi-turn AgentLoop with pluggable strategies, tools and Docker sandbox; full per-sample Agent Trace is recorde...
- 🔥 **[2026.05.26]** Added the [GAIA](https://evalscope.readthedocs.io/en/latest/third_party/gaia.html) agent benchmark (multi-turn ReAct + bash in a Docker sandbox, official rule-based scorer) and generic [MCP server](...

Basic usage or getting-started notes:
- 🔥 **[2026.05.19]** Added support for [SWE-bench_Pro](https://evalscope.readthedocs.io/en/latest/third_party/swe_bench_pro.html) and [τ³-bench](https://evalscope.readthedocs.io/en/latest/third_party/tau3_bench.html): S...
- 🔥 **[2026.05.15]** Introduced **Agent Evaluation Mode**: any benchmark based on DefaultDataAdapter (GSM8K, AIME, IFEval, etc.) can now be driven through a multi-turn AgentLoop with pluggable strategies (function_calli...
- 🔥 **[2026.05.08]** Partnered with [LightSeek](https://lightseek.org/) to launch [TokenSpeed](https://lightseek.org/blog/lightseek-tokenspeed.html), a speed-of-light LLM inference engine for agentic workloads. EvalScop...

- Source: https://github.com/modelscope/evalscope
- Extracted from upstream docs: https://raw.githubusercontent.com/modelscope/evalscope/HEAD/README.md

## Documentation

- https://evalscope.readthedocs.io/en/latest/

## Source

- [Agent Skill Exchange](https://agentskillexchange.com/skills/evaluate-agent-and-model-workflows-with-evalscope/)