# Google Adk Eval

> ADK evaluation framework. Use when writing eval sets, running agent evaluations with `adk eval`, or setting up automated quality checks for agent responses.

- Skill: `eagleisbatman/google-adk-eval` (Agent Skill)
- Install (CLI): `npx skillmds@latest add eagleisbatman/google-adk-eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/eagleisbatman/google-adk-eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: eagleisbatman (https://skillmd.com/u/eagleisbatman)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/eagleisbatman/google-adk-eval

---


# Google ADK — Evaluation

## CLI Command

```bash
adk eval <agent_module_path> <eval_set_path>
```

## Eval Set Format (JSON)

```json
[
  {
    "name": "weather_query_success",
    "data": [
      {
        "query": "What's the weather in New York?",
        "expected_tool_use": [
          {
            "tool_name": "get_weather",
            "tool_input": {"city": "New York"}
          }
        ],
        "reference": "The weather in New York is sunny with a temperature of 25 degrees Celsius."
      }
    ]
  },
  {
    "name": "unknown_city_handling",
    "data": [
      {
        "query": "What's the weather in Atlantis?",
        "expected_tool_use": [
          {
            "tool_name": "get_weather",
            "tool_input": {"city": "Atlantis"}
          }
        ],
        "reference": "Weather information for 'Atlantis' is not available."
      }
    ]
  }
]
```

## Eval Set Fields

| Field | Type | Description |
|-------|------|-------------|
| `name` | str | Test case name |
| `data` | array | List of conversation turns |
| `data[].query` | str | User input message |
| `data[].expected_tool_use` | array | Expected tool calls |
| `data[].reference` | str | Expected/reference response |

## Multi-Turn Eval

```json
[
  {
    "name": "multi_turn_conversation",
    "data": [
      {
        "query": "My name is Alice",
        "reference": "Hello Alice"
      },
      {
        "query": "What's my name?",
        "reference": "Alice"
      }
    ]
  }
]
```

## Running Evaluations

```bash
# Basic eval
adk eval my_agent/ my_agent/eval_set.json

# With specific model for evaluation
adk eval my_agent/ my_agent/eval_set.json --eval_model gemini-2.5-pro

# Output results to file
adk eval my_agent/ my_agent/eval_set.json --output results.json
```

## File Naming Convention

```
my_agent/
├── __init__.py
├── agent.py
├── my_agent_eval_set_001.evalset.json    # Convention: *_eval_set_*.evalset.json
└── my_agent_eval_set_002.evalset.json
```

## Evaluation Criteria

The eval framework checks:

1. **Tool Use**: Did the agent call the expected tools with expected inputs?
2. **Response Quality**: Does the response match the reference (semantic similarity)?
3. **Conversation Flow**: In multi-turn, does the agent maintain context?

## Expected Tool Use Schema

```json
{
  "expected_tool_use": [
    {
      "tool_name": "search_database",
      "tool_input": {
        "query": "user preferences",
        "limit": 10
      }
    }
  ]
}
```

## Writing Good Eval Sets

1. **Cover happy paths**: Normal successful interactions
2. **Cover edge cases**: Invalid inputs, empty results, errors
3. **Cover multi-turn**: Context retention, state management
4. **Be specific in references**: Clear expected outcomes for scoring
5. **Test tool selection**: Verify the agent picks the right tool

## Programmatic Evaluation

```python
from google.adk.evaluation.eval_set import EvalSet

# Load eval set from file
eval_set = EvalSet.model_validate_json(open("eval_set.json").read())

# Programmatic evaluation is primarily done via the CLI:
# adk eval my_agent/ eval_set.json
# For custom evaluation logic, use the Runner directly with test inputs.
```

## Key Rules

- Eval sets are JSON arrays of test cases
- Each test case has a `name` and `data` (list of turns)
- `reference` is the ground truth for scoring
- `expected_tool_use` validates the agent calls the right tools
- Use `adk eval` CLI for quick evaluation runs
- Multi-turn evals test conversation continuity
- Keep eval sets alongside agent code for co-evolution

## Related Skills

- `google-adk-scaffold` — Project structure (eval file placement)
- `google-adk-deploy` — Production deployment after passing evals