# Deepeval Testing

> DeepEval pytest-style LLM testing patterns with built-in metrics, custom evaluators, and CI integration. Use when creating LLM tests, evaluating RAG quality, or measuring faithfulness/relevance.

- Skill: `vanman2024/deepeval-testing` (Agent Skill, multi-file: 5 files)
- Install (CLI): `npx skillmds@latest add vanman2024/deepeval-testing`
- Raw SKILL.md: https://api.skillmd.com/api/skills/vanman2024/deepeval-testing/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: vanman2024 (https://skillmd.com/u/vanman2024)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/vanman2024/deepeval-testing

---


# DeepEval Testing

Skill for pytest-style LLM evaluation with DeepEval.

## Overview

DeepEval provides:

- pytest-compatible LLM tests
- Built-in metrics (faithfulness, relevance, toxicity)
- Custom metric creation
- Async test execution

## Use When

This skill is automatically invoked when:

- Creating pytest-style LLM tests
- Evaluating RAG quality
- Measuring faithfulness/relevance
- Building custom metrics

## Available Scripts

| Script                     | Description                 |
| -------------------------- | --------------------------- |
| `scripts/init-deepeval.sh` | Initialize DeepEval project |
| `scripts/run-tests.sh`     | Run DeepEval tests          |

## Available Templates

| Template                  | Description          |
| ------------------------- | -------------------- |
| `templates/conftest.py`   | pytest configuration |
| `templates/test_basic.py` | Basic test structure |
| `templates/test_rag.py`   | RAG evaluation tests |

## Built-in Metrics

- `AnswerRelevancyMetric`
- `FaithfulnessMetric`
- `ContextualRelevancyMetric`
- `HallucinationMetric`
- `ToxicityMetric`
- `BiasMetric`