# Eval Harness

> Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles

- Skill: `yevhens-hue/eval-harness` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add yevhens-hue/eval-harness`
- Raw SKILL.md: https://api.skillmd.com/api/skills/yevhens-hue/eval-harness/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: yevhens-hue (https://skillmd.com/u/yevhens-hue)
- Updated: 2026-09-21
- Page: https://skillmd.com/skills/yevhens-hue/eval-harness

---


# 🎯 Formal Eval Harness & Benchmarking Skill

This skill defines the implementation of dedicated Evaluation Harness CLI tooling (`npm run eval` / `python eval.py`) for running automated benchmarks on AI agents and prompt updates.

---

## 🛠️ Eval Harness Architecture

```mermaid
flowchart LR
    A[Benchmark Test Dataset] --> B[Eval Harness CLI Engine]
    B -->|Run Model Batch| C[Model Outputs & Telemetry]
    C --> D[Assertion Metrics Evaluator]
    D --> E[Benchmark Report JSON / Terminal Output]
```

### 1. Benchmark Dataset Standard (`eval_dataset.json`)
Maintain a version-controlled benchmark dataset containing test inputs, expected invariants, and source citation keys.

### 2. Metric Scoring Assertions
- **Latency Assertions:** `latency_ms <= 2000`
- **Factual Match Score:** `similarity_score >= 0.85`
- **Hallucination Rate:** `hallucination_count === 0`
- **Cost Budget Assertions:** `cost_per_query_usd <= 0.005`

---

## 💻 Sample Eval Harness Command

```bash
npm run eval -- --dataset=eval_dataset.json --threshold=0.9
```