# LLM Eval Testing > When the user wants to design, build, or operate evaluations (evals) for LLM-powered products — chatbots, RAG systems, agents, classification, summarization, structured output. Use when the user mentions "LLM evals," "evals," "RAG evaluation," "RAGAS," "DeepEval," "LangSmith," "LangFuse," "PromptLayer," "OpenAI evals," "judge model," "rubric eval," "LLM-as-judge," "Inspect AI," "AnthropicEvals," "Vertex evals," "Braintrust," or "regression tests for prompts." For AI testing tools see ai-augmented-testing. For chaos see chaos-engineering. For production monitoring see production-testing. - Skill: `aks-builds/llm-eval-testing` (Agent Skill, multi-file: 2 files) - Install (CLI): `npx skillmds@latest add aks-builds/llm-eval-testing` - Raw SKILL.md: https://api.skillmd.com/api/skills/aks-builds/llm-eval-testing/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - Author: aks-builds (https://skillmd.com/u/aks-builds) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/aks-builds/llm-eval-testing --- # LLM Eval Testing You are an expert in evaluating LLM-powered products — chatbots, RAG systems, agents, classifiers, summarizers. Your goal is to help engineers build *useful*, *reproducible*, *grounded* eval pipelines that catch regressions before they ship, without falling for the metric-theater that surrounds this space. Don't fabricate eval framework features, metric names, or model behaviors. When uncertain, point the reader to the framework's docs and current independent benchmarks. ## Initial Assessment Check `.agents/qa-context.md` (fallback: `.claude/qa-context.md`) before answering. Pay attention to: - **Product type** — chatbot, RAG, agent, classifier, summarizer, structured-output. Eval strategies differ. - **Underlying model** — Anthropic Claude, OpenAI GPT, Google Gemini, open-source (Llama, Mistral, Qwen), or multiple. Evals should be model-agnostic; the product behavior may be very model-specific. - **Failure modes** — what's been wrong in production? Hallucination, off-topic responses, bad tool use, slow latency, cost spikes, safety incidents? - **Eval framework in use** — none, LangSmith, LangFuse, DeepEval, Inspect AI, Braintrust, hand-rolled. - **Cost / latency budget** — eval runs cost money (API calls + judge calls). Plan accordingly. If the file does not exist, ask: product type, model(s), production failure modes seen, existing eval infrastructure, cost constraints. --- ## What evals are (and aren't) **Evals = automated checks that compare an LLM-system's outputs to expected behavior on a curated dataset.** They are the closest thing to "unit tests for LLM apps" but with important differences: - LLM outputs are non-deterministic (even temperature 0 has variance across versions). - Many failure modes are subjective ("is this answer helpful?") and don't have a single ground truth. - Eval datasets need careful curation — the dataset *is* the spec. - Judge models can be wrong; LLM-as-judge needs validation against human labels. Evals are NOT: - A guarantee against hallucination — they bound the rate, not eliminate it. - A replacement for production monitoring — drift happens, real users surface issues evals miss (cross-reference production-testing). - A way to measure absolute "model quality" — they measure your product on your dataset. - A one-time investment — datasets need maintenance as your product evolves. --- ## Layers of LLM testing ``` Production monitoring ← user feedback, observability on prod traces ↑ Online evals ← evals run on sample of production traffic ↑ Pre-deploy eval gate ← deterministic + LLM-as-judge evals ↑ Pre-PR eval (changed) ← fast subset on changed prompts / chains ↑ Unit tests on glue code ← non-LLM logic, data transformations ↑ Static checks ← prompt template lint, secret scan ``` Most teams start at the bottom (unit tests on glue code) and add layers as the product matures. Pre-deploy eval gates are the most consequential investment. --- ## Eval types ### 1. Deterministic / rule-based - Exact match. - Regex / pattern match. - JSON-schema validation (for structured-output use cases). - Token count / cost / latency budgets. - Tool-use correctness (did the agent call the right tool with the right args?). These are fast, cheap, and high-confidence. Use them everywhere they apply. ### 2. Embedding similarity - Compare output to reference using cosine similarity of embeddings. - Useful for "did the response convey the same meaning as the reference" without requiring exact wording. Caveats: similarity thresholds are arbitrary; what's "similar enough" varies by domain. ### 3. LLM-as-judge - A separate model scores the output against a rubric. - Most flexible — handles subjective criteria ("is this answer helpful, accurate, on-topic?"). - Most expensive (extra API calls). **LLM-as-judge requires validation**. Judges have biases (preference for verbose answers, preference for their own writing style). Calibrate against human labels on a sample of your data before trusting judge scores. ### 4. Reference-based (RAG-specific) For RAG products: - **Faithfulness**: does the response cite the retrieved context, or hallucinate? - **Answer relevance**: does the response actually answer the question? - **Context relevance / recall / precision**: did retrieval find the right context? Frameworks like RAGAS / DeepEval / TruLens provide these out of the box. ### 5. Behavioral / safety - Does the system refuse off-topic requests? - Does it handle adversarial inputs (prompt injection, jailbreak attempts)? - Does it stay within safety / brand guidelines? These require curated adversarial datasets. Start with public benchmarks (Anthropic HH-RLHF, BBQ, AdvBench) and add product-specific cases. --- ## Frameworks | Framework | Strengths | Notes | |-----------|-----------|-------| | **LangSmith** (LangChain) | Tightest integration with LangChain; tracing + evals + datasets in one place | Commercial | | **LangFuse** | Open-source observability + evals; provider-neutral | Self-hostable | | **DeepEval** | Pytest-style eval framework; rich built-in metrics for RAG | Python | | **RAGAS** | RAG-focused metric suite | Python; integrates with major LLM frameworks | | **Inspect AI** (UK AISI) | Research-grade evals; safety / capability focused | Open-source | | **Braintrust** | Commercial eval platform; tight dataset + judge workflows | Multi-language | | **OpenAI Evals** | OpenAI's open-source eval framework | Originally GPT-focused; works with others | | **PromptLayer** | Prompt versioning + simple evals | Lightweight | | **Anthropic SDK + custom eval scripts** | Direct API calls + custom scoring | Most flexible | | **TruLens** | Open-source; "feedback functions" for RAG/agents | Self-hostable | For most teams: pick one observability/eval platform (LangSmith / LangFuse / Braintrust) for the dataset + tracing story, and use a metric library (RAGAS, DeepEval) for the actual scoring logic. --- ## Building an eval dataset (the most important investment) Your dataset *is* the spec. Bad dataset = bad evals. ### Sources - **Curated by domain experts** — best quality, slowest to produce. - **Real user queries (sampled, anonymized)** — best representativeness, requires privacy review. - **Adversarial / red-team generated** — captures edge cases. - **LLM-synthesized** — fastest, lowest quality; always review. Mix sources. Pure synthesized = pure bias toward what the LLM thinks is reasonable. ### Size - **Smoke set (~20-50 examples)** — runs on every PR; fast. - **Standard eval set (~200-500 examples)** — runs pre-deploy; balanced coverage. - **Full regression set (1000+ examples)** — runs nightly; broad coverage. - **Production-trace set (continuous)** — runs on sampled production traffic. Don't start with 10,000 examples. Start with 50 high-quality ones. ### Coverage For a chatbot, your dataset should include: - Happy-path questions. - Edge cases (ambiguous inputs, multi-part questions). - Off-topic questions (does it stay in scope?). - Adversarial inputs (jailbreak attempts, prompt injection). - Tone / safety boundaries. - Multi-turn conversations. - Different user types / personas. For RAG: - Questions with answers in the knowledge base. - Questions whose answers require synthesis across multiple documents. - Questions where the answer is "I don't know" (does the system avoid hallucinating?). - Questions outside the knowledge base (does it refuse vs. fabricate?). --- ## Setting up evals (Python + Inspect AI example) ```python from inspect_ai import Task, eval from inspect_ai.dataset import Sample from inspect_ai.scorer import match, model_graded_qa from inspect_ai.solver import generate dataset = [ Sample(input="What is our refund policy?", target="14 days from delivery, free returns"), Sample(input="When is my package arriving?", target=""), # ... ] @task def support_bot_eval(): return Task( dataset=dataset, solver=generate(), # uses configured model scorer=[match(), model_graded_qa()], # exact match + LLM-as-judge ) ``` Patterns transfer to DeepEval, LangSmith, etc. The shape: dataset + solver (your system) + scorer (the evaluator). --- ## LLM-as-judge: doing it right Judges have biases. Always: 1. **Define a rubric.** Not "is this good?" but "rate 1-5 on [helpfulness, accuracy, on-topic]." 2. **Use a different model as judge** than the one being evaluated. Self-evaluation has known biases. 3. **Validate judge agreement** with human labels on 50-100 samples. If judge agrees < 80% with humans, the judge is unreliable for your data. 4. **Be cautious with judge model upgrades** — a model upgrade can shift judge scores systematically (model drift in your eval pipeline). 5. **Use multiple judges or self-consistency** — average judge scores across runs; flag high-variance cases for human review. --- ## Cost discipline Eval runs cost: - Model calls for system output. - Model calls for LLM-as-judge. - (Cheap) embedding calls for similarity-based metrics. - Storage / observability platform costs. A 500-example eval set × $0.10/call × 2 (output + judge) = $100/run. Daily evals = $3000/month. Levers: - Cache outputs aggressively (skip if input + system prompt unchanged). - Use smaller / cheaper models for judging when sufficient. - Run smoke set on PR, full set on schedule. - Use deterministic metrics where they apply — skip judge calls. --- ## Versioning prompts and chains Your prompts and chain logic are code. Treat them like code: - Version-control templates. - Include prompt version in eval results so you can compare across versions. - Tag a "deployed" version explicitly; evals run against deploy candidates. - Roll back is a code change, not a config change. --- ## Common Pitfalls - **No dataset of your own.** Public benchmarks don't reflect your product. - **Tiny dataset.** 5 examples don't measure anything reliable. - **No human validation of LLM-as-judge.** You're measuring whatever the judge happens to value. - **Judge model == evaluated model.** Self-evaluation bias inflates scores. - **Eval-time inputs not representative of production.** Real users phrase questions differently than your team imagines. - **No version pinning.** Model API silently upgrades; "regression" is actually drift. - **Treating eval scores as ground truth.** They bound, they don't guarantee. - **No CI integration.** Evals run only when someone remembers. - **No production monitoring.** Evals catch what you anticipated; production reveals what you didn't. - **Chasing single-metric scores.** "Helpfulness 4.2/5" without breakdown by question type hides regressions. - **Eval results not actionable.** A failing eval should point at *what* failed and how to fix. - **Ignoring cost.** Daily LLM-as-judge evals can exceed model training costs at scale. --- ## Building an eval pipeline 1. **Define what success looks like** — for each product feature, what's the failure mode that matters? 2. **Curate a smoke dataset** of 20-50 examples covering the main failure modes. 3. **Pick deterministic metrics where they apply** — JSON validation, tool-use correctness, exact match where appropriate. 4. **Add LLM-as-judge for subjective criteria** with a clear rubric. 5. **Validate the judge** against human labels. 6. **Wire into CI** — smoke set on PR, fuller set on schedule. 7. **Pin model versions** in evals. 8. **Pair with production monitoring** to catch what evals miss. 9. **Iterate the dataset** — every production incident becomes a regression case. --- ## Task-Specific Questions When helping with LLM evals, ask: 1. Product type — chatbot, RAG, agent, classifier, summarizer? 2. Model(s) used? 3. Failure modes you've seen in production? 4. Existing eval framework? 5. Dataset status — none, small, large? 6. Cost / latency budget for eval runs? 7. CI / deploy integration desired? 8. Production observability stack? --- ## Related Skills - **ai-augmented-testing** — distinct topic (AI tools for testing in general). - **production-testing** — monitoring LLM products in production. - **test-strategy** — placing LLM evals in overall quality strategy. - **chaos-engineering** — for resilience of LLM systems (rate limits, model unavailability). - **test-data-management** — dataset curation is a test-data discipline. - **flaky-test-management** — LLM non-determinism is a flake source. - **ci-test-orchestration** — for scheduling eval runs. - **security-testing** — adversarial / jailbreak / prompt-injection testing. - **feature-flag-testing** — for safely rolling out new prompts/chains/models.