# Evaluations

> Compatibility router for LangWatch evaluation requests. Use only when the user asks for evaluations without making it clear whether they mean pre-deployment experiments or production online evaluations. Routes the request to the focused companion skill and does not implement either workflow itself.

- Skill: `langwatch/evaluations` (Agent Skill)
- Install (CLI): `npx skillmds@latest add langwatch/evaluations`
- Raw SKILL.md: https://api.skillmd.com/api/skills/langwatch/evaluations/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity, AI & ML
- License: MIT
- Author: langwatch (https://skillmd.com/u/langwatch)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/langwatch/evaluations

---


# Route an Evaluation Request

This is a compatibility skill. Do not build an experiment, monitor, or guardrail from this skill.

Classify the user's intent:

| Intent                                                                               | Correct skill        |
| ------------------------------------------------------------------------------------ | -------------------- |
| Batch test a dataset, compare prompts or models, benchmark, create a CI quality gate | `experiments`        |
| Score live traces or threads, monitor production quality, create a guardrail         | `online-evaluations` |

If the request remains ambiguous after inspecting context (a bare "make me an eval" that names neither a dataset nor live traffic), do not create anything yet. This choice picks what gets tested, so it is the user's to make, not a default's. Ask it as a question card and stop; the answer arrives as the next message.

In Langy, ask it with the `question` tool and stop; the options are the concrete alternatives, for example "A dataset, before deployment" and "Live production traffic". Elsewhere, ask the same question as one short line of prose.

A rejected field value is not this kind of choice. If a create later fails with a `validation_error` whose reason names the field and an `expected` list, correct that exact field from the list and retry once. Never turn a fixable slug into a question for the user.

Then hand off:

1. If the correct companion skill is available, load it and follow it instead of continuing here.

2. If `experiments` is missing, tell the user to install it with:

   ```bash
   npx skills@1.5.19 add langwatch/skills/experiments
   ```

3. If `online-evaluations` is missing, tell the user to install it with:

   ```bash
   npx skills@1.5.19 add langwatch/skills/online-evaluations
   ```

Do not recreate the companion skill's instructions from memory. Load the focused skill so its current workflow, safety checks, and verification steps are used.