How do I install the Evaluations skill?

Run npx skillmds@latest add langwatch/evaluations in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Evaluations safe to use?

This skill has not completed SkillMD's automated safety review yet. Capability flags: docs only. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Evaluations?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Evaluations free to use?

Yes. Installing skills from SkillMD is free. This skill is licensed under MIT.

Who published Evaluations?

langwatch (@langwatch) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Evaluations by langwatch — Agent Skill

Name: Evaluations
Rating: 4.2 (5 reviews)
Author: langwatch

Evaluations

Compatibility router for LangWatch evaluation requests. Use only when the user asks for evaluations without making it clear whether they mean pre-deployment experiments or production online evaluations. Routes the request to the focused companion skill and does not implement either workflow itself.

langwatch b34afe2 2.3 KB Updated Sep 10, 2026

File contents

Route an Evaluation Request

This is a compatibility skill. Do not build an experiment, monitor, or guardrail from this skill.

Classify the user's intent:

Intent	Correct skill
Batch test a dataset, compare prompts or models, benchmark, create a CI quality gate	`experiments`
Score live traces or threads, monitor production quality, create a guardrail	`online-evaluations`

If the request remains ambiguous after inspecting context (a bare "make me an eval" that names neither a dataset nor live traffic), do not create anything yet. This choice picks what gets tested, so it is the user's to make, not a default's. Ask it as a question card and stop; the answer arrives as the next message.

In Langy, ask it with the question tool and stop; the options are the concrete alternatives, for example "A dataset, before deployment" and "Live production traffic". Elsewhere, ask the same question as one short line of prose.

A rejected field value is not this kind of choice. If a create later fails with a validation_error whose reason names the field and an expected list, correct that exact field from the list and retry once. Never turn a fixable slug into a question for the user.

Then hand off:

If the correct companion skill is available, load it and follow it instead of continuing here.
If experiments is missing, tell the user to install it with:
```
npx skills@1.5.19 add langwatch/skills/experiments
```
If online-evaluations is missing, tell the user to install it with:
```
npx skills@1.5.19 add langwatch/skills/online-evaluations
```

Do not recreate the companion skill's instructions from memory. Load the focused skill so its current workflow, safety checks, and verification steps are used.

---name: evaluationsdescription: Compatibility router for LangWatch evaluation requests. Use only when the user asks for evaluations without making it clear whether they mean pre-deployment experiments or production online evaluations. Routes the request to the focused companion skill and does not implement either workflow itself.license: MIT---# Route an Evaluation RequestThis is a compatibility skill. Do not build an experiment, monitor, or guardrail from this skill.Classify the user's intent:| Intent | Correct skill || ------------------------------------------------------------------------------------ | -------------------- || Batch test a dataset, compare prompts or models, benchmark, create a CI quality gate | `experiments` || Score live traces or threads, monitor production quality, create a guardrail | `online-evaluations` |If the request remains ambiguous after inspecting context (a bare "make me an eval" that names neither a dataset nor live traffic), do not create anything yet. This choice picks what gets tested, so it is the user's to make, not a default's. Ask it as a question card and stop; the answer arrives as the next message.In Langy, ask it with the `question` tool and stop; the options are the concrete alternatives, for example "A dataset, before deployment" and "Live production traffic". Elsewhere, ask the same question as one short line of prose.A rejected field value is not this kind of choice. If a create later fails with a `validation_error` whose reason names the field and an `expected` list, correct that exact field from the list and retry once. Never turn a fixable slug into a question for the user.Then hand off:1. If the correct companion skill is available, load it and follow it instead of continuing here.2. If `experiments` is missing, tell the user to install it with: ```bash npx skills@1.5.19 add langwatch/skills/experiments ```3. If `online-evaluations` is missing, tell the user to install it with: ```bash npx skills@1.5.19 add langwatch/skills/online-evaluations ```Do not recreate the companion skill's instructions from memory. Load the focused skill so its current workflow, safety checks, and verification steps are used.

Frequently asked questions