# Eval

> Generate a complete Langfuse evaluation pipeline: auto-detect app type, create datasets, LLM-as-judge evaluators, scoring configs, and CI integration.

- Skill: `iabhisekbosepm/eval` (Agent Skill)
- Install (CLI): `npx skillmds@latest add iabhisekbosepm/eval`
- Raw SKILL.md: https://api.skillmd.com/api/skills/iabhisekbosepm/eval/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: iabhisekbosepm (https://skillmd.com/u/iabhisekbosepm)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/iabhisekbosepm/eval

---


# /eval — Langfuse Evaluation Pipeline Generator

Generate evaluation setup for: **$ARGUMENTS**

1. **Discover** — `agents/eval-architect.md` Step 1: Explore codebase, detect LLM app type (RAG, chatbot, agent, code-gen, summarization, classification, extraction), identify prompts, traces, and data flows
2. **Design** — `agents/eval-architect.md` Steps 2-3: Select 4-8 eval dimensions from the dimension matrix, generate seed dataset with 10-20 items covering happy path, edge cases, and adversarial inputs
3. **Generate** — `agents/eval-architect.md` Steps 4-5: Write complete `evals/` directory: config.yaml, evaluators (LLM judge + heuristic), runner.py, ci_eval.py, .env.example
4. **Setup** — `agents/eval-architect.md` Step 6: If `LANGFUSE_SECRET_KEY` is set, run setup_langfuse.py to create dataset and score configs in Langfuse. If not, skip gracefully with instructions.
5. **Document** — `agents/eval-architect.md` Step 7: Generate PM/SME-friendly README with quick start, architecture, and troubleshooting
6. **Verify** — Run verification checklist: correct app type? dimensions appropriate? dataset validates against schema? scripts importable? README clear?

Rules: Always include at least one heuristic evaluator. Never hardcode API keys. README must be understandable by non-developers.