How do I install the Githubcopilot Phoenix Evals skill?

Run npx skillmds@latest add pyfagorass/githubcopilot-phoenix-evals in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

What does the Githubcopilot Phoenix Evals skill do?

Build and run evaluators for AI/LLM applications using Phoenix. It is listed under AI & ML on SkillMD.

Is Githubcopilot Phoenix Evals safe to use?

This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Githubcopilot Phoenix Evals?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Githubcopilot Phoenix Evals free to use?

Yes. Installing skills from SkillMD is free. This skill is licensed under Apache-2.

Who published Githubcopilot Phoenix Evals?

pyfagorass (@pyfagorass) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Githubcopilot Phoenix Evals by pyfagorass

Name: Githubcopilot Phoenix Evals
Author: pyfagorass

File contents

Phoenix Evals

Build evaluators for AI/LLM applications. Code first, LLM for nuance, validate against humans.

Quick Reference

Task	Files
Setup	setup-python, setup-typescript
Decide what to evaluate	evaluators-overview
Choose a judge model	fundamentals-model-selection
Use pre-built evaluators	evaluators-pre-built
Build code evaluator	evaluators-code-python, evaluators-code-typescript
Build LLM evaluator	evaluators-llm-python, evaluators-llm-typescript, evaluators-custom-templates
Batch evaluate DataFrame	evaluate-dataframe-python
Run experiment	experiments-running-python, experiments-running-typescript
Create dataset	experiments-datasets-python, experiments-datasets-typescript
Generate synthetic data	experiments-synthetic-python, experiments-synthetic-typescript
Validate evaluator accuracy	validation, validation-evaluators-python, validation-evaluators-typescript
Sample traces for review	observe-sampling-python, observe-sampling-typescript
Analyze errors	error-analysis, error-analysis-multi-turn, axial-coding
RAG evals	evaluators-rag
Avoid common mistakes	common-mistakes-python, fundamentals-anti-patterns
Production	production-overview, production-guardrails, production-continuous

Workflows

Starting Fresh: observe-tracing-setup → error-analysis → axial-coding → evaluators-overview

Building Evaluator: fundamentals → common-mistakes-python → evaluators-{code|llm}-{python|typescript} → validation-evaluators-{python|typescript}

RAG Systems: evaluators-rag → evaluators-code-* (retrieval) → evaluators-llm-* (faithfulness)

Production: production-overview → production-guardrails → production-continuous

Reference Categories

Prefix	Description
`fundamentals-*`	Types, scores, anti-patterns
`observe-*`	Tracing, sampling
`error-analysis-*`	Finding failures
`axial-coding-*`	Categorizing failures
`evaluators-*`	Code, LLM, RAG evaluators
`experiments-*`	Datasets, running experiments
`validation-*`	Validating evaluator accuracy against human labels
`production-*`	CI/CD, monitoring

Key Principles

Principle	Action
Error analysis first	Can't automate what you haven't observed
Custom > generic	Build from your failures
Code first	Deterministic before LLM
Validate judges	>80% TPR/TNR
Binary > Likert	Pass/fail, not 1-5

---name: githubcopilot-phoenix-evalsdescription: Build and run evaluators for AI/LLM applications using Phoenix.license: Apache-2.0---# Phoenix EvalsBuild evaluators for AI/LLM applications. Code first, LLM for nuance, validate against humans.## Quick Reference| Task | Files || ---- | ----- || Setup | [setup-python](references/setup-python.md), [setup-typescript](references/setup-typescript.md) || Decide what to evaluate | [evaluators-overview](references/evaluators-overview.md) || Choose a judge model | [fundamentals-model-selection](references/fundamentals-model-selection.md) || Use pre-built evaluators | [evaluators-pre-built](references/evaluators-pre-built.md) || Build code evaluator | [evaluators-code-python](references/evaluators-code-python.md), [evaluators-code-typescript](references/evaluators-code-typescript.md) || Build LLM evaluator | [evaluators-llm-python](references/evaluators-llm-python.md), [evaluators-llm-typescript](references/evaluators-llm-typescript.md), [evaluators-custom-templates](references/evaluators-custom-templates.md) || Batch evaluate DataFrame | [evaluate-dataframe-python](references/evaluate-dataframe-python.md) || Run experiment | [experiments-running-python](references/experiments-running-python.md), [experiments-running-typescript](references/experiments-running-typescript.md) || Create dataset | [experiments-datasets-python](references/experiments-datasets-python.md), [experiments-datasets-typescript](references/experiments-datasets-typescript.md) || Generate synthetic data | [experiments-synthetic-python](references/experiments-synthetic-python.md), [experiments-synthetic-typescript](references/experiments-synthetic-typescript.md) || Validate evaluator accuracy | [validation](references/validation.md), [validation-evaluators-python](references/validation-evaluators-python.md), [validation-evaluators-typescript](references/validation-evaluators-typescript.md) || Sample traces for review | [observe-sampling-python](references/observe-sampling-python.md), [observe-sampling-typescript](references/observe-sampling-typescript.md) || Analyze errors | [error-analysis](references/error-analysis.md), [error-analysis-multi-turn](references/error-analysis-multi-turn.md), [axial-coding](references/axial-coding.md) || RAG evals | [evaluators-rag](references/evaluators-rag.md) || Avoid common mistakes | [common-mistakes-python](references/common-mistakes-python.md), [fundamentals-anti-patterns](references/fundamentals-anti-patterns.md) || Production | [production-overview](references/production-overview.md), [production-guardrails](references/production-guardrails.md), [production-continuous](references/production-continuous.md) |## Workflows**Starting Fresh:**[observe-tracing-setup](references/observe-tracing-setup.md) → [error-analysis](references/error-analysis.md) → [axial-coding](references/axial-coding.md) → [evaluators-overview](references/evaluators-overview.md)**Building Evaluator:**[fundamentals](references/fundamentals.md) → [common-mistakes-python](references/common-mistakes-python.md) → evaluators-{code|llm}-{python|typescript} → validation-evaluators-{python|typescript}**RAG Systems:**[evaluators-rag](references/evaluators-rag.md) → evaluators-code-* (retrieval) → evaluators-llm-* (faithfulness)**Production:**[production-overview](references/production-overview.md) → [production-guardrails](references/production-guardrails.md) → [production-continuous](references/production-continuous.md)## Reference Categories| Prefix | Description || ------ | ----------- || `fundamentals-*` | Types, scores, anti-patterns || `observe-*` | Tracing, sampling || `error-analysis-*` | Finding failures || `axial-coding-*` | Categorizing failures || `evaluators-*` | Code, LLM, RAG evaluators || `experiments-*` | Datasets, running experiments || `validation-*` | Validating evaluator accuracy against human labels || `production-*` | CI/CD, monitoring |## Key Principles| Principle | Action || --------- | ------ || Error analysis first | Can't automate what you haven't observed || Custom > generic | Build from your failures || Code first | Deterministic before LLM || Validate judges | >80% TPR/TNR || Binary > Likert | Pass/fail, not 1-5 |

Frequently asked questions

Githubcopilot Phoenix Evals