Results for “llm-evaluation”
74 skills
kintsugi-programmer LLM Evaluation
Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.
0
github Eval Driven Dev
Build automated evaluation pipelines for Python LLM applications using real LLM calls and structured test datasets.
36.2k · bundle
alirezarezvani Eval
Evaluate and rank agent results by metric or LLM judge for an AgentHub session.
20.4k
whd4 Langfuse
Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when: langfuse, llm observability, llm tracing, prompt management, llm evaluation.
0
danstrem2 Langfuse
Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when: langfuse, llm observability, llm tracing, prompt management, llm evaluation.
2
dokhacgiakhoa Langfuse
Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when: langfuse, llm observability, llm tracing, prompt management, llm evaluation.
505 · bundle
More results
antigravity Evaluation
Build evaluation frameworks for agent systems, covering rubric design, test set creation, and automated evaluation pipelines.
42.4k
antigravity Langfuse
Provides expertise in Langfuse for LLM observability, including tracing, prompt management, evaluation, and integration with LangChain, LlamaIndex, and OpenAI.
42.4k
phoroth Langfuse
Instrument LLM applications with Langfuse for tracing, prompt versioning, evaluation, and dataset management across Python and JavaScript SDKs.
3
lucaspmarie-a11y Langfuse
Instruments LLM applications with Langfuse for tracing, observability, and evaluation, covering setup, OpenAI and LangChain integrations, and best practices.
5
jorcan Langfuse
Instrument LLM applications with Langfuse for tracing, prompt management, evaluation, and cost tracking, including integrations with OpenAI, LangChain, and LlamaIndex.
0 · bundle
muratcankoylan Advanced Evaluation
Provides production-grade techniques for evaluating LLM outputs using LLMs as judges, covering direct scoring, pairwise comparison, bias mitigation, rubric generation, and confidence calibration.
16.9k · bundle
github Phoenix Evals
Build and run evaluators for AI/LLM applications using Phoenix, covering error analysis, custom evaluators, experiments, and production monitoring.
36.2k · bundle
qhjqhj00 Phoenix Observability
Self-hosted observability platform for LLM applications, providing tracing, evaluation, datasets, experiments, and real-time monitoring to debug and improve AI systems.
3 · bundle
orchestra-research Phoenix Observability
Trace, evaluate, and monitor LLM applications with an open-source observability platform.
10.4k · bundle
tianhao909 Phoenix Observability
Open-source AI observability platform for LLM tracing, evaluation, and monitoring. Use when debugging LLM applications with detailed traces, running evaluations on datasets, or monitoring production AI systems with real-time insights.
1 · bundle
ssrjkk LLM Eval
Evaluates LLM performance using BLEU, ROUGE metrics and LLM-as-judge. Use for model testing.
2 · bundle
qcmuu Phoenix Observability
Open-source AI observability platform for LLM tracing, evaluation, and monitoring. Use when debugging LLM applications with detailed traces, running evaluations on datasets, or monitoring production AI systems with real-time insights.
0 · bundle
qcmuu Evaluating Llms Harness
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
0 · bundle
orchestra-research Evaluating Llms Harness
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag) using standardized prompts and metrics. Supports HuggingFace, vLLM, and API backends.
10.4k · bundle
kursku Advanced Evaluation
This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.
55 · bundle
lambenthan Review
通用跨模型审查：Review LLM 对任意研究制品进行独立评审，输出结构化评分、wiki 实体映射与改进建议
77
ichichuang Evaluating Llms Harness
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
0 · bundle
tianhao909 Evaluating Llms Harness
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
1 · bundle
orchestra-research Nemo Evaluator Sdk
Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution on local Docker, Slurm HPC, or cloud platforms.
10.4k · bundle
pablolion Bmad Advanced Elicitation
Push the LLM to reconsider, refine, and improve its recent output. Use when user asks for deeper critique or mentions a known deeper critique method, e.g. socratic, first principles, pre-mortem, red team.
12 · bundle
qhjqhj00 Cab Eval
Benchmarks LLM bias by scoring responses to automatically generated open-ended questions across sensitive attributes, producing a composite fitness score from 0 to 5.
3
lucassantana-dev Eval
Evaluate LLM outputs systematically — benchmarks, automated metrics, human preference, and regression tracking
1 · bundle
fukukei23 Multi LLM Review
multi-llm-review
0 · bundle
aniruddhaadak80 Model Benchmark
Benchmark LLM performance across tasks — latency, quality, cost comparison.
0
muratcankoylan Evaluation
Build evaluation frameworks for agent systems with deterministic checks, regression suites, multi-dimensional rubrics, quality gates, production monitoring, and outcome measurement.
16.9k · bundle
salacoste Bmad Advanced Elicitation
Push the LLM to reconsider, refine, and improve its recent output. Use when user asks for deeper critique or mentions a known deeper critique method, e.g. socratic, first principles, pre-mortem, red team.
1 · bundle
whd4 Agent Evaluation
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.
0
delorenj Bmad Advanced Elicitation
Push the LLM to reconsider, refine, and improve its recent output. Use when user asks for deeper critique or mentions a known deeper critique method, e.g. socratic, first principles, pre-mortem, red team.
1 · bundle
fukukei23 Sentaku
選択肢(A/B/C)の深掘り比較→淘汰→推奨で判断負担を下げ判断の質を上げるスキル。5段階（L1固定3点/L1.5案拡張Diverge・自動/L2評価軸マトリクス/L3複数LLM弁証論/L4過去判断照合）。 「比較して」「深掘りして」「メリデメ教えて」「お勧めは？」「徹底的に」「過去の判断と照合」「前にどう決めたっけ」「/sentaku」等で発火。teian(浅)の深掘り要求を受け取り、brainstorming(深:設計全体)と棲み分け。
0
jarbitechture Eval
Evaluate and rank agent results by metric or LLM judge for an AgentHub session.
0