Results for “l-eval”

Evaluates Claude Agent Skills on 10 quality axes with letter grades (A+ through F) and specific improvement recommendations. Use when auditing a skill, comparing skills, prioritizing improvements, or performing quality control on a skill library. Activate on "grade skill", "evaluate skill", "skill quality", "skill audit", "skill review", "rate skill". NOT for creating skills (use skill-architect), grading code quality, or evaluating non-skill documents.

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.

Uses a multi-agent generator-evaluator feedback loop to build high-quality applications from a single prompt, inspired by GANs and Anthropic's harness design.

Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.

Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.

Run a single experiment iteration. Edit the target file, evaluate, keep or discard.

DeepEval — LLM evaluation framework, RAG metrics, hallucination detection, red-teaming, CI/CD integration

Use when creating, editing, evaluating, testing, or verifying ANY skill or skill-related file (SKILL.md, skill resources, skill scripts, or skill assets). If you're asked to evaluate or test a skill's effectiveness, use this skill.

Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when: langfuse, llm observability, llm tracing, prompt management, llm evaluation.

Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when: langfuse, llm observability, llm tracing, prompt management, llm evaluation.

Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when: langfuse, llm observability, llm tracing, prompt management, llm evaluation.

Holistic evaluation framework for language models measuring accuracy, calibration, robustness, and fairness

实验判决门：Review LLM 独立评判实验结果 → 4 种判决路径 → 自动更新 claims confidence、ideas status、graph edges

Run a heuristic evaluation of interactive UIs against Nielsen's 10 usability heuristics and interaction add-ons, scoring the rendered artifact and producing a prioritized fix list.

Route AI agent engineering prompts to architecture, orchestration, evaluation, safety, debugging, context, prompt, MCP, persona, local AI, and Compound Engineering skills. Use when prompts mention agents, agent harnesses, agentic workflows, orchestration, evals, context management, MCP servers, or compound engineering.

Autonomously optimize any Claude Code skill by running it repeatedly, scoring outputs against binary evals, mutating the prompt, and keeping improvements. Based on Karpathy's autoresearch methodology. Use when: optimize this skill, improve this skill, run autoresearch on, make this skill better, self-improve skill, benchmark skill, eval my skill, run evals on. Outputs: an improved SKILL.md, a results log, and a changelog of every mutation tried.

Benchmarks zero-shot in-context learning on BIG-Bench Hard multiple-choice tasks, comparing self-generated demonstrations against direct prompting and chain-of-thought baselines, and reports accuracy.

Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.

Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.

Giskard RAGET (RAG Evaluation Toolkit): automatic testset generation (simple / complex / distracting / conversational), component-level scoring (retriever / generator / rewriter), hallucination and bias tests, CI integration. Compared to RAGAS and DeepEval. USE WHEN: user mentions "Giskard", "RAGET", "Giskard RAG toolkit", "automatic testset generation", "component-level RAG scoring", "hallucination test Giskard" DO NOT USE FOR: general RAGAS usage - use `rag-evaluation`; Stanford ARES - use `ares-framework`; CI/CD wiring - use `continuous-evaluation`

Loop feature-interviews over a brainstorm idea set and consolidate survivors into the roadmap

Understand blockchain technology, interact with smart contracts, and evaluate when distributed ledgers solve real problems.

One-shot lifecycle command that chains init → baseline → spawn → eval → merge in a single invocation.

Test coverage quality analysis — behavioral coverage, critical gap identification, test resilience evaluation.

One-shot lifecycle command that chains init → baseline → spawn → eval → merge in a single invocation.

One-shot lifecycle command that chains init → baseline → spawn → eval → merge in a single invocation.

Use when designing or auditing experiments for an ACL paper, covering tuned LLM baselines, multi-dataset and multilingual evaluation, statistical significance and variance, human evaluation with agreement reporting, contamination and prompt-sensitivity controls, ablations, and error-analysis expectations in NLP reviewing.

Evaluate design from a UX perspective, assessing visual hierarchy, information architecture, emotional resonance, cognitive load, and overall quality with quantitative scoring, persona-based testing, and actionable feedback. Use when the user asks to review, critique, evaluate, or give feedback on a design or component.

Deobfuscates malicious JavaScript code used in web-based attacks, phishing pages, and dropper scripts by reversing encoding layers, eval chains, string manipulation, and control flow obfuscation to reveal the original malicious logic.

Stateful single-mission improvement loop with strict evaluator contract, markdown decision logs, and max-runtime stop behavior

Machine learning in Python with scikit-learn. Use for classification, regression, clustering, model evaluation, and ML pipelines.

Evaluate scientific claims and evidence quality by assessing experimental design, identifying biases and confounders, and applying evidence grading frameworks like GRADE and Cochrane Risk of Bias.

Build and evaluate LLM prompts. Use when crafting system prompts, comparing variants, estimating tokens, or managing prompt templates.

Evaluates automated metrics and vision-language models on identifying granular errors in detailed image descriptions and ranking paired descriptions against human judgments, using macro F1, pairwise accuracy, Spearman rank ρ, and Kendall's τ.