Results for “eval”

Deliberative consultation with Gemini CLI. Ask, evaluate, critique, iterate until workable agreement.

Run a single experiment iteration. Edit the target file, evaluate, keep or discard.

DeepEval — LLM evaluation framework, RAG metrics, hallucination detection, red-teaming, CI/CD integration

Deliberative consultation with Kimi API. Ask, evaluate, critique, iterate until workable agreement.

Deliberative consultation with Codex CLI. Ask, evaluate, critique, iterate until workable agreement.

Create a new AgentHub collaboration session with task, agent count, and evaluation criteria.

Autonomously optimize any Claude Code skill by running it repeatedly, scoring outputs against binary evals, mutating the prompt, and keeping improvements. Based on Karpathy's autoresearch methodology. Use when: optimize this skill, improve this skill, run autoresearch on, make this skill better, self-improve skill, benchmark skill, eval my skill, run evals on. Outputs: an improved SKILL.md, a results log, and a changelog of every mutation tried.

Evaluates and selects Threat Intelligence Platform (TIP) products based on organizational requirements including feed integration capability, STIX/TAXII support, workflow automation, analyst interface, and total cost of ownership.

Connect to an existing Chrome browser via Chrome DevTools Protocol to navigate, click, type, screenshot, evaluate JavaScript, read the accessibility tree, and monitor console or network events.

Coordinates quality assurance workflows by routing testing tasks to specialized sub-skills for API testing, performance benchmarking, test analysis, tool evaluation, and process optimization.

Audits AGENTS.md and CLAUDE.md files for bloat, redundancy, and ineffective rules, then runs A/B evals to prove which instructions are dead weight.

Route AI agent engineering prompts to architecture, orchestration, evaluation, safety, debugging, context, prompt, MCP, persona, local AI, and Compound Engineering skills. Use when prompts mention agents, agent harnesses, agentic workflows, orchestration, evals, context management, MCP servers, or compound engineering.

Evaluates the faithfulness of abstractive summaries by generating questions from summary sentences and verifying if the answers can be extracted from the source document, reporting Pearson and Spearman correlations with human judgments.

Benchmarks systems on the Abstraction and Reasoning Corpus (ARC) by requiring inference of abstract transformation rules from few input-output grid demonstrations and application to novel test cases, reporting the fraction of tasks solved.

Use when the superpowers-evaluator returned REWORK on a batch, when fixing rework items from an evaluation report, or when receiving any code review feedback on superpowers output. Requires technical rigor and verification instead of performative agreement or blind implementation.

Holistic evaluation framework for language models measuring accuracy, calibration, robustness, and fairness

实验判决门：Review LLM 独立评判实验结果 → 4 种判决路径 → 自动更新 claims confidence、ideas status、graph edges

Train, evaluate, export, quantize, and run inference for a Deformable DETR 2D object detection model using TAO, with deformable attention for efficient multi-scale feature processing.

Provides canonical MSBuild property definition patterns including conditional defaults, composition, path normalization, target framework detection, and evaluation order for diagnosing and fixing property issues in .props and .csproj files.

Benchmarks zero-shot in-context learning on BIG-Bench Hard multiple-choice tasks, comparing self-generated demonstrations against direct prompting and chain-of-thought baselines, and reports accuracy.

Use when designing or auditing experiments for an ACL paper, covering tuned LLM baselines, multi-dataset and multilingual evaluation, statistical significance and variance, human evaluation with agreement reporting, contamination and prompt-sensitivity controls, ablations, and error-analysis expectations in NLP reviewing.

Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when: langfuse, llm observability, llm tracing, prompt management, llm evaluation.

Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when: langfuse, llm observability, llm tracing, prompt management, llm evaluation.

Expert in Langfuse - the open-source LLM observability platform. Covers tracing, prompt management, evaluation, datasets, and integration with LangChain, LlamaIndex, and OpenAI. Essential for debugging, monitoring, and improving LLM applications in production. Use when: langfuse, llm observability, llm tracing, prompt management, llm evaluation.

Giskard RAGET (RAG Evaluation Toolkit): automatic testset generation (simple / complex / distracting / conversational), component-level scoring (retriever / generator / rewriter), hallucination and bias tests, CI integration. Compared to RAGAS and DeepEval. USE WHEN: user mentions "Giskard", "RAGET", "Giskard RAG toolkit", "automatic testset generation", "component-level RAG scoring", "hallucination test Giskard" DO NOT USE FOR: general RAGAS usage - use `rag-evaluation`; Stanford ARES - use `ares-framework`; CI/CD wiring - use `continuous-evaluation`

Evaluates image captions by converting them into scene graphs and computing an F-score over semantic propositions, measuring how well a generated caption captures the meaning of an image compared to human references.

Benchmarks energy-function-based safe control algorithms on the BIS (Benchmark of Interactive Safety) dataset, scoring safety, efficiency, and hybrid performance in human-robot and robot co-working scenarios.

Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.

Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.

Evaluate design from a UX perspective, assessing visual hierarchy, information architecture, emotional resonance, cognitive load, and overall quality with quantitative scoring, persona-based testing, and actionable feedback. Use when the user asks to review, critique, evaluate, or give feedback on a design or component.

Evaluate codebase against enterprise ICP for production readiness, compliance, and multi-stakeholder journey coverage

Loop feature-interviews over a brainstorm idea set and consolidate survivors into the roadmap

Understand blockchain technology, interact with smart contracts, and evaluate when distributed ledgers solve real problems.

One-shot lifecycle command that chains init → baseline → spawn → eval → merge in a single invocation.

Test coverage quality analysis — behavioral coverage, critical gap identification, test resilience evaluation.

Address PR review comments by fetching, evaluating, fixing, replying, pushing, and re-requesting review.