Plugins
12 pluginscurated
Run Agent Evaluation
Sets up evaluation framework, runs benchmarks, and produces comparative analysis of agent performance.
9 skills · plugin
@microsoft
Microsoft 365 Agents Toolkit
Toolkit for building and evaluating Microsoft 365 Copilot declarative agents — scaffolding, JSON manifest development, capability configuration, and eval workflows.
6 skills · plugin
curated
ML Model Lifecycle
Train, evaluate, and deploy a production ML system with monitoring.
10 skills · plugin
curated
Design Pricing Strategy
Design a pricing strategy by analyzing market, evaluating financial impact, and recommending pricing models.
6 skills · plugin
curated
Monetization to Billing Pipeline
For product managers and finance ops: brainstorm strategies, evaluate financial impact, then execute billing workflows.
4 skills · plugin
curated
Analyze and Prioritize Feature Requests
Install this pack to categorize, evaluate, and prioritize customer feature requests against product goals.
3 skills · plugin
@owl-listener
Prototyping Testing
Prototyping and testing skills: wireframe specs, usability heuristics, heuristic evaluations, accessibility audits, A/B test design, and benchmark analysis.
8 skills · plugin
curated
Google Cloud Well-Architected
For architects evaluating Google Cloud workloads against the Well-Architected Framework pillars: reliability, cost optimization, and operational excellence.
6 skills · plugin
@owl-listener
Visual Critique
Visual critique skills: hierarchy analysis, brand consistency checks against mood/voice/tokens, composition evaluation, and typography audits — with a /critique-screen command that compiles a prioritised fix list.
7 skills · plugin
@alirezarezvani
Ra Qm Team
14 regulatory affairs & quality management skills for HealthTech/MedTech: ISO 13485 QMS, MDR 2017/745, FDA 510(k)/PMA, GDPR/DSGVO, ISO 27001 ISMS, CAPA management, risk management, clinical evaluation, SOC 2 compliance.
10 skills · plugin
@alirezarezvani
Agenthub
Multi-agent collaboration — spawn N parallel subagents that compete on code optimization, content drafts, research approaches, or any task that benefits from diverse solutions. 7 slash commands (/hub:init, /hub:spawn, /hub:status, /hub:eval, /hub:merge, /hub:board, /hub:run), agent templates, DAG-based orchestration, LLM judge mode, message board coordination.
8 skills · plugin
@alirezarezvani
Engineering Team
32 engineering skills: architecture, frontend, backend, fullstack, QA, DevOps, security, AI/ML, data engineering, Playwright (9 sub-skills), self-improving agent, Stripe integration, TDD guide, tech stack evaluator, Google Workspace CLI, a11y audit (WCAG 2.2), Azure cloud architect, GCP cloud architect, security pen testing, Snowflake development, adversarial-reviewer, ai-security, cloud-security,
16 skills · plugin
Results for “eval”
809 skills
orchestra-research Nemo Evaluator Sdk
Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution on local Docker, Slurm HPC, or cloud platforms.
10.4k · bundle
qcmuu Evaluating Cosmos Policy
Evaluates NVIDIA Cosmos Policy on LIBERO and RoboCasa simulation environments. Use when setting up cosmos-policy for robot manipulation evaluation, running headless GPU evaluations with EGL rendering, or profiling inference latency on cluster or local GPU machines.
0 · bundle
lucassantana-dev Eval
Evaluate LLM outputs systematically — benchmarks, automated metrics, human preference, and regression tracking
1 · bundle
github Arize Experiment
Creates, runs, and analyzes Arize experiments for evaluating and comparing model performance using the ax CLI.
36.2k · bundle
seaworld008 Oracle
Designing and evaluating AI/ML systems: prompt engineering, RAG design, LLM application patterns, AI safety, evaluation frameworks, MLOps, cost optimization. Use for AI pipelines or eval harnesses.
65 · bundle
github Eval Driven Dev
Build automated evaluation pipelines for Python LLM applications using real LLM calls and structured test datasets.
36.2k · bundle
samyakjhaveri Overnight Eval
Launches long-running evaluation batches in isolated tmux sessions with pre-flight verification, monitoring, and post-flight analysis for unattended runs.
0
jarbitechture Eval
Evaluate and rank agent results by metric or LLM judge for an AgentHub session.
0
dylanckawalec Eval
Evaluate and rank agent results by metric or LLM judge for an AgentHub session.
3
alirezarezvani RAG Architect
Design, tune, and evaluate production RAG pipelines with deterministic tools for chunking, pipeline design, and retrieval evaluation.
20.4k · bundle
owl-listener Heuristic Evaluation
Conduct expert heuristic evaluations of digital interfaces using Nielsen's 10 usability heuristics and domain-specific criteria.
1.7k
mocchalera Evaluate Edit
Runs regression evaluations comparing agent edits against human-approved golden projects, and registers new goldens after human approval.
3
samyakjhaveri Post Eval
Runs a post-batch analysis pipeline after an eval completes: verifies results, executes analysis scripts, refreshes dashboards, and writes a summary report.
0
kintsugi-programmer LLM Evaluation
Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.
0
k-dense-ai Scholar Evaluation
Systematically evaluate scholarly work using the ScholarEval framework, providing structured assessment across research quality dimensions including problem formulation, methodology, analysis, and writing with quantitative scoring and actionable feedback.
30.2k · bundle
github Arize Evaluator
Creates and runs LLM-as-judge evaluators on Arize, including managing tasks, column mappings, and continuous monitoring.
36.2k · bundle
samyakjhaveri Eval Grader
Grades and classifies evaluation batch results, applying exclusions, diagnosing failure modes, computing pass rates, and generating summary tables for papers.
0
projectious-work LLM Evaluation
LLM output evaluation — automated metrics, LLM-as-judge, A/B testing, regression testing. Use when measuring LLM output quality, comparing prompt or model versions, building an automated eval pipeline, setting up regression tests for prompt changes, or evaluating RAG systems and bias/safety.
0
dotnet Eval Performance
Diagnose and improve MSBuild project evaluation performance by analyzing phases, glob patterns, import chains, and property functions.
4k
ssrjkk LLM Eval
Evaluates LLM performance using BLEU, ROUGE metrics and LLM-as-judge. Use for model testing.
2 · bundle
jeffallan Prompt Engineer
Designs, optimizes, and evaluates prompts for LLMs, including structured outputs, chain-of-thought, and evaluation frameworks.
10.4k · bundle
projectious-work Eval Gate Authoring
Turn observed run outputs into eval-spec Artifacts, paired Gates, and policy bindings. Use when creating or calibrating automated, human, or LLM-as-judge eval gates for processkit workflows.
0 · bundle
levalencia Setup
Set up a new autoresearch experiment interactively. Collects domain, target file, eval command, metric, direction, and evaluator.
3
jarbitechture Setup
Set up a new autoresearch experiment interactively. Collects domain, target file, eval command, metric, direction, and evaluator.
0 · bundle
dylanckawalec Setup
Set up a new autoresearch experiment interactively. Collects domain, target file, eval command, metric, direction, and evaluator.
3
alirezarezvani Self Eval
Honestly evaluate AI work quality using a two-axis scoring system with mandatory devil's advocate reasoning and cross-session anti-inflation detection.
20.4k
huggingface Huggingface Community Evals
Run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware, with backend selection between vLLM, Transformers, and accelerate.
10.8k · bundle
sinhoneyy Eval
Evaluate and rank agent results by metric or LLM judge for an AgentHub session. Use when the user runs /hub:eval or asks to score, compare, or pick a winner among completed AgentHub agents.
11
thedixitjain Eval
Evaluate and rank agent results by metric or LLM judge for an AgentHub session. Use when the user runs /hub:eval or asks to score, compare, or pick a winner among completed AgentHub agents.
2
jiachen-t-wang Llava Critic Learning To Evaluate Multimodal Models Arxiv 24
LLaVA-Critic: Learning to Evaluate Multimodal Models
6
muratcankoylan Evaluation
Build evaluation frameworks for agent systems with deterministic checks, regression suites, multi-dimensional rubrics, quality gates, production monitoring, and outcome measurement.
16.9k · bundle
orchestra-research Evaluating Llms Harness
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag) using standardized prompts and metrics. Supports HuggingFace, vLLM, and API backends.
10.4k · bundle
agentskillexchange Csp Policy Analyzer
Parses and evaluates Content Security Policy headers using csp-parse and csp-evaluator libraries. Identifies overly permissive directives, missing protections, and generates tightened policy recommendations.
28
qhjqhj00 Menli
Evaluates the robustness and alignment with human judgment of reference-based and reference-free evaluation metrics for machine translation and summarization, particularly under adversarial conditions.
3
qhjqhj00 Aya Eval
Evaluates open-ended generation quality of multilingual LLMs across brainstorming, planning, and long-form tasks, using AYA and DOLLY datasets with qualitative fluency and quality scoring.
3
qhjqhj00 Bbq Eval
Evaluates social bias in question-answering models using the BBQ benchmark, measuring accuracy and a bias score across ambiguous and disambiguated contexts to reveal reliance on stereotypes.
3