Plugins

12 plugins

Run Agent Evaluation

Sets up evaluation framework, runs benchmarks, and produces comparative analysis of agent performance.

9 skills · plugin

Microsoft 365 Agents Toolkit

Toolkit for building and evaluating Microsoft 365 Copilot declarative agents — scaffolding, JSON manifest development, capability configuration, and eval workflows.

6 skills · plugin

ML Model Lifecycle

Train, evaluate, and deploy a production ML system with monitoring.

10 skills · plugin

Design Pricing Strategy

Design a pricing strategy by analyzing market, evaluating financial impact, and recommending pricing models.

6 skills · plugin

Monetization to Billing Pipeline

For product managers and finance ops: brainstorm strategies, evaluate financial impact, then execute billing workflows.

4 skills · plugin

Analyze and Prioritize Feature Requests

Install this pack to categorize, evaluate, and prioritize customer feature requests against product goals.

3 skills · plugin

Prototyping Testing

Prototyping and testing skills: wireframe specs, usability heuristics, heuristic evaluations, accessibility audits, A/B test design, and benchmark analysis.

8 skills · plugin

Google Cloud Well-Architected

For architects evaluating Google Cloud workloads against the Well-Architected Framework pillars: reliability, cost optimization, and operational excellence.

6 skills · plugin

Visual Critique

Visual critique skills: hierarchy analysis, brand consistency checks against mood/voice/tokens, composition evaluation, and typography audits — with a /critique-screen command that compiles a prioritised fix list.

7 skills · plugin

@alirezarezvani

14 regulatory affairs & quality management skills for HealthTech/MedTech: ISO 13485 QMS, MDR 2017/745, FDA 510(k)/PMA, GDPR/DSGVO, ISO 27001 ISMS, CAPA management, risk management, clinical evaluation, SOC 2 compliance.

10 skills · plugin

@alirezarezvani

Multi-agent collaboration — spawn N parallel subagents that compete on code optimization, content drafts, research approaches, or any task that benefits from diverse solutions. 7 slash commands (/hub:init, /hub:spawn, /hub:status, /hub:eval, /hub:merge, /hub:board, /hub:run), agent templates, DAG-based orchestration, LLM judge mode, message board coordination.

8 skills · plugin

@alirezarezvani

Engineering Team

32 engineering skills: architecture, frontend, backend, fullstack, QA, DevOps, security, AI/ML, data engineering, Playwright (9 sub-skills), self-improving agent, Stripe integration, TDD guide, tech stack evaluator, Google Workspace CLI, a11y audit (WCAG 2.2), Azure cloud architect, GCP cloud architect, security pen testing, Snowflake development, adversarial-reviewer, ai-security, cloud-security,

16 skills · plugin

Results for “eval”

362 skills

lucaspmarie-a11y

Instruments LLM applications with Langfuse for tracing, observability, and evaluation, covering setup, OpenAI and LangChain integrations, and best practices.

Computes the InfoLM metric from torchmetrics for evaluating text generation against ground truth, with configurable information measures and sentence-level scoring.

Evaluates machine-generated log summaries without human-written references, using LLM judgment and dense embeddings to score relevance, informativeness, and coherence.

Tao Train Rtdetr

Train, evaluate, distill, quantize, export, and run inference for RT-DETR object detection models using NVIDIA TAO.

Instrument LLM applications with Langfuse for tracing, prompt management, evaluation, and cost tracking, including integrations with OpenAI, LangChain, and LlamaIndex.

Langsmith Observability

LLM observability platform for tracing, evaluation, and monitoring. Use when debugging LLM applications, evaluating model outputs against datasets, monitoring production systems, or building systematic testing pipelines for AI applications.

Langsmith Observability

LLM observability platform for tracing, evaluation, and monitoring. Use when debugging LLM applications, evaluating model outputs against datasets, monitoring production systems, or building systematic testing pipelines for AI applications.

Spawns multiple parallel AI agents that compete on the same task using isolated git worktrees, evaluates results, and merges the best solution.

20.4k · bundle

Manage Arize datasets and examples using the ax CLI: create, list, get, export, and append datasets for evaluation and experimentation.

36.2k · bundle

Run autonomous optimization loops that iteratively improve artifacts against evaluators using hypothesis tree refinement, without overfitting.

30.2k · bundle

Evaluates the naturalness, speaker similarity, and real-time synthesis speed of a Mandarin speech cloning system across diverse practical application scenarios.

Computes the SQuAD metric using torchmetrics, given predictions and ground truth. Use when evaluating question-answering outputs with exact match and F1 scores.

Model Evaluation

Evaluate model quality with task-appropriate metrics and systematic error analysis. Use when: (1) comparing models, (2) analyzing failures, (3) setting go/no-go thresholds. NOT for: production monitoring implementation.

Build and evaluate machine learning models using scikit-learn for classification, regression, clustering, dimensionality reduction, and preprocessing.

30.2k · bundle

Harness Engineering

Designs autonomous agent harnesses with locked evaluators, editable surfaces, durable logging, novelty gates, pruning, rollback, and human approval boundaries.

Phoenix Observability

Self-hosted observability platform for LLM applications, providing tracing, evaluation, datasets, experiments, and real-time monitoring to debug and improve AI systems.

Evaluates how well internal model representations (hidden states) predict token-level information importance in summarization tasks, using NDCG@10 and Spearman's rank correlation.

Prompt Engineer

Designs and optimizes prompts for LLM-powered applications, covering system prompt architecture, context management, output formatting, and evaluation.

Create Skill Test

Scaffolds eval.yaml test files for agent skills in the dotnet/skills repository, including scenario writing, assertion definition, and rubric creation.

Design Everyday Things

Apply foundational design principles—affordances, signifiers, constraints, mappings, and feedback—to evaluate and improve product usability, bridging the gulfs of execution and evaluation.

Scaffolds a coder-eval task suite in a repository by scanning for testable components, writing one real task, and validating it.

Phoenix Observability

Open-source AI observability platform for LLM tracing, evaluation, and monitoring. Use when debugging LLM applications with detailed traces, running evaluations on datasets, or monitoring production AI systems with real-time insights.

Phoenix Observability

Open-source AI observability platform for LLM tracing, evaluation, and monitoring. Use when debugging LLM applications with detailed traces, running evaluations on datasets, or monitoring production AI systems with real-time insights.

Design multi-agent system architectures, generate tool schemas for Anthropic and OpenAI formats, and evaluate execution logs for cost, latency, and failure bottlenecks.

20.4k · bundle

Agent Self Evaluation

Rates an agent's own output on five axes — accuracy, completeness, clarity, actionability, conciseness — producing a structured scorecard with evidence and improvement suggestions.

Evaluates geometric consistency in text-to-video generation by measuring structural and motion coherence across camera trajectories, detecting deformation and occlusion artifacts in static scenes.

lucassantana-dev

Evaluate retrieval quality from the local RAG index

LLM observability with Langfuse — tracing, evals, prompt management, cost tracking

Microsoft Foundry

Deploy, evaluate, fine-tune, and manage Microsoft Foundry agents end-to-end using Azure Developer CLI and MCP tools.

Fine-tune models on Azure AI Foundry using SFT, DPO, or RFT, covering dataset preparation, training job submission, deployment, and evaluation.

Evaluates text-to-speech systems by measuring distributional distance between synthetic and real speech across five factors, producing a scalar score without subjective MOS ratings.

Agentic Engineering

Guides AI agents through engineering workflows with eval-first execution, task decomposition, cost-aware model routing, and review focus for generated code.

Context Engineering Collection

Provides structured guidance for building production-grade AI agent systems through context engineering, covering fundamentals, architectural patterns, operational excellence, and evaluation.

16.9k · bundle

Simulates meiotic recombination to produce offspring genomes from parent pairs, modeling Mendelian segregation, de novo mutation, sex determination, trait inference, and clinical evaluation against a disease registry.

Computes the HARE Score, an entity- and relation-centric metric for evaluating machine-generated histopathology reports against ground truth, using GatorTronS+SapBERT embeddings and relation F1.

Benchmarks LLM bias by scoring responses to automatically generated open-ended questions across sensitive attributes, producing a composite fitness score from 0 to 5.