Results for “rubric-scoring”
22 skills
dvy1987 Eval Rubric Design
Design structured evaluation rubrics for scoring LLM and agent outputs — defining quality dimensions, scoring scales, hard gates, score descriptions, and edge cases. Load when the user asks to create an eval rubric, define evaluation criteria, design scoring dimensions, write an eval spec, or says "what should I evaluate", "design a rubric", "create eval criteria", "define quality dimensions", "evaluation rubric for", "how do I measure quality of". Sub-skill of eval-output orchestrator.
3 · bundle
muratcankoylan Advanced Evaluation
Provides production-grade techniques for evaluating LLM outputs using LLMs as judges, covering direct scoring, pairwise comparison, bias mitigation, rubric generation, and confidence calibration.
16.9k · bundle
forter Forter Agentic Readiness Audit
Audits a website against the Forter Agentic Readiness Guide by running 25 weighted rubrics, scoring each guideline, and producing a prioritized fix report.
106 · bundle
More results
vvieira010-pixel Single Point Rubric Designer
Design a single-point rubric with one criterion and open columns for evidence. Use for student self-assessment, peer feedback, teacher formative feedback, or pre-task planning. Works with any learning target, with or without a band system.
0
vvieira010-pixel Coherent Rubric Logic Builder
Build a five-level rubric with coherent logic for a learning target within a developmental band. Use for Manning methodology programmes where Competent = success. For general curriculum rubrics, use criterion-referenced-rubric-generator instead.
0
qhjqhj00 Auroc
Computes the AUROC metric using torchmetrics, handling binary, multiclass, and multilabel tasks with configurable thresholds and averaging.
3
antigravity Evaluation
Build evaluation frameworks for agent systems, covering rubric design, test set creation, and automated evaluation pipelines.
42.4k
dvy1987 Eval Output
Orchestrator for the eval-output skill suite — evaluate LLM and agent outputs for quality, accuracy, helpfulness, and safety using structured rubrics and LLM-as-judge techniques. Load when the user says "evaluate this output", "score this response", "run an eval", "LLM as judge", "evaluate agent output", "how good is this response", "rate this answer", "eval this", or provides an LLM output that should be assessed for quality. Single entry point for all output evaluation workflows.
3 · bundle
qhjqhj00 Roc
Computes the Receiver Operating Characteristic (ROC) metric using torchmetrics, supporting binary, multiclass, and multilabel tasks.
3
github Agentic Eval
Implement iterative evaluation and refinement loops for AI agent outputs, using self-critique, evaluator-optimizer patterns, and rubric-based scoring to improve quality.
36.2k
jiachen-t-wang Trak Attributing Model Behavior At Scale Arxiv 2303 14186v2
TRAK: Attributing Model Behavior at Scale
6
qhjqhj00 Logauc
Computes the LogAUC metric using the torchmetrics implementation for binary, multiclass, or multilabel classification tasks.
3
dvy1987 Eval Judge
Score LLM and agent outputs using LLM-as-judge techniques — direct scoring against rubrics or pairwise comparison between two outputs. Includes built-in bias mitigation for position bias, length bias, and self-enhancement bias. Load when the user asks to score an output, judge a response, evaluate against a rubric, compare two outputs, do direct scoring, run pairwise comparison, or says "rate this", "which response is better", "score this against the rubric", "judge this output", "LLM as judge this". Sub-skill of eval-output orchestrator.
3 · bundle
thedixitjain Eval
Evaluate and rank agent results by metric or LLM judge for an AgentHub session. Use when the user runs /hub:eval or asks to score, compare, or pick a winner among completed AgentHub agents.
2
qhjqhj00 F1score
Compute the F1Score metric using torchmetrics when predictions and ground-truth labels are available.
3
jarbitechture Eval
Evaluate and rank agent results by metric or LLM judge for an AgentHub session.
0
alirezarezvani Eval
Evaluate and rank agent results by metric or LLM judge for an AgentHub session.
20.4k
georgeqle Key Moments
Rank a topic's user-flow branches by proof priority (value × risk × frequency) right after user-flow-map, ordering the branches, gating variation breadth, and promoting or pruning flows so state-model and ux-variations grow the tree in proof order — writes only existing flow-tree ordering fields, no schema change.
1 · bundle
qhjqhj00 Accuracy
Evaluates an AI judge system's pairwise ranking accuracy on generated commit messages against a heuristic ground truth from five automatic text metrics, using the MCMD dataset.
3
dylanckawalec Eval
Evaluate and rank agent results by metric or LLM judge for an AgentHub session.
3
sinhoneyy Eval
Evaluate and rank agent results by metric or LLM judge for an AgentHub session. Use when the user runs /hub:eval or asks to score, compare, or pick a winner among completed AgentHub agents.
11
kursku Advanced Evaluation
This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.
55 · bundle