Results for “rubric-scoring”

Validate SKILL.md files against a quality rubric, scoring schema and instruction structure out of 100 and reporting a pass/fail verdict with specific fixes.

Implement iterative evaluation and refinement loops for AI agent outputs, using self-critique, evaluator-optimizer patterns, and rubric-based scoring to improve quality.

Generates four improved variations of any installed skill, scores them against a weighted rubric, and applies the winning version while preserving the original.

Design a single-point rubric with one criterion and open columns for evidence. Use for student self-assessment, peer feedback, teacher formative feedback, or pre-task planning. Works with any learning target, with or without a band system.

Reviews resumes for formatting, design, content, and job-specific fit; scores job postings for fit and interview odds; develops rubrics; extracts content from conversations; and audits the resume generation pipeline.

Build a five-level rubric with coherent logic for a learning target within a developmental band. Use for Manning methodology programmes where Competent = success. For general curriculum rubrics, use criterion-referenced-rubric-generator instead.

Computes the AUROC metric using torchmetrics, handling binary, multiclass, and multilabel tasks with configurable thresholds and averaging.

Benchmark-relative quality gate. Scores the draft against the research dossier's beat spec (depth, consensus coverage, evidence) plus AI-tell and voice signals, runs an adversarial read armed with the SERP benchmark, and emits the verdict that gates the pipeline.

Build evaluation frameworks for agent systems, covering rubric design, test set creation, and automated evaluation pipelines.

Spec Craft

Orchestrator for the eval-output skill suite — evaluate LLM and agent outputs for quality, accuracy, helpfulness, and safety using structured rubrics and LLM-as-judge techniques. Load when the user says "evaluate this output", "score this response", "run an eval", "LLM as judge", "evaluate agent output", "how good is this response", "rate this answer", "eval this", or provides an LLM output that should be assessed for quality. Single entry point for all output evaluation workflows.

Defines ideal customer profile filters, scores inbound and outbound leads, and builds a lightweight qualification rubric to sharpen pipeline focus for founder-led sales.

Scores deal margin and risk, routes discount approval to the right human approver, and redlines terms against commercial policy for per-deal review.

Rank an existing set of context chunks by relevance, diversity, freshness, and utility. Use when retrieval has already produced candidates that must be scored or reranked; use context-retrieval when the source corpus still needs to be searched.

Score and prioritize leads based on firmographic fit and behavioral engagement signals, producing ranked tiers for sales team focus. Use when the user requests lead scoring or provides relevant inputs for this workflow.

Score raw leads as HOT/WARM/COOL based on config-driven weights from agency.config.json

PASS/FAIL eval rubrics and alignment loops for Sid Orchestra. Use when the user says sid evals, @sid-evals, grade this, eval gate, alignment score, or wants to stop AI slop with evaluation gates.

Computes the Receiver Operating Characteristic (ROC) metric using torchmetrics, supporting binary, multiclass, and multilabel tasks.

Create, apply, and manage AgentRC policies to customize readiness scoring, disable checks, override impact levels, set pass-rate thresholds, and enforce CI gating.

PASS/FAIL eval rubrics and alignment loops for Sid Orchestra (global). Use when the user says sid evals, @sid-evals, grade this, eval gate, alignment score, or wants to stop AI slop. Works in any workspace; bootstraps EVALS.md from ~/.cursor/skills/sid-orchestra/templates if missing.

Evaluate and improve interface usability using heuristic analysis based on Nielsen's 10 heuristics, Krug's laws, and severity ratings.

Prioritize product work with explicit criteria, scoring, tradeoffs, and decision rationale.

Judges agent outputs against LaRuche's quality standards, returning a structured scorecard with scores, verdict, and actionable corrections.

TRAK: Attributing Model Behavior at Scale

The choice is not about scale or sophistication.

Design grading plans. TRIGGERS - Use when user needs help with grading-plan related tasks.

Computes the LogAUC metric using the torchmetrics implementation for binary, multiclass, or multilabel classification tasks.

Lightweight ContentShake AI self-check the /draft stage can call before saving. Returns just SEO + Quality scores (no full optimization) so the writer knows whether the draft is in winning territory before /quality-check runs. Fails soft when SEMRUSH_API_KEY is unset.

Score LLM and agent outputs using LLM-as-judge techniques — direct scoring against rubrics or pairwise comparison between two outputs. Includes built-in bias mitigation for position bias, length bias, and self-enhancement bias. Load when the user asks to score an output, judge a response, evaluate against a rubric, compare two outputs, do direct scoring, run pairwise comparison, or says "rate this", "which response is better", "score this against the rubric", "judge this output", "LLM as judge this". Sub-skill of eval-output orchestrator.

Evaluate and rank agent results by metric or LLM judge for an AgentHub session. Use when the user runs /hub:eval or asks to score, compare, or pick a winner among completed AgentHub agents.