Results for “rubric-scoring”

Computes the AUROC metric using torchmetrics, handling binary, multiclass, and multilabel tasks with configurable thresholds and averaging.

Spec Craft

Rank an existing set of context chunks by relevance, diversity, freshness, and utility. Use when retrieval has already produced candidates that must be scored or reranked; use context-retrieval when the source corpus still needs to be searched.

Score raw leads as HOT/WARM/COOL based on config-driven weights from agency.config.json

PASS/FAIL eval rubrics and alignment loops for Sid Orchestra. Use when the user says sid evals, @sid-evals, grade this, eval gate, alignment score, or wants to stop AI slop with evaluation gates.

Create, apply, and manage AgentRC policies to customize readiness scoring, disable checks, override impact levels, set pass-rate thresholds, and enforce CI gating.

PASS/FAIL eval rubrics and alignment loops for Sid Orchestra (global). Use when the user says sid evals, @sid-evals, grade this, eval gate, alignment score, or wants to stop AI slop. Works in any workspace; bootstraps EVALS.md from ~/.cursor/skills/sid-orchestra/templates if missing.

Prioritize product work with explicit criteria, scoring, tradeoffs, and decision rationale.

The choice is not about scale or sophistication.

Design grading plans. TRIGGERS - Use when user needs help with grading-plan related tasks.

Computes the LogAUC metric using the torchmetrics implementation for binary, multiclass, or multilabel classification tasks.

Criteria for deciding when, where and how to remember information. Use this skill whenever you might need to update a core memory block, file something under memory/, or write to a state file. It is CRITICAL that you use this skill.

Design grading plans. TRIGGERS - Use when user needs help with grading-plan related tasks.

Use to build and maintain a reusable bank of QA criteria as observable decision rules shared across scorecards. Trigger for "criterion library", "standardise our QA criteria", "write better rubric items", "was professional is too vague", versioning criteria across teams, or turning adjectives into pass/fail decision rules with examples.

Build a USASpending relationship snapshot for the incumbent and buying agency on a pursuit row. Use when user needs award flows and rel counts before battlecard or teaming work — deterministic from DuckDB bulk.

Detect and evaluate arbitrage opportunities across sportsbooks and prediction markets. Calculate guaranteed-profit scenarios, middle bets, and cross-platform price discrepancies. Use when comparing odds across books, calculating arb percentages, evaluating middle opportunities, or building odds-comparison workflows. Also trigger for 'arb bet', 'sure bet', 'arbitrage', 'odds comparison', 'middle bet', 'risk-free bet', or 'line shopping'.

Deterministically score, route, rank, and select at most one fully vetted Pleasur.ai Stage 01 blog-keyword candidate using product-fit-dominant business value, traffic opportunity, brand fit, DR-relative winnability, and a free-seeker penalty. Use only after BID and AIO evaluation are complete.

Routes quality reviews to specialized critic agents based on file type or flags, covering peer review, code review, and manuscript polish.

This skill should be used when the user asks to "design interview processes", "create hiring pipelines", "calibrate interview loops", "generate interview questions", "design competency matrices", "analyze interviewer bias", "create scoring rubrics", "build question banks", or "optimize hiring systems". Use for designing role-specific interview loops, competency assessments, and hiring calibration systems.