Results for “grading”

Grades individual test methods and produces a compact markdown table with a letter grade, score band, and one-line note for each test.

Provides production-grade techniques for evaluating LLM outputs using LLMs as judges, covering direct scoring, pairwise comparison, bias mitigation, rubric generation, and confidence calibration.

Design structured evaluation rubrics for scoring LLM and agent outputs — defining quality dimensions, scoring scales, hard gates, score descriptions, and edge cases. Load when the user asks to create an eval rubric, define evaluation criteria, design scoring dimensions, write an eval spec, or says "what should I evaluate", "design a rubric", "create eval criteria", "define quality dimensions", "evaluation rubric for", "how do I measure quality of". Sub-skill of eval-output orchestrator.

Evaluate LLM outputs systematically — benchmarks, automated metrics, human preference, and regression tracking

Evaluates RAG pipelines using a filesystem-based benchmark with corpus/ and train.json, running evaluate_rag.py to tune retrieval and generation flags and interpret RAGAS metrics.

TRAK: Attributing Model Behavior at Scale

選択肢(A/B/C)の深掘り比較→淘汰→推奨で判断負担を下げ判断の質を上げるスキル。5段階（L1固定3点/L1.5案拡張Diverge・自動/L2評価軸マトリクス/L3複数LLM弁証論/L4過去判断照合）。 「比較して」「深掘りして」「メリデメ教えて」「お勧めは？」「徹底的に」「過去の判断と照合」「前にどう決めたっけ」「/sentaku」等で発火。teian(浅)の深掘り要求を受け取り、brainstorming(深:設計全体)と棲み分け。

Evaluate and rank agent results by metric or LLM judge for an AgentHub session.

Evaluates the correlation between automatic text generation scores and human quality ratings, including robustness to domain and quality drift, using metrics like Kendall's Tau and Pearson correlation.

This skill should be used when the user asks to "implement LLM-as-judge", "compare model outputs", "create evaluation rubrics", "mitigate evaluation bias", or mentions direct scoring, pairwise comparison, position bias, evaluation pipelines, or automated quality assessment.

Evaluates LLM performance using BLEU, ROUGE metrics and LLM-as-judge. Use for model testing.

针对 Modular RAG MCP Server 项目的老师式复习 Agent。按章节带领用户系统复习项目知识点，每道题互动问答、给出参考答案，复习结束后记录掌握进度，每次开始时回顾上次进度并建议继续或复习。Use when user says '复习项目', '帮我复习', '带我复习', '开始复习', '项目复习', 'review project', 'study review', '学习复习', '复盘', or wants to systematically review and study the project.

Score LLM and agent outputs using LLM-as-judge techniques — direct scoring against rubrics or pairwise comparison between two outputs. Includes built-in bias mitigation for position bias, length bias, and self-enhancement bias. Load when the user asks to score an output, judge a response, evaluate against a rubric, compare two outputs, do direct scoring, run pairwise comparison, or says "rate this", "which response is better", "score this against the rubric", "judge this output", "LLM as judge this". Sub-skill of eval-output orchestrator.

Two-stage review engine with four-layer quality model for the WinDAGs meta-DAG. Receives completed node outputs and produces ReviewResult containing QualityVector. Stage 1 (Haiku) checks Floor + Wall on every node. Stage 2 (Sonnet) runs Ceiling evaluation conditionally using economic escalation formula. Enforces BC-EVAL-001 through BC-EVAL-006. Activate when operating as the Evaluator role in the meta-DAG, when reviewing node outputs, when computing quality vectors, or when deciding Stage 2 escalation.

针对 Modular RAG MCP Server 项目的老师式复习 Agent。按章节带领用户系统复习项目知识点，每道题互动问答、给出参考答案，复习结束后记录掌握进度，每次开始时回顾上次进度并建议继续或复习。Use when user says '复习项目', '帮我复习', '带我复习', '开始复习', '项目复习', 'review project', 'study review', '学习复习', '复盘', or wants to systematically review and study the project.

Evaluates Claude Agent Skills on 10 quality axes with letter grades (A+ through F) and specific improvement recommendations. Use when auditing a skill, comparing skills, prioritizing improvements, or performing quality control on a skill library. Activate on "grade skill", "evaluate skill", "skill quality", "skill audit", "skill review", "rate skill". NOT for creating skills (use skill-architect), grading code quality, or evaluating non-skill documents.

Evaluates an AI judge system's pairwise ranking accuracy on generated commit messages against a heuristic ground truth from five automatic text metrics, using the MCMD dataset.

Evaluate and rank agent results by metric or LLM judge for an AgentHub session.

Strategic search planning for agent-driven research. Generates structured search-term matrices with tiered fallback strategies, engine-specific operators, and grading criteria before executing any searches. Use this skill whenever research requires more than a single search query — comparing technologies, verifying claims across sources, surveying a landscape, investigating a multi-faceted question, or building evidence for a decision. Do NOT use for quick factual lookups, fetching a single known URL, or questions answerable from a single source. Covers tech, academic, regulatory, and general domains. Think of it as "research planning" — the matrix is the plan, execution comes after.