# Hierarchical Rubric Based Artifact Grading

> Use this skill when a user wants evaluator data for grading complex artifacts against a rubric, such as codebases, research replications, long reports, logs, or multi-file submissions. Trigger it when the request sounds like 'break the task into rubric checks', 'grade the artifact against detailed criteria', 'evaluate partial progress', or 'make judge data for complex outputs that cannot be scored by one simple metric'. Plain-language examples include: 'judge whether the project meets the checklist', 'score a submission using leaf-node criteria', 'evaluate artifact quality step by step', and 'build judge data for rubric-based grading'.

- Skill: `dingxingdi/hierarchical-rubric-based-artifact-grading-2` (Agent Skill, multi-file: 5 files)
- Install (CLI): `npx skillmds@latest add dingxingdi/hierarchical-rubric-based-artifact-grading-2`
- Raw SKILL.md: https://api.skillmd.com/api/skills/dingxingdi/hierarchical-rubric-based-artifact-grading-2/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Research & Search
- Author: dingxingdi (https://skillmd.com/u/dingxingdi)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/dingxingdi/hierarchical-rubric-based-artifact-grading-2

---


# Skill: hierarchical-rubric-based-artifact-grading
## 1. Capability Definition & Real Case
* **Professional Definition**: This capability is the ability to generate, inspect, and judge evaluation instances where the target output is an evaluative artifact rather than a task solution, and where the evaluator must ground its verdict in explicit evidence, criteria, or safety policy. In this capability, the Agent is not solving the original user problem; it is assessing the quality, correctness, safety, or robustness of another response, trajectory, or artifact and producing a score, preference, label, ranking, or critique.
* **Dimension Hierarchy**: Structured Evidence Evaluation->Evidence-and-Verifier-Grounded Evaluation->hierarchical-rubric-based-artifact-grading
### Real Case
**[Case 1]**
* **Initial Environment**: An evaluation environment contains a research-replication submission with source code, a reproduction script, execution logs, and generated outputs. A hierarchical rubric specifies leaf criteria such as whether a particular experiment ran, whether a result table was reproduced, and whether a module was implemented correctly. The evaluator must grade leaf requirements and aggregate them upward.
* **Real Question**: How should this artifact be graded against the rubric?
* **Real Trajectory**: Read the rubric tree, inspect the relevant files for a selected leaf node, assign binary pass or fail, and repeat for all necessary leaves. Propagate the weighted scores upward to compute a final artifact score. Record which evidence files supported each grading decision.
* **Real Answer**: The submission should receive partial credit because some code-development and execution criteria are met, but several result-match requirements remain unsatisfied.
* **Why this demonstrates the capability**: This case demonstrates artifact grading for complex outputs that cannot be judged holistically with a single prompt. The evaluator must use hierarchical decomposition, inspect different evidence types, and preserve partial-credit logic. It therefore tests rubric-aware evaluation, evidence routing, and weighted score aggregation.

## Pipeline Execution Instructions
To synthesize data for this capability, you must strictly follow a 3-phase pipeline. **Do not hallucinate steps.** Read the corresponding reference file for each phase sequentially:

1. **Phase 1: Environment Exploration**
   Read the exploration guidelines to discover raw knowledge seeds:
   `references/EXPLORATION.md`

2. **Phase 2: Trajectory Selection**
   Once Phase 1 is complete, read the selection criteria to evaluate the trajectory:
   `references/SELECTION.md`

3. **Phase 3: Data Synthesis**
   Once a trajectory passes Phase 2, read the synthesis instructions to generate the final data:
   `references/SYNTHESIS.md`