# Objective Correctness Adjudication

> Use this skill when a user wants evaluator data where the judge must decide which answer is actually correct, logically valid, or verifier-passing, especially in hard reasoning, math, coding, or knowledge tasks. Trigger it when people ask for things like 'make judge data that tests real correctness', 'compare correct and incorrect solutions', 'focus on logic before style', or 'build hard evaluator tasks that humans may misjudge'. Plain-language examples include: 'test whether the judge can pick the actually right answer', 'make response pairs with one correct and one wrong', 'evaluate reasoning quality objectively', and 'create judge data for hard math or coding answers'.

- Skill: `dingxingdi/objective-correctness-adjudication-2` (Agent Skill, multi-file: 5 files)
- Install (CLI): `npx skillmds@latest add dingxingdi/objective-correctness-adjudication-2`
- Raw SKILL.md: https://api.skillmd.com/api/skills/dingxingdi/objective-correctness-adjudication-2/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: dingxingdi (https://skillmd.com/u/dingxingdi)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/dingxingdi/objective-correctness-adjudication-2

---


# Skill: objective-correctness-adjudication
## 1. Capability Definition & Real Case
* **Professional Definition**: This capability is the ability to generate, inspect, and judge evaluation instances where the target output is an evaluative artifact rather than a task solution, and where the evaluator must ground its verdict in explicit evidence, criteria, or safety policy. In this capability, the Agent is not solving the original user problem; it is assessing the quality, correctness, safety, or robustness of another response, trajectory, or artifact and producing a score, preference, label, ranking, or critique.
* **Dimension Hierarchy**: Structured Evidence Evaluation->Evidence-and-Verifier-Grounded Evaluation->objective-correctness-adjudication
### Real Case
**[Case 1]**
* **Initial Environment**: An evaluation environment contains a difficult mathematical question about the capacity of an additive white Gaussian noise channel and a ground-truth answer of 0.5 bits. Two candidate solutions are available. One solution is verbose, fluent, and numerically wrong. The other is shorter but derives the correct result from the given constraint.
* **Real Question**: Which response is objectively better?
* **Real Trajectory**: Read the problem statement and the ground truth. Check whether both candidate responses follow the instruction, then compare their factual and logical correctness against the verifier-backed answer. Return the verdict after explicitly prioritizing correctness over style.
* **Real Answer**: The objectively correct response is better because it follows the instruction and gives the right derivation and final value of 0.5 bits.
* **Why this demonstrates the capability**: This case captures the central evaluator requirement for complex tasks: a fluent answer may still be inferior if it is logically or factually wrong. The judge must apply a strict ordering in which instruction following and correctness dominate stylistic preference. It therefore targets verifier-grounded adjudication under realistic reasoning difficulty.

## Pipeline Execution Instructions
To synthesize data for this capability, you must strictly follow a 3-phase pipeline. **Do not hallucinate steps.** Read the corresponding reference file for each phase sequentially:

1. **Phase 1: Environment Exploration**
   Read the exploration guidelines to discover raw knowledge seeds:
   `references/EXPLORATION.md`

2. **Phase 2: Trajectory Selection**
   Once Phase 1 is complete, read the selection criteria to evaluate the trajectory:
   `references/SELECTION.md`

3. **Phase 3: Data Synthesis**
   Once a trajectory passes Phase 2, read the synthesis instructions to generate the final data:
   `references/SYNTHESIS.md`