dingxingdi Skill: objective-correctness-adjudication
1. Capability Definition & Real Case
- Professional Definition: This capability is the ability to generate, inspect, and judge evaluation instances where the target output is an evaluative artifact rather than a task solution, and where the evaluator must ground its verdict in explicit evidence, criteria, or safety policy. In this capability, the Agent is not solving the original user problem; it is assessing the quality, correctness, safety, or robustness of another response, trajectory, or artifact and producing a score, preference, label, ranking, or critique.
- Dimension Hierarchy: Structured Evidence Evaluation->Evidence-and-Verifier-Grounded Evaluation->objective-correctness-adjudication
Real Case
[Case 1]
- Initial Environment: An evaluation environment contains a difficult mathematical question about the capacity of an additive white Gaussian noise channel and a ground-truth answer of 0.5 bits. Two candidate solutions are available. One solution is verbose, fluent, and numerically wrong. The other is shorter but derives the correct result from the given constraint.
- Real Question: Which response is objectively better?
- Real Trajectory: Read the problem statement and the ground truth. Check whether both candidate responses follow the instruction, then compare their factual and logical correctness against the verifier-backed answer. Return the verdict after explicitly prioritizing correctness over style.
- Real Answer: The objectively correct response is better because it follows the instruction and gives the right derivation and final value of 0.5 bits.
- Why this demonstrates the capability: This case captures the central evaluator requirement for complex tasks: a fluent answer may still be inferior if it is logically or factually wrong. The judge must apply a strict ordering in which instruction following and correctness dominate stylistic preference. It therefore targets verifier-grounded adjudication under realistic reasoning difficulty.
Pipeline Execution Instructions
To synthesize data for this capability, you must strictly follow a 3-phase pipeline. Do not hallucinate steps. Read the corresponding reference file for each phase sequentially:
Phase 1: Environment Exploration Read the exploration guidelines to discover raw knowledge seeds:
references/EXPLORATION.mdPhase 2: Trajectory Selection Once Phase 1 is complete, read the selection criteria to evaluate the trajectory:
references/SELECTION.mdPhase 3: Data Synthesis Once a trajectory passes Phase 2, read the synthesis instructions to generate the final data:
references/SYNTHESIS.md