# Hiring Scorecard Builder

> Builds structured interview scorecards tied to role-specific signals to reduce bias and improve calibration. Use when opening a new role or auditing an existing interview loop for consistency.

- Skill: `skillmedev/hiring-scorecard-builder` (Agent Skill)
- Install (CLI): `npx skillmds@latest add skillmedev/hiring-scorecard-builder`
- Raw SKILL.md: https://api.skillmd.com/api/skills/skillmedev/hiring-scorecard-builder/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: SkillMedev (https://skillmd.com/u/skillmedev)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/skillmedev/hiring-scorecard-builder

---


# Hiring Scorecard Builder

Unstructured interviews have poor predictive validity and amplify the biases of whoever talks most. A scorecard forces interviewers to decide what matters before meeting the candidate and to gather evidence, not impressions.

## Step 1 - Define the Role's Top 4-6 Signals

Start from the first 90-day outcomes, not a list of desired traits. Ask: what does success look like concretely at 30, 60, and 90 days? Then work backward to the signals a candidate would have to demonstrate to make those outcomes likely. Good signals are observable in a conversation. Bad signals are adjectives ('smart', 'passionate', 'culture fit').

## Step 2 - Map Signals to Interview Slots

Each interviewer owns 1-2 signals. Never have two interviewers independently assess the same signal with no calibration plan - duplication wastes time and creates conflicting data. Assign technical depth signals to engineers, cross-functional signals to PMs or peers, and leadership/judgment signals to the hiring manager.

## Step 3 - Write the Scoring Rubric

For each signal, write what a 1 (does not meet bar), 3 (meets bar), and 5 (exceeds bar) looks like in observable terms. Example for 'navigates ambiguity': a 1 asks for requirements to be fully specified before acting; a 3 defines scope, makes assumptions explicit, and validates them; a 5 proactively creates structure where none exists and coaches others through it. The rubric must be written before interviews start.

## Step 4 - Require Evidence, Not Ratings

The scorecard submission form has two fields per signal: a numeric score AND a required evidence quote from the candidate. No evidence, no score. This prevents post-hoc rationalization and makes debrief discussions concrete.

## Step 5 - Run a Calibrated Debrief

Collect scores independently before the debrief meeting. Start by reading scores aloud before anyone speaks. Then discuss only the signals where scores diverge by 2 or more points - those are the interesting ones. The hiring manager does not share their view first; they synthesize last.

## Escape Hatches

For a high-volume pipeline, reduce to 3 signals and a 3-point scale. For a senior or staff role, add a written exercise component and score the artifact, not the person's delivery style.

## Deliverable

Produce a complete scorecard package: the 4-6 named signals each traced to a 30/60/90-day outcome, the signal-to-interviewer slot map, a written 1/3/5 rubric per signal in observable terms, the two-field submission form (score plus required evidence quote), and the debrief protocol (independent submission, scores read aloud first, hiring manager synthesizes last).

## Do NOT

- Do not write the rubric after interviews begin - a bar defined mid-loop bends toward the candidates already seen, which is exactly the post-hoc rationalization scorecards exist to prevent.
- Do not use adjective signals ('strong communicator', 'culture fit') - anything not observable in a conversation becomes a proxy for likability and similarity bias.
- Do not let interviewers share impressions before scores are submitted; early opinions anchor the group and erase the independent-signal value of a panel.
- Do not average divergent scores into a middle number - a 1 and a 5 on the same signal means the interviewers saw different things or hold different bars, and that disagreement is the data.
- Do not assign more than 2 signals to one interviewer; coverage gets shallow and the interview becomes a checklist sprint instead of an evidence-gathering conversation.
- Do not carry a scorecard between materially different roles; signals derived from one role's 90-day outcomes do not transfer to another's.

## Quality bar

- Every signal traces to a concrete 30/60/90-day outcome and is observable in an interview.
- The 1/3/5 rubric exists in writing before the first interview is scheduled.
- Every submitted score has a verbatim evidence quote attached; scores without evidence are returned, not accepted.
- The debrief discusses only divergences of 2+ points, and the hiring manager speaks last.