Skill Assessment

Skill assessment

Confidence correlates poorly with competence in both directions, and self-assessment is least reliable at the extremes. Useful assessment observes performance on representative work.

Method

1. Assess on realistic tasks. Work resembling what the person will actually do, since abstract puzzles measure something else (see technical-interviews).
2. Watch the process, not just the result. How someone approaches an unfamiliar problem predicts future performance better than whether they solved this one.
3. Probe the boundary. Where their knowledge stops and whether they recognise it, since knowing what you do not know is the most transferable skill.
4. Discount confidence. Fluent explanation and correct explanation are different, and confidence is often inversely related to depth in novices.
5. Use multiple signals. A single assessment is noisy, and independent observations reduce the error (see agent-ensemble-voting).
6. Assess against the level needed, not against yourself. The question is whether they can do this work, not whether they match your approach.
7. Give the assessment back. An evaluation that never reaches the person cannot help them improve (see feedback-for-learning).

Boundaries

Assessment predicts imperfectly, and interview performance correlates loosely with job performance. Assessing people has fairness and legal implications in employment contexts (see agent-people-ops-desk). Self-assessment is unreliable and needs external calibration.