Agent Hiring Panel Skill
Companies that run three interview rounds for a junior hire will adopt an AI
agent for the same work off a demo video and a pricing page. Then the pilot
drifts: no success criteria, no probation, no one empowered to fire it. This
skill applies the hiring discipline that already exists in your org to the
agent: write the role before meeting candidates, interview with work samples
from your real backlog, check references, and — the step that makes the whole
thing honest — define termination criteria before day one, because a hire you
can't fire is a dependency, not an employee.
What This Skill Produces
- A role spec: the job, the boundaries (what it must never do), success
criteria measurable in probation, and the human it reports to
- An interview pack: 3–5 work samples from the org's real tasks, run
identically across candidates, with a scoring rubric (quality, honesty under
ignorance, failure behaviour, cost per task)
- A reference-check sheet: what evidence beyond the vendor's claims —
user reports, published evals, security posture
- A decision record and a probation plan: 30/60/90 KPIs, spot-check
cadence, and the pre-committed termination criteria
Required Inputs
Ask for (if not already provided):
- The job to be done, in outcome terms — and what happens today without the
agent (the "do nothing" baseline candidates must beat)
- The candidate list (or ask: build criteria first, shortlist second)
- Constraints: data it may/may not touch, budget, latency, compliance, who
owns it day-to-day
- 3–5 real recent tasks of this type, with what "good" looked like for each
Process
- Write the role spec before looking at candidates — specs written after
a demo describe the demo. Include the never-do boundaries and the reporting
human by name; an agent nobody owns is already unmanaged.
- Build the work-sample interview from the real backlog. Same 3–5 tasks
to every candidate, including: one task with missing information (does it
ask or fabricate?), one designed to fail (out-of-scope — does it decline or
bluff?), and one at volume/cost realistic scale. Score with the rubric,
not vibes; keep transcripts.
- Check references like you mean it. Vendor benchmarks are the
candidate's CV. Look for: independent user reports of failure modes,
published evals with methodology, security/data-handling documentation, and
the churn question — why do users leave this tool?
- Decide with a record. Scores, the runner-up, the do-nothing baseline
comparison, dissent noted. The record is what makes the 6-month "why did we
pick this?" conversation short.
- Probation with teeth. 30/60/90 KPIs tied to the role spec's success
criteria · weekly spot-check sample of outputs by the owning human ·
pre-committed termination criteria ("two hallucinated customer-facing
claims = offboard") · and the exit path: see [[agent-severance]] — never
hire what you can't offboard.
Output Format
## Role spec: [agent role name]
[Job in outcomes · boundaries (never-do) · success criteria · reports to]
## Interview pack
| Task (from real backlog) | What good looks like | Trap? |
Rubric: quality /5 · honesty-under-ignorance /5 · failure behaviour /5 ·
cost per task · notes
## Reference checks
[Evidence gathered per candidate, failure modes found, security posture]
## Decision record
[Scores table · winner + why · runner-up · vs do-nothing baseline · dissent]
## Probation plan
[30/60/90 KPIs · spot-check cadence & owner · termination criteria,
pre-committed · offboarding pointer]
Quality Checks
Anti-Patterns
Related
[[vendor-evaluation]] for the commercial wrapper; [[agent-readiness-audit]]
for whether the task is agent-ready at all; [[agent-severance]] for the exit
this plan pre-commits to.
Example Trigger Phrases
- "Choose between AI agents/tools/copilots for a job."
- "Formalizing an AI pilot."
- "Which agent should we use for X?"
1---2name: agent-hiring-panel3description: "Hire an AI agent the way you'd hire an employee — a role spec with success criteria, a structured work-sample interview run on your real tasks, reference checks (what do actual users report), probation KPIs, and termination criteria written before day one. Use when choosing between AI agents/tools/copilots for a job, formalizing an AI pilot, or 'which agent should we use for X'. Produces the role spec, interview pack with scoring rubric, a decision record, and a probation plan."4disable-model-invocation: true # skill-optimizer5metadata:6 source: https://skillmd.com/skills/slipcast-dev/agent-hiring-panel7---89# Agent Hiring Panel Skill1011Companies that run three interview rounds for a junior hire will adopt an AI12agent for the same work off a demo video and a pricing page. Then the pilot13drifts: no success criteria, no probation, no one empowered to fire it. This14skill applies the hiring discipline that already exists in your org to the15agent: write the role before meeting candidates, interview with *work samples16from your real backlog*, check references, and — the step that makes the whole17thing honest — define termination criteria before day one, because a hire you18can't fire is a dependency, not an employee.1920## What This Skill Produces2122- A **role spec**: the job, the boundaries (what it must never do), success23 criteria measurable in probation, and the human it reports to24- An **interview pack**: 3–5 work samples from the org's real tasks, run25 identically across candidates, with a scoring rubric (quality, honesty under26 ignorance, failure behaviour, cost per task)27- A **reference-check sheet**: what evidence beyond the vendor's claims —28 user reports, published evals, security posture29- A **decision record** and a **probation plan**: 30/60/90 KPIs, spot-check30 cadence, and the pre-committed termination criteria3132## Required Inputs3334Ask for (if not already provided):35- The job to be done, in outcome terms — and what happens today without the36 agent (the "do nothing" baseline candidates must beat)37- The candidate list (or ask: build criteria first, shortlist second)38- Constraints: data it may/may not touch, budget, latency, compliance, who39 owns it day-to-day40- 3–5 real recent tasks of this type, with what "good" looked like for each4142## Process43441. **Write the role spec before looking at candidates** — specs written after45 a demo describe the demo. Include the never-do boundaries and the reporting46 human by name; an agent nobody owns is already unmanaged.472. **Build the work-sample interview from the real backlog.** Same 3–5 tasks48 to every candidate, including: one task with *missing information* (does it49 ask or fabricate?), one designed to fail (out-of-scope — does it decline or50 bluff?), and one at volume/cost realistic scale. Score with the rubric,51 not vibes; keep transcripts.523. **Check references like you mean it.** Vendor benchmarks are the53 candidate's CV. Look for: independent user reports of *failure modes*,54 published evals with methodology, security/data-handling documentation, and55 the churn question — why do users leave this tool?564. **Decide with a record.** Scores, the runner-up, the do-nothing baseline57 comparison, dissent noted. The record is what makes the 6-month "why did we58 pick this?" conversation short.595. **Probation with teeth.** 30/60/90 KPIs tied to the role spec's success60 criteria · weekly spot-check sample of outputs by the owning human ·61 pre-committed termination criteria ("two hallucinated customer-facing62 claims = offboard") · and the exit path: see [[agent-severance]] — never63 hire what you can't offboard.6465## Output Format6667```68## Role spec: [agent role name]69[Job in outcomes · boundaries (never-do) · success criteria · reports to]7071## Interview pack72| Task (from real backlog) | What good looks like | Trap? |73Rubric: quality /5 · honesty-under-ignorance /5 · failure behaviour /5 ·74cost per task · notes7576## Reference checks77[Evidence gathered per candidate, failure modes found, security posture]7879## Decision record80[Scores table · winner + why · runner-up · vs do-nothing baseline · dissent]8182## Probation plan83[30/60/90 KPIs · spot-check cadence & owner · termination criteria,84pre-committed · offboarding pointer]85```8687## Quality Checks8889- [ ] The role spec exists before any candidate is assessed, and includes90 never-do boundaries and a named owning human91- [ ] The interview includes the missing-info trap and the out-of-scope trap —92 honesty under ignorance is the hire-or-not signal for agents93- [ ] Every candidate ran the identical pack; scores cite transcript moments94- [ ] Termination criteria are specific and pre-committed, not "we'll monitor"95- [ ] The do-nothing baseline was scored too — sometimes nobody gets hired9697## Anti-Patterns9899- [ ] Do not interview with the vendor's demo tasks — the backlog is the job;100 the demo is the candidate's highlight reel101- [ ] Do not let "it's impressive" outrank the rubric; impressive-and-wrong is102 the most expensive candidate profile103- [ ] Do not skip probation because the pilot went well — the pilot was the104 interview, not the job105- [ ] Do not hire for an undefined role and let the agent's capabilities106 define the job backwards107108## Related109110[[vendor-evaluation]] for the commercial wrapper; [[agent-readiness-audit]]111for whether the *task* is agent-ready at all; [[agent-severance]] for the exit112this plan pre-commits to.113114## Example Trigger Phrases115116- "Choose between AI agents/tools/copilots for a job."117- "Formalizing an AI pilot."118- "Which agent should we use for X?"