Probe Matrix

What it does

Turns "how do I test this cheaply?" into one decision: the single question you most need answered, matched to the cheapest probe that can answer it honestly. It picks from four probes — paper/sketch, Wizard-of-Oz, concierge, agent-concierge — writes down what the chosen probe will still lie about, and pre-registers the pass/fail read-out before you run. If you already picked a probe and it lies about the thing you want to learn, the skill kills it and names the honest one. Then it hands you to the matching runbook for the how-to.

The Icarus reframe

The generic fidelity ladder picks a probe by how much you want to build — paper, clickable, coded, MVP — as if the only question were cost. That is the wrong axis. Every probe is honest about some questions and lies about others, and the honesty is fixed by the highest evidence rung the probe can legitimately produce: a paper sketch tops out at a comprehension read (0.5), so it cannot tell you about trust, which is behaviour under real stakes (0.7); a Wizard-of-Oz produces real usage behaviour toward a faked backend, so it lies about feasibility and unit cost; a concierge collects real money and the real workflow but lies about scale. This skill routes by that honesty, not by cost — cheapest honest probe wins — and its sharp move is the refusal: it will not recommend a probe that lies about the very thing you are trying to learn, and it tells you which probe to run instead. A probe you cannot trust to answer the question is more expensive than the one that costs more and does.

When to use / When NOT

Use it when you have a claim you are unsure of and want to know the cheapest honest way to move it up the evidence ladder — before you build.

Do not use it for these — hand off instead:

This skill selects the probe and pre-registers the read-out. It does not run the probe, does not gate the problem, and does not score signals you already have.

Method

Fill in template.md. Six steps.

Step 1 — Extract the ONE question. One testable question about one unknown. If the fellow lists three, rank by riskiest-assumption-first (probability wrong × cost if wrong) and take the top one. Reject "test the idea" and "see if it works" — too broad to route. A question that names a solution ("test whether the mobile app works") is a solution-in-disguise: reframe to the underlying job first (problem-statement), then return.

Step 2 — Classify the learning target. Match the question to one row. This is what the question is really asking about.

Step 3 — Read the honesty contract and pick the cheapest honest probe. The contract is fixed. The top rung column is why each probe is honest or not: a probe lies about any claim that needs a rung higher than it can produce.

Map target → cheapest honest probe:

For target F, say so plainly: no cheap probe is honest about feasibility or scale. Do not fake it with a probe that will lie. Name the smallest real build track instead, or narrow the question to a sub-part a probe can answer honestly (e.g. per-task cost via agent-concierge).

Step 4 — Kill check. If the fellow already named a probe, look it up in the "lies about" column for their target. If it lies about the target, KILL it: name exactly what it lies about, name the rung the question needs vs the rung that probe tops out at, and reroute to the cheapest honest probe. This is the kill line — recommending or endorsing a probe that lies about the thing being tested is a fail.

Step 5 — Pre-register the pass/fail read-out. Write the threshold before you run, and state which evidence rung a pass moves the claim to. The read-out must land on behaviour or money. "Operators said they trust it" is verbal (0.3) and is not a valid trust read-out; "operators acted on the alert within N minutes, unprompted, on M of the last shifts" is behaviour (0.7) and is. No moving the goalposts after.

Step 6 — Route to the runbook. Hand the plan to the matching runbook skill for the how-to. probe-matrix stops at selection + read-out; it does not operate the probe.

Evidence standard

A probe exists to move one claim up the ladder — money 1.0, behaviour 0.7, artefact 0.5, verbal 0.3, opinion 0.1 — and its honesty is capped by the top rung it can produce (Step 3). The read-out must be pre-registered on behaviour or money; an opinion or verbal read-out does not count as a passed probe, however warm. The skill's own bar: it may only recommend a probe whose top rung reaches the rung the question needs. Kill line — recommending a probe that lies about the thing being tested is an auto-fail. Tag every empirical claim [Fact], [Assumption], or [Hypothesis]. If the honest answer is "no cheap probe reaches this rung" (target F), say that; do not launder a dishonest probe into a false green light.

Gotchas

• Trust is not demand. A fake-door or landing page measures click-demand and lies about trust exactly as a paper sketch does — both top out below the behaviour-under-stakes rung trust needs. When the question is "will they trust / adopt it," route to concierge or a field pilot, never a paper or landing probe, no matter how cheap.
• Cheapest ≠ honest. The router picks the cheapest honest probe, not the cheapest probe. If paper is $0 but lies about the target, its real cost is a wrong answer you will act on. Do not let the price column win Step 3.
• Wizard-of-Oz feasibility trap. A WoZ that "works" tempts the fellow to claim the thing is buildable. It is not evidence of that — the backend was human. WoZ is honest about pull, and lies about feasibility, unit cost, and scale. Keep those claims out of the read-out.
• One question, one probe. A fellow with three questions wants three probes and a month. Force the single riskiest question (Step 1). Routing a probe against a bundle produces a read-out that answers none of them cleanly.
• Don't over-escalate the mundane. When the real question is legibility, paper is the correct answer and the whole job. Do not talk the fellow up a rung into a WoZ or concierge for a question a 30-minute sketch answers honestly.
• The obvious pick still owes you a lie. On a mundane question the probe choice is uncontroversial (legibility → paper), so the "lies about" line feels redundant and gets left thin or blank — and that is exactly where it is dropped most. Even for the obvious probe, still state what it lies about. The honesty contract earns its keep when the choice is easy, not only when it is contested.

Examples

examples/sample.md — Barrier Intelligence wants to test whether operators trust the gas alert, and proposes a paper mock-up. The skill kills the paper probe (it lies about trust), reroutes to a concierge probe with a behaviour read-out on a live shift, and names what concierge itself will still lie about (scale, self-serve UX).

Related skills

Supersedes prototyping-pretotyping. That skill ranks probes on a fidelity ladder (L0 pretotype → L4 MVP) keyed to how much you build, and lists methods by assumption type — but it never says what a method cannot honestly tell you. It will happily point a paper prototype at demand or a landing page at a value prop, and its behaviour-over-opinion guardrail is a reminder, not a refusal. probe-matrix absorbs its useful bones — riskiest-assumption-first (Step 1), set-the-criteria- before-testing (Step 5), behaviour over opinion (throughout) — and beats it with the honesty contract and the kill line: it names what each probe lies about and refuses the dishonest one. On "test whether operators trust the alert," the old skill routes to a paper or clickable prototype (workflow fidelity) or a fake-door (demand), both of which lie about trust; this skill kills those and routes to concierge/field.

Composes with: evidence-ladder (defines the rungs this router matches against; run it to score the signal a probe returns), problem-quality-scorecard (gates the problem before you spend on any probe), problem-statement (reframe a solution-in-disguise before routing), and the four runbooks (paper-sketch-probe, wizard-of-oz-probe, concierge-probe, agent-concierge-probe) that this skill selects and hands off to.