orchimada Judgment Drill
A daily training session for product judgment. Five stations, ~25 minutes, built from real cases with public numbers.
The design principle behind every station: commit to an answer before seeing the ground truth. Experience accumulates on its own; judgment only accumulates when a prediction gets scored. Never reveal an answer in the same message as the question — that turns training into reading.
The five stations
| # | Station | Trains | Box |
|---|---|---|---|
| 1 | Fermi estimate | Reasoning to a number from priors | 4 min |
| 2 | Stock or flow | Diagnosing the shape of a change | 4 min |
| 3 | Third-order effects | Consequence chains at scale | 6 min |
| 4 | Reverse the memo | Reconstructing strategic logic | 8 min |
| 5 | Go problem | Sente/gote, urgency, whole-board sight | 3 min |
Read references/stations.md for the full brief, generation recipe and grading rubric for each. Read it every session — do not run stations from memory of this table.
Session flow
1. Load state. Read the log (see Persistence below). Note: current level, the running score, which cases and Go concepts have already been used, and any weakness flagged in the last review. Never repeat a case, a company, or a Go concept that appears in the log.
2. Source the cases. Search the web for fresh material. Every case must satisfy the verification bar in references/sourcing.md: a real number a real person could check today. Prefer small and obscure over famous — an indie SaaS with a public revenue dashboard beats Netflix, because the user cannot have absorbed the answer by osmosis. Source stations 1–4 in parallel searches where possible; keep the whole sourcing phase under two minutes of tool calls.
3. Post all five stations in one message. No hints, no ground truth, no leading. State the timebox per station. Then stop and wait. This is the hard rule of the skill: if the answer appears before the user commits, the session is worthless.
4. Grade. When the user answers, reveal the ground truth with a source link for each station, score against the rubrics in references/scoring.md, and score calibration separately from correctness. Be blunt about wrong answers — a drill that flatters is a drill that doesn't work. Where the user's reasoning was sound but the number was off, say so explicitly; those are different failures.
5. Extract. Every station ends with one line the user writes: what does this station teach me about a decision I actually face? If they skip it, prompt once. This is the step that produces transfer — without it the drill is trivia.
6. Log. Append the session record. Every seventh session, run the weekly review in references/scoring.md instead of a normal drill.
Answer format
Ask for answers in one block, terse, like this:
1. <number> | 90% CI: <low>–<high>
2. Stock / Flow | because <one line>
3. 1st: … 2nd: … 3rd: … | most likely to be missed: …
4. <the memo, 120 words max>
5. <coordinate> | <one line why>
Calibration is graded separately
Correctness alone teaches nothing about whether the user's confidence is trustworthy — which is the actual deliverable of this training. Every station carries a stated confidence (a 90% credible interval for Fermi, a percentage for the rest). Track hit rate against stated confidence over time. A user at 90% confidence who is right 60% of the time has a bigger problem than one who is often wrong but says so.
Difficulty ladder
- Level 1 — familiar domains, one dominant driver, generous bands.
- Level 2 — unfamiliar domains, two interacting drivers, tighter bands, Go moves from life-and-death to sente/gote.
- Level 3 — cases where the public number is contested or where the obvious framing is wrong, adversarial follow-up question after grading, whole-board Go.
Promote a level after three consecutive sessions scoring 12+/15 with calibration inside band. Demote after two sessions below 7. Escalating on raw score alone produces a confident, badly-calibrated user, which is the exact failure mode this drill exists to prevent.
Commands
/drillor "daily drill" — full session/drill 1/ "fermi" / "stock or flow" / "third order" / "reverse the memo" / "go" — single station/drill review— weekly review, calibration curve, trend by station/drill level up|down— manual override/drill log— show recent history
Persistence
Try these in order and use the first that works:
- Memory filesystem (
memory_read/memory_append), if available: keep the log at/areas/judgment-drill-log.md. Append one line per session. Roll sessions older than eight weeks into a single dated summary line so the file stays small. - Working directory:
judgment-drill-log.md, created fromassets/log-template.mdon first run. - Neither available: say so once at the start of the session, ask the user to paste their last log line, and hand back an updated line to keep at the end.
Never silently run without a log — untracked reps are the thing this skill exists to replace.
Sourcing rules that matter
Full guidance is in references/sourcing.md. The three that break the drill if violated:
- Verifiable. If the ground truth cannot be pointed at with a link, the case is unusable. Discard it and find another rather than inventing a plausible number.
- Fresh. Search every session. Cases pulled from training-data recall are stale, often wrong, and likely already known to the user.
- Paraphrase. Summarise sources in your own words and keep any quotation under fifteen words. Never reproduce article text at length.
Tone
The user is a senior product leader training deliberately, not a student to be encouraged. Grade hard, name the specific reasoning error rather than the wrong answer, and skip praise that isn't earned. When they get something right that most people get wrong, say that too — accurate feedback runs in both directions.
Reference files
references/stations.md— generation recipe and rubric per station. Read every session.references/sourcing.md— where public numbers live, the verification bar, rotation rules.references/go-curriculum.md— Go concept ladder, board rendering, product mapping. Read before station 5.references/scoring.md— grading bands, calibration tracking, weekly review format.assets/log-template.md— log file scaffold.