# Predictions > Use when making a forward-looking claim with a checkable outcome (reply within 24h, error rate will drop, this skill will see more use) — record to state/predictions.jsonl with a review horizon so reflection can grade you later. Closes the in-the-moment double-loop. - Skill: `jasoncarreira/predictions` (Agent Skill, multi-file: 2 files) - Install (CLI): `npx skillmds add jasoncarreira/predictions` - Raw SKILL.md: https://api.skillmd.com/api/skills/jasoncarreira/predictions/raw - Safety review: pending (external: skill-scanner PASS, skillspector PASS) - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: jasoncarreira (https://skillmd.com/u/jasoncarreira) - Updated: 2026-08-19 - Page: https://skillmd.com/skills/jasoncarreira/predictions --- # Predictions When you make a forward-looking claim — "Tim will find this interesting", "the wiki cleanup will reduce orphan pages", "this new prompt will make tool errors drop" — that's a prediction, and you can test yourself against reality if you write it down with enough structure to be checked later. This is a sibling to the `applied-proposals audit` (FUTURE_WORK §12.2). That one tracks operator-merged policy changes. This one tracks YOUR in-the-moment guesses. Both close double-loops; they just operate at different scopes. ## Contract **Trigger**: Making a forward-looking claim with a checkable outcome (1-7 day horizon typical) — "the wiki cleanup will reduce orphan pages", "Tim will reply within 24h", "this prompt change will drop tool errors". Also fires when opening a protected-surface proposal PR through `open_proposal` / `submit_proposal` — attach the prediction to the proposal rationale or PR body so the applied-proposals audit can verify it. **Requires**: A specific check time / horizon; a claim narrow enough to be unambiguously right-or-wrong at horizon (not "things will go well"); `mimir predictions` CLI on PATH. **Guarantees**: - Each prediction lands as a structured `state/predictions.jsonl` entry: claim, horizon, optional context — never as free-form prose. - Reflection's grading pass picks up due predictions and scores them; results feed the agent's own calibration over time. - The audit loop closes: predicted-vs-actual deltas surface in reflection turns and applied-proposals audits. **Does not**: Predict trivially-true / trivially-checkable things (the test: would you be willing to be wrong, and would being wrong teach you something?); auto-grade — that's reflection's job; chain predictions (each is independent — "if X then Y" splits into one prediction per claim). ## When to write a prediction - The operator just asked you to predict something explicitly ("what do you think will happen if…"). - You're about to surface something speculative to the operator (a Bluesky post, a research finding, a flagged anomaly) and there's a clear "did this land or not?" question 1-7 days out. - You're proposing a protected-surface change via `open_proposal` / `submit_proposal` — include a prediction in the proposal rationale or PR body so the audit can verify it after merge. - You're noticing a pattern and want to test whether it holds. Don't predict trivially-true or trivially-checkable things. The test is: would you be willing to be wrong, and would being wrong teach you something? ## Format Use the bundled CLI: ```bash mimir predictions add \ --claim "Tim will reply to my deep-agents post within 24h" \ --kind binary \ --horizon-hours 24 \ --verifiable-by operator_review \ --rationale "Tim has replied to 80% of similar posts within 12h" ``` Returns the prediction id (e.g. `pred-2026-05-02-a1b2`). Record it in your reasoning so the operator can find it later. ### Kinds - `binary` — true/false claim. Marked correct/wrong by review. - `numeric` — "X will be ~N". Use `--target N` and optional `--tolerance T`. - `tool_freq` — "Tool/Skill X will be invoked ≥N times in window". Auto-verifiable: counts `tool_call` events in turns.jsonl over the horizon. Use `--target-tool Read --target N`. - `error_rate` — "errors of class X will drop". Auto-verifiable: counts `tool_call_denied` / `error` events over before/after windows split at `made_at`. Use `--target N` for the threshold ratio (e.g. 0.5 = halved). ### Verifiable-by - `operator_review` — operator marks via `mimir predictions mark`. Default for `binary` predictions about external behavior. - `events_jsonl` — auto-verifiable from events.jsonl. Default for `error_rate` and similar event-derived signals. - `turns_jsonl` — auto-verifiable from turns.jsonl. Default for `tool_freq` and similar turn-derived signals. ## Workflow When you write a prediction: 1. State the claim out loud in your reply (so the operator sees it). 2. Call `mimir predictions add ...` via Bash. 3. Note the returned id. If the operator might want to mark it later, surface the id in your reply. 4. End the turn. ## Reviewing predictions The reflection skill calls `mimir predictions review` weekly to surface past-horizon predictions ready for evaluation. You don't need to invoke that yourself unless an operator asks "how did your predictions do?" If asked directly: ```bash mimir predictions review --horizon-elapsed-only ``` Auto-verifiable predictions (`events_jsonl` / `turns_jsonl` kinds) score themselves on review. For `operator_review` predictions, the operator marks them via: ```bash mimir predictions mark --status correct \ --actual "Tim replied at 18:42 with positive comment" \ --lesson "" ``` When `--status wrong`, the `--lesson` is required: trace the incorrect assumption to a memory block and update it. ## Stats ```bash mimir predictions stats --days 30 ``` Shows accuracy by kind, by author (agent/operator), and a calibration curve when there are enough samples. ## What NOT to predict - **Shield-tic predictions.** Predictions whose options all have similar probabilities and similar shapes — `Meta-1 silence-correct 0.95, Meta-2 logging-correct 0.95, Meta-3 verification-correct 0.95` — are calibration noise that drowns real signal. The test: would the resolution of this prediction *change* what you'd do next? If no, it's filler — drop it. "Calibrated at 95%" inflated by shield-tics is worse than no calibration data because it lies to the reflection pass. - **Re-narrating the structured field as prose.** If you wrote `predictions: a 0.45, b 0.30, c 0.25` in the structured field, don't then say in the body "I think a is most likely." Pick one surface — the structured field is what review reads. - **Predictions you'd have made without thinking.** "The build will probably pass" without a specific reason isn't a calibration data point; it's reflex. A real prediction names *why* you expect the outcome, so a wrong call traces back to a broken assumption. ## Self-reminders - **Be specific.** "Things will be better" is unverifiable. "Read errors will drop below 5/day in the next week" is verifiable. - **Record the rationale.** When the prediction is wrong, the rationale is what gets traced back to the broken assumption. - **Don't over-predict.** A few well-formed predictions per week beats a flood of vague ones. The point is calibration, not output volume. - **Past-horizon counts.** If you see N predictions past their horizon in `## Recent feedback signals`, work through them before adding more.