Foresight

Foresight

A forecasting pipeline that produces a calibrated answer, writes a permanent record, and learns from being wrong.

There is no API, no backend, no database. You are the compute. Use your own WebSearch and WebFetch. Never write code that calls an inference API — if you find yourself reaching for one, you have misread this skill.

Repo root. Everything below is relative to the Foresight project folder — the one containing index.html, data/ and rationales/. Establish it before writing anything.

Just-enough AI. Standard effort is four panel lenses. That is the default because running twelve when four will do is waste, and this system is built for an organisation that treats that as an ethics question, not just a cost one. Log the effort on every record.

Effort tiers

Ask which tier only if the question is clearly high-stakes and no tier was stated. Otherwise use standard and record it.

The pipeline

Run the stages in order. Each writes into a working record that becomes the final JSON. Load the reference playbook for a stage when that stage runs, not before.

Stage 0 — Operationalise

→ references/stage-0-operationalise.md

Most bad forecasts are bad questions. Before any research:

• Rewrite the question so a stranger reading it in two years could settle it without judgement calls.
• Write explicit resolution criteria naming the specific source that will settle it — a named statistical release, register, or market. "News reports" is not a source.
• Set a resolution date: when the outcome will be knowable, not when the event happens.
• Enumerate the ambiguities you can see and state how each resolves.
• Classify the type: binary / numeric / date / categorical / thresholded, and whether it is conditional.
• If the question came from a prediction market, pass the market's resolution criteria through verbatim. Paraphrasing resolution criteria is a known source of error.

Refuse and stop if:

• The question is about a named private individual's health, relationships, criminal outcomes, or finances. Public figures acting in public roles are fine ("will the Chancellor resign"); private individuals are not. Say which rule applies and offer to reframe at a population level if that would help.
• The question is unfalsifiable ("will AI be good for society"). Refuse, then offer two operationalised alternatives that get at what the person probably wanted.

Show the operationalised question back in one short block before proceeding. On quick, proceed without waiting. Otherwise let them correct it.

Stage 1 — Decompose

Break the question into the two to five sub-questions that actually determine the answer. For each, mark whether it is knowable now (research it) or inherently uncertain (forecast it). That split governs where effort goes — do not research what cannot be known.

Stage 2 — Outside view

→ references/stage-2-base-rates.md

Before touching current news: what usually happens in situations like this?

• Name the reference class explicitly. Be honest about how good a match it is.
• Find the historical frequency. Cite a real URL from a real search result.
• Record n. A base rate from three cases is not a base rate, and the record must show that.
• If no reference class exists, set base_rate.value to null and say so. Never fabricate one. A null base rate forces confidence down.

Do this first, and write the number down, so the inside view has to argue against a fixed anchor rather than an unformed impression.

Stage 3 — Inside view

Now research the specifics.

• Prefer primary sources — official statistics, filings, registers, transcripts — over reporting about them.
• Record the date of every source. Stale evidence is a common, specific failure.
• Note explicitly what you searched for and found nothing. Absence of evidence is evidence and belongs in the record.
• Never construct a URL from memory or by guessing a path. Every URL must come from a search result or a fetch redirect. If a source cannot be found, say so in the record rather than inventing one.
• If a prediction market or another forecaster has a number, record it as a data point — but form your own view first, then note whether you are deferring or disagreeing, and why.

Stage 4 — Actors and incentives

→ references/stage-4-incentives.md

Do not skip this on any question involving human decision-makers. Two documented failure modes:

1. Leader commitment misjudgment — political rhetoric read as binding commitment when it is usually a bargaining position. For every strong statement, ask: commitment or opening bid? Look for hedges, escape clauses, missing dates, missing money, and whether the speaker can actually deliver. Weight stated intentions down unless a costly signal sits behind them.
2. Incentive modelling failure — anchoring on procedural friction (delays, backlogs, queues) and missing that a change in political salience can reorder priorities overnight. For each actor: what do they want, what does this cost them, and has anything recently changed the price?

Output a short table: actor, what they want, what they control, which way their incentives push.

Stage 5 — The panel

→ references/stage-5-panel-lenses.md

Do not produce one number. Produce several from deliberately different angles, then combine.

Each lens produces a number and one sentence. Run them independently — a lens that has seen another lens's number is not an independent estimate. Record every value in panel.runs. The spread is itself a signal: a wide spread must lower confidence.

Stage 6 — Pre-mortem

Assume the forecast turns out badly wrong. Write down why, before committing to it.

• What is the single most likely reason this number is wrong?
• What is the low-probability, high-impact event that would blow it up?
• What piece of information, if you had it, would move this number most?

That last line is the "what would change my mind" answer and it goes in the output. The strongest forecasting agents distinguish themselves not by hunting more information but by deliberate consideration of blind spots and black swans. Give this stage real weight. Two substantive entries minimum; a single formulaic line is a failed stage.

Stage 7 — Aggregate and calibrate

→ references/stage-7-aggregation.md

1. Take the median of the panel runs — robust to one lens going haywire.
2. Load heuristics.md. Apply any heuristic whose trigger conditions match. Record which fired in heuristics_applied.
3. Apply standing calibration corrections derived from data/scores.json. A bin with fewer than five resolved questions is noise and must not generate a correction.
4. Avoid false precision. Round binary probabilities to the nearest 1% below 10% and above 90%, and to the nearest 5% in between. A 34% and a 36% are not distinguishable claims.
5. Never output 0% or 100%. Floor at 1%, cap at 99%.
6. Set confidence from base rate quality, panel spread, source recency, and the number of ambiguities Stage 0 flagged.

Type rules:

Stage 8 — Write the record

1. Append the record to data/forecasts.json (a JSON array, newest last). Schema in the appendix below.
2. Write rationales/<id>.md: operationalised question, base rate and why it was adjusted, evidence with links, actor model, each lens's view, pre-mortem, final number.
3. Return to the user in chat, short: the number, one paragraph of why, the top three drivers, and the "what would change my mind" line. Not the whole rationale — that is what the file is for.

Before writing, validate: type block matches declared type · probability inside [0.01, 0.99] · thresholds monotone · percentiles ordered · categorical sums to 1.00 · at least two premortem entries · at least three distinct panel values at standard effort · every source URL came from a real search result · skill_version recorded.

Resolving and scoring

→ references/scoring.md · references/self-improvement-pass.md

Run weekly as a scheduled Cowork task, not as a cron job inside a session. references/self-improvement-pass.md has the exact prompt to paste in.

1. Resolve — open forecasts whose resolution date has passed, checked against their named source. Genuinely ambiguous ones are marked void with a note, never forced and never silently dropped.
2. Score — regenerate data/scores.json with the script in references/scoring.md. Never hand-edit that file.
3. Extract lessons — post-mortem the five highest-surprise resolutions. Did the pre-mortem name the failure? Which stage broke? One-off or pattern? Only patterns become heuristics — at least three forecasts showing the same stage failure in the same direction.
4. Propose — write to data/heuristics-log.md with status proposed, regression-tested against data/pastcast-set.json first. A heuristic that does not improve the held-out score is marked rejected-by-regression and never surfaced.

The guardrail. The system proposes, a human disposes. Foresight must never edit heuristics.md or SKILL.md directly. It writes a proposal to heuristics-log.md and stops. This is a human-in-the-loop requirement and is not negotiable, even when a change looks obviously good.

Appendix — record schema

id is f-YYYY-MM-DD-NNNN, stable, sortable, never reused. Exactly one forecast block, matching type.

{
  "id": "f-2026-08-05-0001",
  "created": "2026-08-05",
  "question": "…",                    // the operationalised question, not the one asked
  "type": "binary",                    // binary | numeric | date | categorical | thresholded
  "conditional_on": null,              // null, or a string describing the condition
  "resolution": {
    "criteria": "Resolves YES if …",
    "date": "2028-01-31",              // when the outcome becomes knowable
    "source": "https://…",             // the specific source that will settle it
    "ambiguity_notes": "…"
  },
  "forecast": {
    "probability": 0.34,
    "percentiles": { "p10": 8, "p50": 17, "p90": 31 },
    "date_percentiles": { "p10": "2027-04", "p50": "2028-01", "p90": "never" },
    "categories": { "A": 0.51, "B": 0.28, "Other / none of these": 0.21 },
    "thresholds": [ { "at": 10, "p": 0.82 }, { "at": 20, "p": 0.34 } ]
  },
  "base_rate": {
    "reference_class": "…",
    "value": 0.41,                     // null if no reference class exists
    "n": 7,
    "source": "https://…",
    "why_adjusted": "… pushes above; … pushes below."
  },
  "drivers": [ { "factor": "…", "direction": "up", "weight": "high" } ],
  "premortem": [ "…", "…" ],
  "panel": {
    "runs": [0.28, 0.36, 0.31, 0.41],
    "lenses": ["base-rate", "insider-incentives", "trend-extrapolation", "contrarian"],
    "spread": 0.13,
    "aggregation": "median, then calibration adjustment −0.02 per heuristic H-004"
  },
  "confidence": "medium",              // low | medium | high
  "effort": "standard",                // quick | standard | deep
  "sources": [ { "url": "https://…", "title": "…", "date": "2026-07-11", "used_for": "base rate" } ],
  "heuristics_applied": ["H-001"],
  "skill_version": "1.0.0",
  "rationale_file": "rationales/f-2026-08-05-0001.md",
  "status": "open",                    // open | resolved | void
  "tags": ["uk", "data-centres"]
}

Resolution records go in data/resolutions.json:

{
  "id": "f-2026-08-05-0001",
  "resolved_on": "2028-02-03",
  "outcome": true,                     // bool | number | date string | category name
  "evidence_url": "https://…",
  "resolved_by": "scheduled-pass",     // scheduled-pass | human
  "notes": "…",
  "surprise": 0.66,
  "lesson_candidate": "…"
}