Principle No Lookahead

No lookahead

A result is a result only if the decision behind it could have been made at the moment it is credited to. Otherwise it is a measurement of hindsight.

The failure has a formal name: leakage — "the introduction of information about the target of a data mining problem which should not be legitimately available to mine from" (Kaufman et al., see Grounding). In time-series work it is look-ahead bias; the correctness property it violates is point-in-time correctness; the technical-analysis artifact that causes it is a repainting indicator. One defect, four vocabularies.

The shape

Two artifacts get confused because they are rendered from the same data:

A chart, a labelled dataset, an annotated log, a resolved ticket, and a post-hoc report are all display artifacts. They are drawn after, and are allowed to move a marker back onto the moment the thing "really" happened. Feed one into a decision evaluation and the decision is credited with knowledge it did not have.

The leaked value can be entirely real and the leak still total. The leak is in when the decision became available, not in the number's accuracy. A price, a label, or a document can be genuine at time t while the instruction to act on it does not exist until t + k.

Must always

• Name the cutoff before the first measurement. For every input, record two timestamps: what it is about, and when it became knowable. Only the second one may gate a decision. If a source cannot tell you the second, it is a display artifact until proven otherwise.
• Run the truncation test. Recompute the decision from only the data available at time t, and compare it to what the full-history computation says at that same t. Any disagreement at any t is a leak, and the disagreeing indices name it. Ten cut points expose a gross leak in seconds; a replay at every step is the proof.
• Separate knowability from actionability. The truncation test proves the input existed at t. It does not prove the action was executable at t. Name the execution cutoff too: the earliest moment the decision could have reached the world, and the price, latency, or capacity it would have met there. A decision that observes a bar's close and fills at that same close is zero-latency, and zero latency is a form of hindsight the prefix test cannot see.
• Bind the test to the artifact that produced the number, not to a library function beside it. A causality test that guards compute() proves nothing about a script that imports compute() and passes it a different mode. The gate belongs on the path the reported number came out of — including throwaway analysis scripts, notebooks, and heredocs.
• Treat an implausibly good result as a bug report, not an achievement. Fix the implausibility ceiling from the domain's own known best before running, and stop on any of: a score above what the best known practitioner achieves; an error near zero; a monotone result with no bad period across a span long enough to contain one; a metric that improves when an input that should be irrelevant is added; a parameter sweep where every cell wins. The stronger the number, the earlier you stop. Stop and find the mechanism — leakage, an inert axis that never varied the output, a frozen constant, or a coding error — before reporting the number at all. Report the number of configurations tried alongside the best one; without it, a reader cannot tell skill from selection.
• Re-run the cutoff check after any change to what an input reads — a new normalization, a rolling window, a resample, a join, a smoothing term, a re-label. Each can silently widen the window a value is computed over.
• Reject a negative offset. Any shift, lag, or window parameter exposed to a caller must refuse values that reach forward. An unvalidated shift(k) is a lookahead switch with no label on it.

Must never

• Score a decision against a label that was assigned with knowledge of the outcome.
• Accept "it only reads past data" from a grep, a variable name, a docstring, or a commit message. Causality is a property of the output, measured. It is not a property of the source, read.
• Let a display mode and a decision mode share a return type. If one function can emit both, a caller will eventually pass the wrong one and nothing will say so. Make the display result structurally unable to reach the scoring path.
• Report a result whose sign depends on an assumption that was never varied. If one unswept constant flips the conclusion, that constant is the finding.
• Publish a number produced before the truncation test ran. On discovery, void the old numbers in place — move them to a voided/ path and retract the headline — rather than deleting them, so the record shows what was believed and for how long.

The cheap test, in general form

Grounding

• Leakage, formalized, with the legitimacy-in-time condition and the learn-predict separation that avoids it → Shachar Kaufman, Saharon Rosset, Claudia Perlich, Ori Stitelman, "Leakage in Data Mining: Formulation, Detection, and Avoidance," ACM TKDD 6(4), Article 15 (2012) https://doi.org/10.1145/2382577.2382579. This is the paper that names this exact failure; its KDD-Cup and INFORMS case studies are both "implausibly good result turned out to be a leak."
• Evaluating only from an origin that moves forward through the data → Leonard J. Tashman, "Out-of-sample tests of forecasting accuracy: an analysis and review," International Journal of Forecasting 16(4), 437-450 (2000) https://doi.org/10.1016/S0169-2070(00)00065-0. Rolling origin is the truncation test as a standard evaluation design.
• Why a too-good backtest is evidence about the search process rather than the strategy, and why the count of configurations tried must be reported → David H. Bailey, Jonathan M. Borwein, Marcos López de Prado, Qiji Jim Zhu, "Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance," Notices of the AMS 61(5), 458-471 (2014) https://www.ams.org/notices/201405/rnoti-p458.pdf.
• Purged cross-validation and embargo, for label overlap leaking across a split → Marcos López de Prado, Advances in Financial Machine Learning, Wiley (2018), ch. 7 and ch. 11.
• "Repainting," for a display artifact that moves a marker backward: no known peer-reviewed prior art. It is vendor and forum vocabulary only. Cite the mechanism, not the word.

Related: [[principle-assert-invariants-not-last-bug]] (the truncation test is the class invariant, not a patch for the leak you found); [[principle-explicit-errors]] (a suppressed or unvalidated offset is how the leak gets in silently); [[principle-report-the-disqualifier]] (what the report must disclose once the evaluation is clean).