AlphaNova playbook
Cycle 1 cost 39 research rounds and 34 uploads to reach one positive score at
rank 19. Almost all of that was learning the rules of the game, not building
models. The rules are written down here. Cycle 2 started at round 39 and still
repeated round 1 on a different construct (section 11). The method block below
is the fix. Read it before anything else.
Repo: C:/Users/skf_s/alphanova. Read comp6_scientist_release/COMPETITION.md
and the repo CLAUDE.md before writing model code. Full ledger:
research/SCOREBOARD.md (every round, every retraction). Every number in this
skill, with its source and the experiments behind it, is in evidence.md next to
this file. Read that when you need detail; this file carries the decisions.
The method: read this first
Sections 0-11 are the reference. They say how to pass the gate and what is
dead. They do not find alpha. This block is the method. Cycle 2 ran without it
and repeated cycle 1 (section 11).
- Stage 0, before the first run. Load
/quant-ml-protocol. Open
docs/EXPERIMENT-PROTOCOL.md in the repo. Add the cycle's registry entry and
read the trial-ledger count (N = 116 files, 66 uploads at the end of cycle 2).
A run not declared there is a diagnostic, never evidence.
- The yardstick, quoted next to every number. n_independent ~207 (24,800
scored rows / 120 h). SE(IC) ~0.015, so the detectable IC floor is ~0.03,
above the cycle-1 board leader's +0.028. Sharpe noise bar 0.0127 on the
7-period confirm window; the pooled 31-period bar is registry item P0. The
expected maximum of N=116 noise draws is ~3.1 SE. Every local best so far
sits inside that ceiling. A local number inside the bar never justifies the
next run of the same kind.
- The upload bar (Keith, "B", Sep-2). Local net Sharpe >= 0.02 on all 31
periods (
runner.py --full --gauge-fix), one look, pre-registered, above the
process-null p95 (registry item P1), union-pot corr <= 0.5 to every admitted
signal including ours (research/50_c2pot_corr.py). Below the bar = no slot.
The one exception is the declared anchor, one per cycle. Unspent slots cost
nothing. A sub-bar admission costs a pot ticket for the season (F5).
- The stop rule. A construct family gets the anchor plus two
single-variable probes. No third slot unless one scores above +0.0135 on the
server (the cycle-1 Q3 cutoff, the protocol's tripwire). Ten local runs of
one construct with no new input or mechanism = stop and write the Stage 5.6
self-audit before run eleven. Cycle 2 ran 47 runner shell calls (10 loops, 134 run logs) and 30 of 32
uploads on one ridge.
- Hunt constructs and inputs, not parameters. The unexplored space is the
hunt map in section 8 (inventory of
research/families/, Sep-2). Each
candidate gets a registry id, a mechanism, a kill criterion, and one look.
- Delegate the width. One Sonnet sub-agent per registry entry, launched in
one batch. Brief: the construct spec, the hard rules (section 7), the three
gate commands, the bar, the ledger line format. It returns the family file,
the runner line, invariants 6/6, the crash screen, the union-pot max corr.
The main thread judges against the bar and writes the GATE line. Never two
agents on one file. The main thread does not hand-sweep. Zero skills loaded
after 40 tool calls is a defect, not a style.
- Stage 6 before any slot. Surface to Keith: the bar restated, every metric
with its yardstick, the trial count, the NOT-DONE table, the caution flags.
Wait for the sign-off. Then the GATE line in
research/SCOREBOARD.md, then
the upload, then verify the counter moved.
0. The data, in ten lines
Phase 1 solved this. Do not re-derive it.
- Target ≈ cross-sectionally z-scored 120-hour forward sum of the
proprietary returns, clipped ±5. Horizon scan peaks exactly at h=120 (IC
0.94-0.97). So: forecasting 5-day relative performance, re-estimated hourly.
- Six features. F1-F4 are ONE slow factor (pairwise corr 0.96-0.985, autocorr
~0.97). F5 is fast, F6 very slow and tiny. All linear ICs are ~0 by design;
textbook signals score zero here.
- The factor is the poisoned dimension. Every ablation that added factor
levels made the model worse.
- Rotation. Each period is independently rotated in asset space. Inner
products survive it; ranks and absolute values do not. A single-frame IC is
partly frame luck (the same construct read +0.0115, -0.000 and +0.0075 in three
frames). Only
runner.py --full walking across periods is honest.
- The static-tilt trap. A causal expanding mean of the target's own past rows
scores IC +0.074, the largest number in the data, and it is worthless: a static
tilt survives label shuffling, so the gate's null contains all of it. Always
de-tilt. The k>1 discovery in §2 is the inversion of this trap, not an exception.
- Panel rules: returns are proprietary (never reconstruct from outside data);
tickers are not stable across periods; period files are cumulative (never
concatenate); leading exact-zero rows are warm-up (drop them in
train()).
1. The five facts that decide everything
F1. The gate is degenerate for hand-written rules. The server retrains your
model on shuffled labels and rejects it if the shuffled model still scores well.
If train() is a no-op, the shuffled model is your signal, so it scores
identically, so the test collapses to "reject if it performs". Proof: mfb_240
(admitted) and mfb_inverse (rejected) are the same file with one sign flipped.
Ten hand-written rules died this way in cycle 1. Never submit a model-free
rule. Ever.
F2. No local number ranks candidates, except the rank of concentration.
Cycle 1 wrote this as "within-construct sweeps transfer". Cycle 2 falsified the
general form.
- Across constructs, the runner lies.
corr(local net Sharpe, official) = -0.558, measured across different families. ml_apex beat ml_overtilt 2-4x
on every local metric and scored worse, IC flipping sign (+0.0243 runner to
-0.0039 server). Never rank two different constructs by a runner number.
- Within one construct, the k sweep ordered k = 3, 7, 15, 25 correctly on all
four server points (-0.0050, -0.0019, -0.0003, +0.0003). That is four
points on one axis. On cycle 2's fit-row axes the local order was
non-evidence:
win read -0.0091 locally and scored best (+0.0049); local IC
is anti-correlated with server IC inside one construct (-0.78, section 3).
- Every delta in the section 2 table (0.0003 to 0.0053) is inside the noise bar
0.0127 (section 6). Rank order on four points is not magnitude evidence.
So: a single-variable sweep is a local diagnostic on a registered construct. It
is never evidence for a slot; the bar is (method, item 3). What transfers is the
RANK of local gauge-fixed concentration (+0.96) and of local ic_std. Nothing
local measures the sign of server IC. Every model fitted across heterogeneous
signals in cycle 1 failed to rank within one family, including the field
regression in F3.
F3. The scoring function is known. Regression over 64 scored signals from
the leaderboard API (research/33_leaderboard_study.py), leave-one-out R² 0.237:
Sharpe ≈ 0.58·IC + 0.024·concentration − 0.013·IC_std − 0.011
concentration (= ‖mean of the gauge-fixed unit vectors‖, i.e. directional
stability over time) correlates +0.477 with official Sharpe, the strongest single
correlate on the board. compression_loss is exactly IC·(1 − 1/concentration)
and carries no extra information.
Use it to size the target, never to rank two variants of one signal. It is
fitted across 64 different constructs, so it inherits the F2 problem: it predicts
otv_k3 (-0.0036) above otv_k25 (-0.0039) and the server returned -0.0050 and
+0.0003. Its IC coefficient is too large and its concentration coefficient too
small for a single family.
F4. Prize needs a POSITIVE Sharpe, then top-3 rank inside the quality set.
Cycle-2 rules (2026-09-01) dropped "statistically significant": quality-set
step 1 is now any positive Sharpe, then ranked, then |corr| ≤ 0.5 against
every higher-ranked signal and the legacy pot (the fund's background signals no
longer occupy space). Under the new rule cycle 1 closed with Q=36 (pot
saturated, top 3 paid 60/25/15, Q3 cutoff +0.0135) and otv_k25 was Q36, the
last one in. Negative Sharpe cannot win anything, at any rank. Ten free
novelty checks per cycle report legacy-pot novelty before a slot is spent;
use one on every candidate that is kin to something already admitted.
F5. Everything admitted in cycle 1 is burned. At cycle close every entry
that passed entry checks moves into the season legacy pot and keeps occupying
its neighbourhood for the rest of the season. All 20 of our admitted signals are
in there. Worse: all over-de-tilted signals correlate 0.82-0.99 with each other
regardless of features (research/36_candidate_corr.py; alt_feat shares no
inputs with ml_apex and still correlates 0.954). The whole over-de-tilt
family is one ticket and otv_k25 is now the holder (highest-ranked of the
seven, so the greedy selection keeps it and prunes the rest behind it). Cycle 2
needs a different mechanism, not different inputs.
2. Where cycle 1 finished, and what the k-curve showed
Best signal we own: otv_k25, server Sharpe +0.0003, rank 19. The only
positive score of the campaign. It is ml_overtilt with one number changed:
TILT_K 7 to 25.
The four single-variable probes that produced it, all at span 96 on f1's model
except where noted:
| file |
k |
spans |
capacity |
Sharpe |
IC |
IC_std |
conc |
| otv_k25 |
25 |
96 |
f1 |
+0.0003 |
+0.0034 |
0.1823 |
0.3110 |
| otv_k15 |
15 |
96 |
f1 |
-0.0003 |
+0.0038 |
0.1852 |
0.3078 |
| otv_minimal |
7 |
96 |
minimal |
-0.0012 |
+0.0065 |
0.1953 |
0.2963 |
| ml_overtilt |
7 |
96 |
f1 |
-0.0019 |
+0.0049 |
0.1941 |
0.2960 |
| otv_s2k7 |
7 |
240/480 |
f1 |
-0.0023 |
+0.0026 |
0.1923 |
0.2992 |
| ml_apex |
25 |
240/480 |
f1 |
-0.0031 |
-0.0039 |
0.1794 |
0.3157 |
| otv_k3 |
3 |
96 |
f1 |
-0.0050 |
+0.0067 |
0.2102 |
0.2620 |
Three readings, all load-bearing:
- Sharpe is monotone in k. IC falls while it does (+0.0067 down to +0.0034).
What rises is stability: concentration 0.2620 to 0.3110, IC_std 0.2102 down to
0.1823. k trades IC for directional stability, and the trade pays. Do not
treat IC as the thing to maximise.
- The multi-timescale blend is what killed
ml_apex, not the high k. At
fixed k=7, moving to spans 240/480 costs 0.0004; at k=25 it costs 0.0034. Slow
span blending is the poison, and it is worse the higher k goes.
- Capacity down worked, and stacks with nothing yet.
otv_minimal posts the
highest IC at its output stage (+0.0065 vs +0.0049) purely from depth 2 /
3 leaves / 25 rounds. It was never tried at k=25.
Do not build otv_minimal at k=25. Checked Sep-1: local pooled corr +0.989
with otv_k25, and the server novelty check said NOT NOVEL (no slot spent). Any
sig - k*tilt build with k>1 is a copy of the pot occupant regardless of inputs
or capacity. The over-de-tilt family is closed for season 1.
Board leaders for scale: IC +0.0281 at conc 0.337, and IC +0.0256 at conc 0.397.
Our +0.0003 made the quality set under the cycle-2 rule (Q36 of 36) but the
paid cutoff was +0.0135 (F4).
3. Kill list: measured dead, do not re-run
Each line cost real rounds in cycle 1. Re-testing any of them needs a new
argument, written down first.
Constructions
- Hand-written / model-free rules of any kind. See F1. 10/10 rejected when good.
- Re-using over-de-tilt (
sig − k·tilt, k>1) or anything correlating > 0.5 with
otv_k25. Burned by F5. The mechanism is still worth carrying; the construct
is not.
- Ensembling across model families: rank-averaging 5 families gave IC +0.0051 at
concentration 0.0843, below its own components' 0.21. Averaging did not buy
stability.
- Label engineering (rank-smoothed target): identical model, only the label
changed, runner IC +0.0058 → -0.0092. It hurt.
- Learning-to-rank (LightGBM
lambdarank): IC -0.0140. NDCG is top-weighted,
IC weights the whole cross-section uniformly. Wrong objective by construction.
- CatBoost: lost to LightGBM; ordered boosting was the worst variant tried.
- Multi-timescale output blending at unit de-tilt: rejected by the gate. At k=25
it is admitted but strictly worse than a single span (§2).
- Turnover engineering (step-hold, banding, tranching, band-pass, longer holds).
Halving turnover made the score worse (-0.0073 → -0.0103, IC -0.0131 →
-0.0237). At step 240 the whole bill is ~0.0011, so a zero-alpha signal beats
our best. Costs were never the binding constraint.
- Hyperparameter optimisation as practised in cycle 1. 8 families × 25 Optuna
trials returned selection ICs of +0.073 to +0.100 for every family, including
KNN and ElasticNet matching gradient boosting. With ~120h autocorrelation the
se of a period's mean IC is ≈0.035, so the expected max of 25 noise draws is
≈+0.07: it was fitting noise, and the numbers said so before the confirm window
did. Confirm result: mean selection IC +0.0876 → mean confirm IC -0.0206,
zero of eight positive,
corr(selection, confirm) = -0.582. It ranked the
families inversely. It also pushed every family toward more capacity, which
server data later showed is the wrong direction. If you tune, tune capacity
downward and score on a window the tuner never saw.
Diagnostics and axes
- Hour-of-day / day-of-week: IC spread 0.0064 vs per-bucket se 0.0027. Dead.
- Confidence-based book sizing: IC autocorrelation is +0.899 at lag 1 but
-0.002 at lag 120, and 120h is the rebalance horizon. The lag-1 number is
mechanical (EWM smoothing), not skill persistence. Nothing to size on.
- Feature engineering as a route to decorrelation: F5 shows it does not work.
- Local shuffle rehearsal as a ranking or blocking device: it failed in both
directions (best-margin
zoo_xgb rejected, worst-margin zoo_hgb admitted;
it failed ml_overtilt, which the server then admitted at our best rank).
Keep it only as a crash screen. Never let it block an upload.
- The gate/cost trap map (round 21): rehearsal output claiming only one
output configuration can clear the gate, and that profitable and admissible are
disjoint. The server contradicted it by admitting the most persistence-leaning
signal we ever built. Non-evidence.
- Runner
city novelty and concentration as server-equivalent LEVELS. The
runner computes novelty against a shipped city database the organiser has said
to ignore. Runner concentration is construct-dependent in scale: f1 read
0.0758 local vs 0.2265 server, ml_overtilt 0.3153 vs 0.2960. No fixed
conversion exists. But it transfers in RANK: calibration on all 20 cycle-1
admitted signals (research/41_local_vs_server.py, round 40b) gave
corr(local conc, server conc) +0.96 across trained models and +0.88 inside
the over-de-tilt family, and local conc is the strongest local correlate of
server Sharpe (+0.88 / +0.90). Local ic_std transfers the same way
(negative for Sharpe). Local IC does not transfer at all: -0.39 against
server IC across trained models, -0.78 inside one construct. The common
metric to build against is rank of local concentration and rank of local
ic_std, never local IC. Nothing local measures the sign of server IC.
- Any hand-built proxy for a server metric. A raw-position stability proxy
read 0.114 where the real gauge-fixed concentration was 0.0709, and it was on
the record as a pre-registered bar. Concentration lives in the rotating target
frame; nothing computed in raw position space tracks it.
- The dashboard's validation score. It disagrees with the test score, and the
test score is what ranks you (
mid_fac_blend: validation +0.0373, test
-0.0073). A high validation number is not good news.
- A custom evaluation harness. One was built in cycle 1, disagreed with
runner.py twice (6x underestimate, then a sign flip), and every number it
produced had to be retracted. runner.py --full --gauge-fix is the only
local harness. It is what the server runs.
Dead theories about the gate (six, all pattern-fits to fewer than 15
verdicts): temporal-op syntax, signal speed, fund-city correlation, compression
loss, a strength ceiling, in-sample Sharpe. The theory that survived came from a
designed experiment: one file, one sign flipped, opposite verdicts. Design the
discriminator; do not curve-fit the verdict list.
4. What actually works
Three ingredients, each earned from a server verdict:
A trained model. Shuffling labels genuinely changes it, so the null is
proper and the gate admits it while it performs. Every trained model we sent
was admitted except one; 5 of 6 carried positive server IC.
Small capacity. One of only two monotone relationships ever found in
server data (the other is the k-curve in §2):
| model |
capacity |
server IC |
| f1_interactions |
depth 3, 7 leaves, 40 rounds |
+0.0071 |
| zoo_lgb |
depth 6, 12 leaves, 95 rounds |
+0.0047 |
| zoo_hgb |
depth 7, 193 iters |
+0.0034 |
| zoo_et |
91 trees, depth 4 |
+0.0018 |
| zoo_rf |
145 trees, depth 5 |
+0.0004 |
| zoo_mlp |
64x32x16 |
-0.0071 |
Server IC falls as capacity rises. Optuna maximised selection-window IC
and pushed every family bigger; every bigger model scored worse. If you tune,
tune downward.
Causal de-tilt of the output, and the coefficient is the biggest single
lever found all cycle. The persistent cross-sectional component is
anti-predictive: keep it (k=0) and IC goes negative. Removing more than one
unit adds it back negated, which is predictive and persistent at once. Server
Sharpe rises monotonically from k=3 to k=25 and it moved our best score by
0.0053, more than every feature, model-class and cost experiment combined.
The construct itself is burned for season 1 (F5), but the mechanism -
deliberately amplifying an anti-predictive persistent component - is the one
idea worth carrying into a new construct.
Structural skeleton: baseline_predictor.py in this skill directory (a copy of
research/families/ml_overtilt.py with the burned k stripped back to 1). Reuse
the shape (warm-up drop, 120-row embargo, csrank helper, EWM, de-tilt,
de-mean), not the signal.
5. The opening move for a new cycle
Do these before writing any model code. None of it costs a submission slot.
STOP. Load /quant-ml-protocol and open docs/EXPERIMENT-PROTOCOL.md.
No run, no slot, no sweep before the protocol file has the cycle's registry
entry and the trial-ledger count. Then list the skills and agents that apply
and delegate the construct hunt to parallel sub-agents. Cycle 2 skipped this
step and repeated cycle 1 exactly (section 11). The method block at the top
is the plan; sections 1-10 are the reference.
Confirm the cycle's rules page. Dates, slot count, pot. Windows run
boundary to boundary on the 1st and 15th UTC.
Pull the leaderboard and the city map from the API, re-fit the field
regression (research/33_leaderboard_study.py). The coefficients move as the
board fills. This is free information and in cycle 1 it sat unused for 23
rounds while two wrong theories drove the work.
Re-extract the data if the cycle ships new periods. Data is gitignored;
source is Downloads/files.zip. Period files are cumulative, never
concatenate them.
Re-run baseline_predictor.py through runner.py --full --gauge-fix to
confirm the harness and data are wired. Expect a roughly-zero Sharpe. That is
the smoke test, not a result.
Write the cycle's pre-registration into research/SCOREBOARD.md before
the first upload: what mechanism, what the bar is, what each verdict will
discriminate.
Then spend slots only against the bar
Cycle 1 spent 14 slots learning the rules. Cycle 2 spent 20 on parameter
variants of one construct. Neither found alpha. The slot plan is the method
block, items 3 and 4:
| Slots |
Purpose |
| 1 |
Anchor. The declared construct with the largest server novelty angle. Free novelty check first. The one sub-bar exception per cycle. |
| 0-19 |
Bar-clearing constructs only. Registry id, one look, above the process-null p95, union-pot corr <= 0.5 to every admitted signal, GATE line on record. |
| <= 2 per family |
Single-variable probes on a family that has an anchor. No third slot unless one scored above +0.0135 on the server. |
Unspent slots are not a loss; they cost nothing. A sub-bar admission is a loss:
it is a permanent pot ticket (F5), and cycle 2's 20 admissions are now one
ticket (section 11). Two rules stay: never upload two variables changed at once,
and never upload a signal correlating >0.5 with one already admitted.
6. Slot economy and measurement discipline
- 20 slots per cycle. Rejections refund their slot and do not enter the
legacy pot. Unused slots expire worthless at cycle close.
- 10 free novelty checks per cycle (added cycle 2). A check reports novelty
against the legacy pot without spending a slot; a novel signal auto-proceeds
to scoring. Spend one before the anchor and before any kin of an admitted
signal. They expire with the cycle too.
- The wrong moves, in order: a sub-bar admission (permanent pot ticket), a
duplicate of an admitted signal (the same), then an unspent slot (costs only
the option). Cycle 2 ran this order backwards and spent 20 slots on one ticket.
- Every upload is an experiment with a written prediction. Before uploading,
record in the ledger what each possible verdict would prove. An upload that
cannot change what you do next is a wasted slot.
- The prediction starts with a GATE line, before the score line: "gate:
admitted | rejected, because ". Rules on record: model-free rules are
rejected when they perform (F1); slow, persistent output at k <= 1 is rejected
(cycle 1 round 5b gate law, round 32b
ml_multiscale, cycle 2 ridge_slow:
EWM-480 inputs, span 1920, k=1). A gate line that says "rejected" means the
file does not go up. Cycle 2 spent an onboard on a file whose own ledger entry
carried that prediction as branch (c) instead of as the headline.
- Change one variable at a time. Cycle 1's
ml_apex moved k and the spans
together, so its failure was unattributable and cost four follow-up probes to
decompose.
- Verdict classes seen:
OK, Overfitted submission, Failed to run,
Processing (the path admitted signals take; rejections flip straight to
Overfitted).
- Uploads are manual through the browser (
alphanova.tech, account cookedjay).
Verify the counter moved. Matching the file picker's filename label is not
proof the upload landed; that produced a false "submitted" claim in cycle 1.
Measurement discipline. Adopted mid-cycle, never falsified, keep all of it.
- The noise bar: 0.0127. Block bootstrap (120-row blocks, 2000 resamples) of
the confirm-window net Sharpe. Any local improvement under ~0.013 is luck.
Most of cycle 1's "wins" were inside it.
- Plateau, not argmax. Take a config whose grid neighbours are also good. A
lone spike is a discretisation artifact: an 800-row evaluation block fits only
3-4 rebalances, so step-aligned grids are spiky by construction.
- Two windows, counted looks. Selection = periods 001-024, confirm = 025-031.
One confirm look per decision, budgeted in advance. By round 7 the confirm
window had had 12+ looks and was spent; after that only server history was clean.
- Multiplicity honesty. Carry a "Tried" count per family in the ledger so a
best-of-13 is never written up as a single measurement.
- Overlap-adjusted significance. The 120h-overlapping label means naive
t-stats divide by ~sqrt(120). At 120-240h holds the effective sample is ~100-200
observations, so mean IC -0.013 is only t ≈ -0.87. Correlated signals are one
observation, not six.
- Pre-register. Candidate, bars, and what each verdict proves, written before
the run. It is what made cycle 1's failed predictions legible instead of
rationalisable.
7. Hard rules for the submission file
Server-side auto-rejection. Check every one before upload.
- No
assert on row order, derived values, or the target (cycle-2 rule). The
gate retrains on shuffled labels; a crashed retrain fails the whole run.
predict() must be pure inference and may see rows overlapping train().
Screen for it: .venv/Scripts/python research/40_crash_screen.py <file>
trains on real and on row-permuted labels and checks the output. Crash
screen only; it says nothing about the gate verdict.
- One
.py file. All logic inside the single Predictor subclass. Module-level
imports only. No top-level helpers, no global state.
predict() output cross-sectionally de-meaned: p.sub(p.mean(axis=1), axis=0).
- No future-looking operations anywhere. No
shift(-n), no bfill, no
center=True rolling windows, never touch the target inside predict().
Only .shift(1) is sanctioned.
- CPU only.
train() < 4 min, predict() < 60 s, RAM < 8 GB. Profile
predict() for instance-based models: zoo_knn returned "Failed to run" on
the 60 s budget.
- Never modify anything in
comp6_scientist_release/.
- Extra deps via a PEP 723 header. Pre-installed: numpy, pandas, scikit-learn,
xgboost, lightgbm, pyarrow.
- Ticker identity is not stable across periods. Cross-sectional structure only.
- Embargo the last 120 rows of the training tail. The label is a 120h forward
sum, so the tail leaks.
Mandatory pre-upload gates:
python research/11_gate_guard.py <file> # static legality, must say LEGAL
python -m pytest research/tests/test_invariants.py -k <family> # 6/6
python runner.py <file> --full --gauge-fix # from comp6_scientist_release/
The guard caught a banned centred window on a training label in cycle 1. Keep it
in the loop.
8. The open problem, and the untried mechanisms
The open problem: no local quantity predicts the server score. Not across
constructs, and for IC and Sharpe not within one either (F2). Concentration rank
and ic_std rank transfer; nothing local measures the sign of server IC. So the
local instrument is the bar plus the process null (method, items 2-3), and the
server is the only oracle of sign. Design for it: few uploads, each a different
construct, each with a written prediction. Rejections being free makes a wrong
construct cheap. A wrong admission is not free (F5).
The bar, from the field regression, at IC_std ≈ 0.19:
| concentration |
server IC needed for Sharpe > 0 |
| 0.20 |
+0.0150 |
| 0.30 |
+0.0109 |
| 0.40 |
+0.0068 |
We reach concentration 0.31 and IC +0.0034. Treat the table as a rough target,
not a ranking device (F3): the k-curve cleared zero at an IC the table says is
far too low, because stability was doing work the coefficients understate.
Explored space (inventory of research/families/, 116 files, Sep-2, by
grep; re-read a file before you cite it). Inputs: F1-F4 ranks, abs-ranks,
averages, negation, the F1-F3 and F3-F4 spreads; F5 and F6 ranks, EWM-then-rank
and rank-then-EWM at spans 240-960, squares; a 480h and a 240h F6 change;
per-feature cross-sectional dispersion and rank x dispersion
(xsec_ctx_mlp.py); rotation-covariant unit-norm rows (f2_covariant.py).
Models: ridge, Huber, ElasticNet, LightGBM regression / multiclass / lambdarank,
MLP, KNN, RF, ET, HGB, XGBoost, seed-bagged LightGBM (f1_bag.py), cross-family
rank ensemble. Output: TILT_K 0-25, output EWM 24-1920, step-hold 120, recency
weights 0.5-8, WINDOW 0.5, tranching, EWM tilt estimator, output csrank. Labels:
de-tilted, rank-smoothed, quintile classes. Training: EMBARGO 120, LOOKBACK
120-960, one global fit per train(). Every temporal op is .ewm(). Every
normalisation is cross-sectional.
The hunt map: not found in research/families/, ranked by distance from the
pot. Each item is one registry entry, one sub-agent, one look against the bar.
- Per-asset time-series normalisation. Zero
.rolling( and zero groupby
calls in 116 files. Every input is a cross-sectional rank. A per-asset
z-score or percentile against its own history is a different object.
- Feature deltas and accelerations at several horizons. One delta exists
(F6, 480h). No delta ladder, no second difference on any feature.
- F5 volatility or regime measures. F5 appears only as a rank or an EWM
rank. No rolling std, no dispersion of F5, no regime flag.
- Rolling per-asset autocorrelation or sign-persistence as an input.
Measured once as a diagnostic (IC autocorr +0.899 at lag 1), never fed in.
- Sign-agreement across F1-F4. The spreads use magnitude differences; no
concordance count exists.
- Bagging across training windows, not across seeds (
f1_bag.py bags
seeds).
- Per-period re-fitting inside one submission. Every file fits once.
- Row-wise gauge normalisation of the output (cycle-1 item, still untried).
Concentration is the norm of the mean unit vector; normalising each row
before the de-mean targets it directly.
- Feature-PC neutralisation. Project the prediction off the first PC of the
feature panel instead of its own time-mean tilt.
- Embargoed early stopping. An inner purged split chooses
N_ROUNDS.
Tried since cycle 1 and off the list: seed-bagging (f1_bag.py),
classification framing (f1_cls.py), recency-weighted training (cycle 2,
rs_rec_*, admitted at +0.0039 to +0.0043, inside the noise). A sweep of a
hunt-map item's parameters is a local diagnostic after it clears the bar, not
before.
9. File map
In this skill directory:
| File |
What it is |
evidence.md |
The full measured record: data, metrics, every failed lane with numbers, the instruments that lied, the six dead gate theories |
baseline_predictor.py |
Structural skeleton to start from, k reset to 1 |
In the repo:
| Path |
What it is |
comp6_scientist_release/COMPETITION.md |
The law. Gate mechanics at lines ~215-240, legacy pot ~23-38, quality set ~60-92 |
comp6_scientist_release/runner.py |
The only trustworthy local harness |
comp6_scientist_release/city_tools.py:105 |
concentration definition |
research/SCOREBOARD.md |
Append-only ledger, 39 rounds, every retraction |
research/11_gate_guard.py |
Static legality check, mandatory |
research/tests/test_invariants.py |
Per-family invariant suite |
research/04_shuffle_gate.py |
Shuffle rehearsal. Crash screen only, never a blocker |
research/33_leaderboard_study.py |
Field regression over the scored board |
research/36_candidate_corr.py |
Correlation matrix, run before spending a slot |
research/37_untried_axes.py |
Diagnoses an axis before you build on it |
research/40_crash_screen.py |
Trains on real and permuted labels; crash screen before upload, says nothing about the verdict |
research/41_local_vs_server.py |
Local-vs-server calibration on the 20 cycle-1 admissions (conc rank +0.96) |
research/50_c2pot_corr.py |
Union-pot screen (cycle-1 pot + our cycle-2 admissions); merges subset runs by family |
docs/EXPERIMENT-PROTOCOL.md |
Governs every run: sample-size math, the bar, registry P0-P3, NOT-DONE table, tripwire |
research/families/rs_s96_win.py |
Cycle-2 best: +0.0049, rank 1 on day 1. Pot-blocks every ridge-on-F5/F6 variant |
research/families/xsec_ctx_mlp.py |
The only creative input set (rank x dispersion, 22 columns); never uploaded; base of the cycle-3 anchors |
research/families/otv_k25.py |
Cycle-1 best: +0.0003, rank 19. Burned by the pot |
research/families/otv_minimal.py |
Highest IC at its output stage (+0.0065) via minimal capacity |
research/families/f1_interactions.py |
Best raw server IC we ever posted (+0.0071) |
research/families/otv_recency.py |
Recency weighting; verdicted in cycle 2 as rs_rec_*, inside the noise |
docs/plans/2026-08-17-cycle2-design.md |
Mostly stale: its levers 1 and 4 were later falsified |
10. Why cycle 1 took 39 rounds
Written so it does not repeat. One root cause, six expressions.
Root cause: we optimised what we could measure instead of measuring what we
were scored on.
We submitted before understanding the gate. Half the uploads were
hand-written rules. F1 was derivable from COMPETITION.md on day 1 without
spending a single slot.
We trusted local metrics for 20 rounds. corr(local, official) = -0.558 was
computable after the first four verdicts and was not computed until round 17.
We did not read the field until round 23. The leaderboard API hands you 64
scored signals with IC, concentration, IC_std and Sharpe. One regression gives
the scoring function, free, on day 1. It falsified two live theories the hour
it ran.
We fitted six theories to the verdict list instead of designing experiments
against it. Every curve-fit died. The designed one-variable experiment solved
it in a single upload.
We built a second harness alongside the official runner. It disagreed twice
and every number it produced was retracted.
We theorised where we should have swept. The campaign's only positive
score came from four cheap probes that each changed one number, fired on the
last day. Thirty rounds of theory produced -0.0019; the probes produced
+0.0003 and corrected a wrong attribution at the same time. A parameter sweep
against the real scorer beats a model of the real scorer. Cycle 2 then
inverted the error: it swept where it should have hunted, 30 of 32 uploads
on one construct (section 11). The sweep is a local diagnostic on a
registered construct. It never plans the slots; the bar does.
The corresponding habits: derive the gate before uploading; measure the
local-versus-official correlation as soon as four verdicts exist; fit the field
regression on day 1 and re-fit as the board fills; spend early local runs,
never slots, on a single-variable sweep of the most suspicious parameter; keep
exactly one harness, the official one.
11. Cycle 2 record (Sep 1-2, 2026): what worked, what did not
Outcome. 32 uploads, 20 admitted, 12 rejected, all 20 slots spent by Sep-2
01:00. Best: rs_s96_win +0.0049 (rank 1, the sole Q1 star on day 1, server IC
-0.0001). The cycle-1 Q3 cutoff was +0.0135. Nothing we built is above the noise,
locally or on the server. Full detail: research/SCOREBOARD.md lines 1990-2300.
Worked (server-verified).
- A trained ridge on two slow inputs (F5, F6 EWM-480 ranks), output EWM span 96,
unit de-tilt: admitted every time at spans 24-144 and lookbacks 240-480. The
gate passes it. It scores about zero.
- The fit-row lever: the same ridge fitted on recent rows only (recency weights,
or the newest half) sat above every plain fit, 7/7 vs 0/13, and moved the score
from +0.0006 to +0.0049. Still inside the noise bar.
- Local gauge-fixed concentration rank transfers to the server (+0.96). No other
local number does.
- Pre-registered GATE lines, one variable per file: 13/13 correct on admission by
the end. The gate is understood; every verdict was attributable.
Did not work.
- Local Sharpe and IC as a ranking device, again. Local IC is anti-correlated
with server IC inside one construct (-0.78);
win read -0.0091 locally and
scored best. On fit-row levers the local order is non-evidence.
- Gate-out at k=1: output span 192+, input lookback 960, any nonlinear term of
the ranked inputs (product, squares), LightGBM on slow inputs, Huber loss. The
one-input theory was falsified (
f6 admitted).
- Concentration alone does not pay:
rts had the board-high conc 0.2186 at
-0.0029.
- Sweeping one construct. 30 of 32 uploads were parameter variants of one ridge.
All 20 admissions are now ONE pot ticket (0.75-1.0 among themselves) and every
held build is pot-blocked.
- No new features in cycle 2. Zero. The xsec context set (cycle 1) is the only
creative input set built, and it was never uploaded.
Self-audit (quant-ml-protocol Stage 5.6), written Sep-2.
- Data: clean. Contemporaneous features, warm-up dropped, 120-row embargo.
- Statistics: the sample-size math was never written. n_indep ~207 gives
SE(IC) ~0.015, so no achievable IC is locally detectable; the Sharpe bar
0.0127 was known and every "win" sat inside it. Trial count: 116 family files,
66 uploads. The local maxima are noise maxima.
- Code: one defect, fixed (
50_c2pot_corr.py overwrote its table on subset runs).
- Process: no protocol file, no trial ledger, no process null, and the
quant-ml-protocol skill was not used. Slots went to sweeps of a construct known
to sit at zero. Fixed Sep-2:
docs/EXPERIMENT-PROTOCOL.md v1 in the repo.
Cycle-3 rule (Keith, "B", Sep-2). No slot below local net Sharpe 0.02 on all
31 periods, one look, pre-registered, above the process-null p95. Hunt new
constructs and new inputs, not parameters. The protocol file governs.
1---2name: alphanova3description: Playbook for the AlphaNova quant competition (cycles 3-5). Use for AlphaNova repo work, Predictor submissions, or the overfitting gate.4---56# AlphaNova playbook78Cycle 1 cost 39 research rounds and 34 uploads to reach one positive score at9rank 19. Almost all of that was learning the rules of the game, not building10models. The rules are written down here. Cycle 2 started at round 39 and still11repeated round 1 on a different construct (section 11). The method block below12is the fix. Read it before anything else.1314**Repo:** `C:/Users/skf_s/alphanova`. Read `comp6_scientist_release/COMPETITION.md`15and the repo `CLAUDE.md` before writing model code. Full ledger:16`research/SCOREBOARD.md` (every round, every retraction). **Every number in this17skill, with its source and the experiments behind it, is in `evidence.md` next to18this file.** Read that when you need detail; this file carries the decisions.1920## The method: read this first2122Sections 0-11 are the reference. They say how to pass the gate and what is23dead. They do not find alpha. This block is the method. Cycle 2 ran without it24and repeated cycle 1 (section 11).25261. **Stage 0, before the first run.** Load `/quant-ml-protocol`. Open27 `docs/EXPERIMENT-PROTOCOL.md` in the repo. Add the cycle's registry entry and28 read the trial-ledger count (N = 116 files, 66 uploads at the end of cycle 2).29 A run not declared there is a diagnostic, never evidence.302. **The yardstick, quoted next to every number.** n_independent ~207 (24,80031 scored rows / 120 h). SE(IC) ~0.015, so the detectable IC floor is ~0.03,32 above the cycle-1 board leader's +0.028. Sharpe noise bar 0.0127 on the33 7-period confirm window; the pooled 31-period bar is registry item P0. The34 expected maximum of N=116 noise draws is ~3.1 SE. Every local best so far35 sits inside that ceiling. A local number inside the bar never justifies the36 next run of the same kind.373. **The upload bar (Keith, "B", Sep-2).** Local net Sharpe >= 0.02 on all 3138 periods (`runner.py --full --gauge-fix`), one look, pre-registered, above the39 process-null p95 (registry item P1), union-pot corr <= 0.5 to every admitted40 signal including ours (`research/50_c2pot_corr.py`). Below the bar = no slot.41 The one exception is the declared anchor, one per cycle. Unspent slots cost42 nothing. A sub-bar admission costs a pot ticket for the season (F5).434. **The stop rule.** A construct family gets the anchor plus two44 single-variable probes. No third slot unless one scores above +0.0135 on the45 server (the cycle-1 Q3 cutoff, the protocol's tripwire). Ten local runs of46 one construct with no new input or mechanism = stop and write the Stage 5.647 self-audit before run eleven. Cycle 2 ran 47 runner shell calls (10 loops, 134 run logs) and 30 of 3248 uploads on one ridge.495. **Hunt constructs and inputs, not parameters.** The unexplored space is the50 hunt map in section 8 (inventory of `research/families/`, Sep-2). Each51 candidate gets a registry id, a mechanism, a kill criterion, and one look.526. **Delegate the width.** One Sonnet sub-agent per registry entry, launched in53 one batch. Brief: the construct spec, the hard rules (section 7), the three54 gate commands, the bar, the ledger line format. It returns the family file,55 the runner line, invariants 6/6, the crash screen, the union-pot max corr.56 The main thread judges against the bar and writes the GATE line. Never two57 agents on one file. The main thread does not hand-sweep. Zero skills loaded58 after 40 tool calls is a defect, not a style.597. **Stage 6 before any slot.** Surface to Keith: the bar restated, every metric60 with its yardstick, the trial count, the NOT-DONE table, the caution flags.61 Wait for the sign-off. Then the GATE line in `research/SCOREBOARD.md`, then62 the upload, then verify the counter moved.6364---6566## 0. The data, in ten lines6768Phase 1 solved this. Do not re-derive it.6970- **Target** ≈ cross-sectionally z-scored **120-hour forward sum** of the71 proprietary returns, clipped ±5. Horizon scan peaks exactly at h=120 (IC72 0.94-0.97). So: forecasting 5-day relative performance, re-estimated hourly.73- **Six features.** F1-F4 are ONE slow factor (pairwise corr 0.96-0.985, autocorr74 ~0.97). F5 is fast, F6 very slow and tiny. **All linear ICs are ~0** by design;75 textbook signals score zero here.76- **The factor is the poisoned dimension.** Every ablation that added factor77 levels made the model worse.78- **Rotation.** Each period is independently rotated in asset space. Inner79 products survive it; **ranks and absolute values do not**. A single-frame IC is80 partly frame luck (the same construct read +0.0115, -0.000 and +0.0075 in three81 frames). Only `runner.py --full` walking across periods is honest.82- **The static-tilt trap.** A causal expanding mean of the target's own past rows83 scores IC +0.074, the largest number in the data, and it is worthless: a static84 tilt survives label shuffling, so the gate's null contains all of it. Always85 de-tilt. The k>1 discovery in §2 is the inversion of this trap, not an exception.86- **Panel rules:** returns are proprietary (never reconstruct from outside data);87 tickers are not stable across periods; period files are cumulative (never88 concatenate); leading exact-zero rows are warm-up (drop them in `train()`).8990---9192## 1. The five facts that decide everything9394**F1. The gate is degenerate for hand-written rules.** The server retrains your95model on shuffled labels and rejects it if the shuffled model still scores well.96If `train()` is a no-op, the shuffled model *is* your signal, so it scores97identically, so the test collapses to "reject if it performs". Proof: `mfb_240`98(admitted) and `mfb_inverse` (rejected) are the same file with one sign flipped.99Ten hand-written rules died this way in cycle 1. **Never submit a model-free100rule. Ever.**101102**F2. No local number ranks candidates, except the rank of concentration.**103Cycle 1 wrote this as "within-construct sweeps transfer". Cycle 2 falsified the104general form.105106- *Across constructs*, the runner lies. `corr(local net Sharpe, official) =107 -0.558`, measured across different families. `ml_apex` beat `ml_overtilt` 2-4x108 on every local metric and scored worse, IC flipping sign (+0.0243 runner to109 -0.0039 server). Never rank two different constructs by a runner number.110- *Within one construct*, the k sweep ordered k = 3, 7, 15, 25 correctly on all111 four server points (-0.0050, -0.0019, -0.0003, **+0.0003**). That is four112 points on one axis. On cycle 2's fit-row axes the local order was113 non-evidence: `win` read -0.0091 locally and scored best (+0.0049); local IC114 is anti-correlated with server IC inside one construct (-0.78, section 3).115- Every delta in the section 2 table (0.0003 to 0.0053) is inside the noise bar116 0.0127 (section 6). Rank order on four points is not magnitude evidence.117118So: a single-variable sweep is a local diagnostic on a registered construct. It119is never evidence for a slot; the bar is (method, item 3). What transfers is the120RANK of local gauge-fixed concentration (+0.96) and of local ic_std. Nothing121local measures the sign of server IC. Every model fitted *across* heterogeneous122signals in cycle 1 failed to rank *within* one family, including the field123regression in F3.124125**F3. The scoring function is known.** Regression over 64 scored signals from126the leaderboard API (`research/33_leaderboard_study.py`), leave-one-out R² 0.237:127128```129Sharpe ≈ 0.58·IC + 0.024·concentration − 0.013·IC_std − 0.011130```131132`concentration` (= ‖mean of the gauge-fixed unit vectors‖, i.e. directional133stability over time) correlates +0.477 with official Sharpe, the strongest single134correlate on the board. `compression_loss` is exactly `IC·(1 − 1/concentration)`135and carries no extra information.136137**Use it to size the target, never to rank two variants of one signal.** It is138fitted across 64 different constructs, so it inherits the F2 problem: it predicts139`otv_k3` (-0.0036) above `otv_k25` (-0.0039) and the server returned -0.0050 and140+0.0003. Its IC coefficient is too large and its concentration coefficient too141small for a single family.142143**F4. Prize needs a POSITIVE Sharpe, then top-3 rank inside the quality set.**144Cycle-2 rules (2026-09-01) dropped "statistically significant": quality-set145step 1 is now any positive Sharpe, then ranked, then `|corr| ≤ 0.5` against146every higher-ranked signal and the legacy pot (the fund's background signals no147longer occupy space). Under the new rule cycle 1 closed with Q=36 (pot148saturated, top 3 paid 60/25/15, Q3 cutoff +0.0135) and `otv_k25` was Q36, the149last one in. Negative Sharpe cannot win anything, at any rank. **Ten free150novelty checks per cycle** report legacy-pot novelty before a slot is spent;151use one on every candidate that is kin to something already admitted.152153**F5. Everything admitted in cycle 1 is burned.** At cycle close every entry154that passed entry checks moves into the season legacy pot and keeps occupying155its neighbourhood for the rest of the season. All 20 of our admitted signals are156in there. Worse: all over-de-tilted signals correlate 0.82-0.99 with each other157*regardless of features* (`research/36_candidate_corr.py`; `alt_feat` shares no158inputs with `ml_apex` and still correlates 0.954). **The whole over-de-tilt159family is one ticket and `otv_k25` is now the holder** (highest-ranked of the160seven, so the greedy selection keeps it and prunes the rest behind it). Cycle 2161needs a different mechanism, not different inputs.162163---164165## 2. Where cycle 1 finished, and what the k-curve showed166167Best signal we own: **`otv_k25`**, server Sharpe **+0.0003**, rank 19. The only168positive score of the campaign. It is `ml_overtilt` with one number changed:169`TILT_K` 7 to 25.170171The four single-variable probes that produced it, all at span 96 on f1's model172except where noted:173174| file | k | spans | capacity | Sharpe | IC | IC_std | conc |175|---|---|---|---|---|---|---|---|176| **otv_k25** | 25 | 96 | f1 | **+0.0003** | +0.0034 | 0.1823 | 0.3110 |177| otv_k15 | 15 | 96 | f1 | -0.0003 | +0.0038 | 0.1852 | 0.3078 |178| otv_minimal | 7 | 96 | **minimal** | -0.0012 | **+0.0065** | 0.1953 | 0.2963 |179| ml_overtilt | 7 | 96 | f1 | -0.0019 | +0.0049 | 0.1941 | 0.2960 |180| otv_s2k7 | 7 | 240/480 | f1 | -0.0023 | +0.0026 | 0.1923 | 0.2992 |181| ml_apex | 25 | 240/480 | f1 | -0.0031 | -0.0039 | 0.1794 | 0.3157 |182| otv_k3 | 3 | 96 | f1 | -0.0050 | +0.0067 | 0.2102 | 0.2620 |183184Three readings, all load-bearing:1851861. **Sharpe is monotone in k. IC falls while it does** (+0.0067 down to +0.0034).187 What rises is stability: concentration 0.2620 to 0.3110, IC_std 0.2102 down to188 0.1823. **k trades IC for directional stability, and the trade pays.** Do not189 treat IC as the thing to maximise.1902. **The multi-timescale blend is what killed `ml_apex`, not the high k.** At191 fixed k=7, moving to spans 240/480 costs 0.0004; at k=25 it costs 0.0034. Slow192 span blending is the poison, and it is worse the higher k goes.1933. **Capacity down worked, and stacks with nothing yet.** `otv_minimal` posts the194 highest IC at its output stage (+0.0065 vs +0.0049) purely from depth 2 /195 3 leaves / 25 rounds. It was never tried at k=25.196197**Do not build `otv_minimal` at k=25.** Checked Sep-1: local pooled corr +0.989198with `otv_k25`, and the server novelty check said NOT NOVEL (no slot spent). Any199`sig - k*tilt` build with k>1 is a copy of the pot occupant regardless of inputs200or capacity. The over-de-tilt family is closed for season 1.201202Board leaders for scale: IC +0.0281 at conc 0.337, and IC +0.0256 at conc 0.397.203Our +0.0003 made the quality set under the cycle-2 rule (Q36 of 36) but the204paid cutoff was +0.0135 (F4).205206---207208## 3. Kill list: measured dead, do not re-run209210Each line cost real rounds in cycle 1. Re-testing any of them needs a new211argument, written down first.212213**Constructions**214- Hand-written / model-free rules of any kind. See F1. 10/10 rejected when good.215- Re-using over-de-tilt (`sig − k·tilt`, k>1) or anything correlating > 0.5 with216 `otv_k25`. Burned by F5. The *mechanism* is still worth carrying; the construct217 is not.218- Ensembling across model families: rank-averaging 5 families gave IC +0.0051 at219 concentration **0.0843**, below its own components' 0.21. Averaging did not buy220 stability.221- Label engineering (rank-smoothed target): identical model, only the label222 changed, runner IC +0.0058 → **-0.0092**. It hurt.223- Learning-to-rank (LightGBM `lambdarank`): IC **-0.0140**. NDCG is top-weighted,224 IC weights the whole cross-section uniformly. Wrong objective by construction.225- CatBoost: lost to LightGBM; ordered boosting was the worst variant tried.226- Multi-timescale output blending at unit de-tilt: rejected by the gate. At k=25227 it is admitted but strictly worse than a single span (§2).228- Turnover engineering (step-hold, banding, tranching, band-pass, longer holds).229 Halving turnover made the score **worse** (-0.0073 → -0.0103, IC -0.0131 →230 -0.0237). At step 240 the whole bill is ~0.0011, so a zero-alpha signal beats231 our best. **Costs were never the binding constraint.**232- **Hyperparameter optimisation as practised in cycle 1.** 8 families × 25 Optuna233 trials returned selection ICs of +0.073 to +0.100 for *every* family, including234 KNN and ElasticNet matching gradient boosting. With ~120h autocorrelation the235 se of a period's mean IC is ≈0.035, so the expected max of 25 noise draws is236 ≈+0.07: it was fitting noise, and the numbers said so before the confirm window237 did. Confirm result: mean selection IC +0.0876 → **mean confirm IC -0.0206,238 zero of eight positive**, `corr(selection, confirm) = -0.582`. It ranked the239 families *inversely*. It also pushed every family toward more capacity, which240 server data later showed is the wrong direction. **If you tune, tune capacity241 downward and score on a window the tuner never saw.**242243**Diagnostics and axes**244- Hour-of-day / day-of-week: IC spread 0.0064 vs per-bucket se 0.0027. Dead.245- Confidence-based book sizing: IC autocorrelation is +0.899 at lag 1 but246 **-0.002 at lag 120**, and 120h is the rebalance horizon. The lag-1 number is247 mechanical (EWM smoothing), not skill persistence. Nothing to size on.248- Feature engineering as a route to decorrelation: F5 shows it does not work.249- Local shuffle rehearsal as a ranking or blocking device: it failed in both250 directions (best-margin `zoo_xgb` rejected, worst-margin `zoo_hgb` admitted;251 it failed `ml_overtilt`, which the server then admitted at our best rank).252 Keep it only as a crash screen. Never let it block an upload.253- The **gate/cost trap map** (round 21): rehearsal output claiming only one254 output configuration can clear the gate, and that profitable and admissible are255 disjoint. The server contradicted it by admitting the most persistence-leaning256 signal we ever built. Non-evidence.257- Runner `city novelty` and `concentration` as server-equivalent LEVELS. The258 runner computes novelty against a shipped city database the organiser has said259 to ignore. Runner concentration is construct-dependent in scale: `f1` read260 0.0758 local vs 0.2265 server, `ml_overtilt` 0.3153 vs 0.2960. No fixed261 conversion exists. But it transfers in RANK: calibration on all 20 cycle-1262 admitted signals (`research/41_local_vs_server.py`, round 40b) gave263 corr(local conc, server conc) +0.96 across trained models and +0.88 inside264 the over-de-tilt family, and local conc is the strongest local correlate of265 server Sharpe (+0.88 / +0.90). Local `ic_std` transfers the same way266 (negative for Sharpe). **Local IC does not transfer at all: -0.39 against267 server IC across trained models, -0.78 inside one construct.** The common268 metric to build against is rank of local concentration and rank of local269 ic_std, never local IC. Nothing local measures the sign of server IC.270- **Any hand-built proxy for a server metric.** A raw-position stability proxy271 read 0.114 where the real gauge-fixed concentration was 0.0709, and it was on272 the record as a pre-registered bar. Concentration lives in the rotating target273 frame; nothing computed in raw position space tracks it.274- The dashboard's **validation score**. It disagrees with the test score, and the275 test score is what ranks you (`mid_fac_blend`: validation +0.0373, test276 -0.0073). A high validation number is not good news.277- A custom evaluation harness. One was built in cycle 1, disagreed with278 `runner.py` twice (6x underestimate, then a sign flip), and every number it279 produced had to be retracted. **`runner.py --full --gauge-fix` is the only280 local harness. It is what the server runs.**281282**Dead theories about the gate** (six, all pattern-fits to fewer than 15283verdicts): temporal-op syntax, signal speed, fund-city correlation, compression284loss, a strength ceiling, in-sample Sharpe. The theory that survived came from a285*designed* experiment: one file, one sign flipped, opposite verdicts. Design the286discriminator; do not curve-fit the verdict list.287288---289290## 4. What actually works291292Three ingredients, each earned from a server verdict:2932941. **A trained model.** Shuffling labels genuinely changes it, so the null is295 proper and the gate admits it while it performs. Every trained model we sent296 was admitted except one; 5 of 6 carried positive server IC.2972. **Small capacity.** One of only two monotone relationships ever found in298 server data (the other is the k-curve in §2):299300 | model | capacity | server IC |301 |---|---|---|302 | f1_interactions | depth 3, 7 leaves, 40 rounds | **+0.0071** |303 | zoo_lgb | depth 6, 12 leaves, 95 rounds | +0.0047 |304 | zoo_hgb | depth 7, 193 iters | +0.0034 |305 | zoo_et | 91 trees, depth 4 | +0.0018 |306 | zoo_rf | 145 trees, depth 5 | +0.0004 |307 | zoo_mlp | 64x32x16 | -0.0071 |308309 **Server IC falls as capacity rises.** Optuna maximised selection-window IC310 and pushed every family bigger; every bigger model scored worse. If you tune,311 tune *downward*.3123. **Causal de-tilt of the output, and the coefficient is the biggest single313 lever found all cycle.** The persistent cross-sectional component is314 anti-predictive: keep it (k=0) and IC goes negative. Removing *more* than one315 unit adds it back negated, which is predictive and persistent at once. Server316 Sharpe rises monotonically from k=3 to k=25 and it moved our best score by317 0.0053, more than every feature, model-class and cost experiment combined.318 The construct itself is burned for season 1 (F5), but the *mechanism* -319 deliberately amplifying an anti-predictive persistent component - is the one320 idea worth carrying into a new construct.321322Structural skeleton: `baseline_predictor.py` in this skill directory (a copy of323`research/families/ml_overtilt.py` with the burned k stripped back to 1). Reuse324the *shape* (warm-up drop, 120-row embargo, csrank helper, EWM, de-tilt,325de-mean), not the signal.326327---328329## 5. The opening move for a new cycle330331Do these before writing any model code. None of it costs a submission slot.3323330. **STOP. Load `/quant-ml-protocol` and open `docs/EXPERIMENT-PROTOCOL.md`.**334 No run, no slot, no sweep before the protocol file has the cycle's registry335 entry and the trial-ledger count. Then list the skills and agents that apply336 and delegate the construct hunt to parallel sub-agents. Cycle 2 skipped this337 step and repeated cycle 1 exactly (section 11). The method block at the top338 is the plan; sections 1-10 are the reference.3393401. **Confirm the cycle's rules page.** Dates, slot count, pot. Windows run341 boundary to boundary on the 1st and 15th UTC.3422. **Pull the leaderboard and the city map from the API**, re-fit the field343 regression (`research/33_leaderboard_study.py`). The coefficients move as the344 board fills. This is free information and in cycle 1 it sat unused for 23345 rounds while two wrong theories drove the work.3463. **Re-extract the data** if the cycle ships new periods. Data is gitignored;347 source is `Downloads/files.zip`. Period files are cumulative, never348 concatenate them.3494. **Re-run `baseline_predictor.py` through `runner.py --full --gauge-fix`** to350 confirm the harness and data are wired. Expect a roughly-zero Sharpe. That is351 the smoke test, not a result.3525. **Write the cycle's pre-registration into `research/SCOREBOARD.md`** before353 the first upload: what mechanism, what the bar is, what each verdict will354 discriminate.355356### Then spend slots only against the bar357358Cycle 1 spent 14 slots learning the rules. Cycle 2 spent 20 on parameter359variants of one construct. Neither found alpha. The slot plan is the method360block, items 3 and 4:361362| Slots | Purpose |363|---|---|364| 1 | **Anchor.** The declared construct with the largest server novelty angle. Free novelty check first. The one sub-bar exception per cycle. |365| 0-19 | **Bar-clearing constructs only.** Registry id, one look, above the process-null p95, union-pot corr <= 0.5 to every admitted signal, GATE line on record. |366| <= 2 per family | **Single-variable probes** on a family that has an anchor. No third slot unless one scored above +0.0135 on the server. |367368Unspent slots are not a loss; they cost nothing. A sub-bar admission is a loss:369it is a permanent pot ticket (F5), and cycle 2's 20 admissions are now one370ticket (section 11). Two rules stay: never upload two variables changed at once,371and never upload a signal correlating >0.5 with one already admitted.372373---374375## 6. Slot economy and measurement discipline376377- 20 slots per cycle. **Rejections refund their slot** and do not enter the378 legacy pot. Unused slots expire worthless at cycle close.379- **10 free novelty checks per cycle** (added cycle 2). A check reports novelty380 against the legacy pot without spending a slot; a novel signal auto-proceeds381 to scoring. Spend one before the anchor and before any kin of an admitted382 signal. They expire with the cycle too.383- The wrong moves, in order: a sub-bar admission (permanent pot ticket), a384 duplicate of an admitted signal (the same), then an unspent slot (costs only385 the option). Cycle 2 ran this order backwards and spent 20 slots on one ticket.386- **Every upload is an experiment with a written prediction.** Before uploading,387 record in the ledger what each possible verdict would prove. An upload that388 cannot change what you do next is a wasted slot.389- **The prediction starts with a GATE line, before the score line:** "gate:390 admitted | rejected, because <rule>". Rules on record: model-free rules are391 rejected when they perform (F1); slow, persistent output at k <= 1 is rejected392 (cycle 1 round 5b gate law, round 32b `ml_multiscale`, cycle 2 `ridge_slow`:393 EWM-480 inputs, span 1920, k=1). A gate line that says "rejected" means the394 file does not go up. Cycle 2 spent an onboard on a file whose own ledger entry395 carried that prediction as branch (c) instead of as the headline.396- Change one variable at a time. Cycle 1's `ml_apex` moved k and the spans397 together, so its failure was unattributable and cost four follow-up probes to398 decompose.399- Verdict classes seen: `OK`, `Overfitted submission`, `Failed to run`,400 `Processing` (the path admitted signals take; rejections flip straight to401 Overfitted).402- Uploads are manual through the browser (`alphanova.tech`, account `cookedjay`).403 **Verify the counter moved.** Matching the file picker's filename label is not404 proof the upload landed; that produced a false "submitted" claim in cycle 1.405406**Measurement discipline.** Adopted mid-cycle, never falsified, keep all of it.407408- **The noise bar: 0.0127.** Block bootstrap (120-row blocks, 2000 resamples) of409 the confirm-window net Sharpe. **Any local improvement under ~0.013 is luck.**410 Most of cycle 1's "wins" were inside it.411- **Plateau, not argmax.** Take a config whose grid neighbours are also good. A412 lone spike is a discretisation artifact: an 800-row evaluation block fits only413 3-4 rebalances, so step-aligned grids are spiky by construction.414- **Two windows, counted looks.** Selection = periods 001-024, confirm = 025-031.415 One confirm look per decision, budgeted in advance. By round 7 the confirm416 window had had 12+ looks and was spent; after that only server history was clean.417- **Multiplicity honesty.** Carry a "Tried" count per family in the ledger so a418 best-of-13 is never written up as a single measurement.419- **Overlap-adjusted significance.** The 120h-overlapping label means naive420 t-stats divide by ~sqrt(120). At 120-240h holds the effective sample is ~100-200421 observations, so mean IC -0.013 is only t ≈ -0.87. Correlated signals are one422 observation, not six.423- **Pre-register.** Candidate, bars, and what each verdict proves, written before424 the run. It is what made cycle 1's failed predictions legible instead of425 rationalisable.426427---428429## 7. Hard rules for the submission file430431Server-side auto-rejection. Check every one before upload.432433- No `assert` on row order, derived values, or the target (cycle-2 rule). The434 gate retrains on shuffled labels; a crashed retrain fails the whole run.435 `predict()` must be pure inference and may see rows overlapping `train()`.436 Screen for it: `.venv/Scripts/python research/40_crash_screen.py <file>`437 trains on real and on row-permuted labels and checks the output. Crash438 screen only; it says nothing about the gate verdict.439- One `.py` file. All logic inside the single `Predictor` subclass. Module-level440 imports only. No top-level helpers, no global state.441- `predict()` output cross-sectionally de-meaned: `p.sub(p.mean(axis=1), axis=0)`.442- No future-looking operations anywhere. No `shift(-n)`, no `bfill`, no443 `center=True` rolling windows, never touch the target inside `predict()`.444 Only `.shift(1)` is sanctioned.445- CPU only. `train()` < 4 min, `predict()` < 60 s, RAM < 8 GB. Profile446 `predict()` for instance-based models: `zoo_knn` returned "Failed to run" on447 the 60 s budget.448- Never modify anything in `comp6_scientist_release/`.449- Extra deps via a PEP 723 header. Pre-installed: numpy, pandas, scikit-learn,450 xgboost, lightgbm, pyarrow.451- Ticker identity is not stable across periods. Cross-sectional structure only.452- Embargo the last 120 rows of the training tail. The label is a 120h forward453 sum, so the tail leaks.454455**Mandatory pre-upload gates:**456```457python research/11_gate_guard.py <file> # static legality, must say LEGAL458python -m pytest research/tests/test_invariants.py -k <family> # 6/6459python runner.py <file> --full --gauge-fix # from comp6_scientist_release/460```461The guard caught a banned centred window on a training label in cycle 1. Keep it462in the loop.463464---465466## 8. The open problem, and the untried mechanisms467468**The open problem: no local quantity predicts the server score.** Not across469constructs, and for IC and Sharpe not within one either (F2). Concentration rank470and ic_std rank transfer; nothing local measures the sign of server IC. So the471local instrument is the bar plus the process null (method, items 2-3), and the472server is the only oracle of sign. Design for it: few uploads, each a different473construct, each with a written prediction. Rejections being free makes a wrong474construct cheap. A wrong admission is not free (F5).475476**The bar, from the field regression, at IC_std ≈ 0.19:**477478| concentration | server IC needed for Sharpe > 0 |479|---|---|480| 0.20 | +0.0150 |481| 0.30 | +0.0109 |482| 0.40 | +0.0068 |483484We reach concentration 0.31 and IC +0.0034. Treat the table as a rough target,485not a ranking device (F3): the k-curve cleared zero at an IC the table says is486far too low, because stability was doing work the coefficients understate.487488**Explored space (inventory of `research/families/`, 116 files, Sep-2, by489grep; re-read a file before you cite it).** Inputs: F1-F4 ranks, abs-ranks,490averages, negation, the F1-F3 and F3-F4 spreads; F5 and F6 ranks, EWM-then-rank491and rank-then-EWM at spans 240-960, squares; a 480h and a 240h F6 change;492per-feature cross-sectional dispersion and rank x dispersion493(`xsec_ctx_mlp.py`); rotation-covariant unit-norm rows (`f2_covariant.py`).494Models: ridge, Huber, ElasticNet, LightGBM regression / multiclass / lambdarank,495MLP, KNN, RF, ET, HGB, XGBoost, seed-bagged LightGBM (`f1_bag.py`), cross-family496rank ensemble. Output: TILT_K 0-25, output EWM 24-1920, step-hold 120, recency497weights 0.5-8, WINDOW 0.5, tranching, EWM tilt estimator, output csrank. Labels:498de-tilted, rank-smoothed, quintile classes. Training: EMBARGO 120, LOOKBACK499120-960, one global fit per `train()`. Every temporal op is `.ewm()`. Every500normalisation is cross-sectional.501502**The hunt map: not found in `research/families/`, ranked by distance from the503pot.** Each item is one registry entry, one sub-agent, one look against the bar.5045051. **Per-asset time-series normalisation.** Zero `.rolling(` and zero `groupby`506 calls in 116 files. Every input is a cross-sectional rank. A per-asset507 z-score or percentile against its own history is a different object.5082. **Feature deltas and accelerations at several horizons.** One delta exists509 (F6, 480h). No delta ladder, no second difference on any feature.5103. **F5 volatility or regime measures.** F5 appears only as a rank or an EWM511 rank. No rolling std, no dispersion of F5, no regime flag.5124. **Rolling per-asset autocorrelation or sign-persistence as an input.**513 Measured once as a diagnostic (IC autocorr +0.899 at lag 1), never fed in.5145. **Sign-agreement across F1-F4.** The spreads use magnitude differences; no515 concordance count exists.5166. **Bagging across training windows**, not across seeds (`f1_bag.py` bags517 seeds).5187. **Per-period re-fitting inside one submission.** Every file fits once.5198. **Row-wise gauge normalisation of the output** (cycle-1 item, still untried).520 Concentration is the norm of the mean unit vector; normalising each row521 before the de-mean targets it directly.5229. **Feature-PC neutralisation.** Project the prediction off the first PC of the523 feature panel instead of its own time-mean tilt.52410. **Embargoed early stopping.** An inner purged split chooses `N_ROUNDS`.525526Tried since cycle 1 and off the list: seed-bagging (`f1_bag.py`),527classification framing (`f1_cls.py`), recency-weighted training (cycle 2,528`rs_rec_*`, admitted at +0.0039 to +0.0043, inside the noise). A sweep of a529hunt-map item's parameters is a local diagnostic after it clears the bar, not530before.531532---533534## 9. File map535536In this skill directory:537538| File | What it is |539|---|---|540| `evidence.md` | The full measured record: data, metrics, every failed lane with numbers, the instruments that lied, the six dead gate theories |541| `baseline_predictor.py` | Structural skeleton to start from, k reset to 1 |542543In the repo:544545| Path | What it is |546|---|---|547| `comp6_scientist_release/COMPETITION.md` | The law. Gate mechanics at lines ~215-240, legacy pot ~23-38, quality set ~60-92 |548| `comp6_scientist_release/runner.py` | The only trustworthy local harness |549| `comp6_scientist_release/city_tools.py:105` | `concentration` definition |550| `research/SCOREBOARD.md` | Append-only ledger, 39 rounds, every retraction |551| `research/11_gate_guard.py` | Static legality check, mandatory |552| `research/tests/test_invariants.py` | Per-family invariant suite |553| `research/04_shuffle_gate.py` | Shuffle rehearsal. Crash screen only, never a blocker |554| `research/33_leaderboard_study.py` | Field regression over the scored board |555| `research/36_candidate_corr.py` | Correlation matrix, run before spending a slot |556| `research/37_untried_axes.py` | Diagnoses an axis before you build on it |557| `research/40_crash_screen.py` | Trains on real and permuted labels; crash screen before upload, says nothing about the verdict |558| `research/41_local_vs_server.py` | Local-vs-server calibration on the 20 cycle-1 admissions (conc rank +0.96) |559| `research/50_c2pot_corr.py` | Union-pot screen (cycle-1 pot + our cycle-2 admissions); merges subset runs by family |560| `docs/EXPERIMENT-PROTOCOL.md` | Governs every run: sample-size math, the bar, registry P0-P3, NOT-DONE table, tripwire |561| `research/families/rs_s96_win.py` | **Cycle-2 best: +0.0049, rank 1 on day 1.** Pot-blocks every ridge-on-F5/F6 variant |562| `research/families/xsec_ctx_mlp.py` | The only creative input set (rank x dispersion, 22 columns); never uploaded; base of the cycle-3 anchors |563| `research/families/otv_k25.py` | Cycle-1 best: +0.0003, rank 19. Burned by the pot |564| `research/families/otv_minimal.py` | Highest IC at its output stage (+0.0065) via minimal capacity |565| `research/families/f1_interactions.py` | Best raw server IC we ever posted (+0.0071) |566| `research/families/otv_recency.py` | Recency weighting; verdicted in cycle 2 as `rs_rec_*`, inside the noise |567| `docs/plans/2026-08-17-cycle2-design.md` | Mostly stale: its levers 1 and 4 were later falsified |568569---570571## 10. Why cycle 1 took 39 rounds572573Written so it does not repeat. One root cause, six expressions.574575**Root cause: we optimised what we could measure instead of measuring what we576were scored on.**5775781. We submitted before understanding the gate. Half the uploads were579 hand-written rules. F1 was derivable from `COMPETITION.md` on day 1 without580 spending a single slot.5812. We trusted local metrics for 20 rounds. `corr(local, official) = -0.558` was582 computable after the first four verdicts and was not computed until round 17.5833. We did not read the field until round 23. The leaderboard API hands you 64584 scored signals with IC, concentration, IC_std and Sharpe. One regression gives585 the scoring function, free, on day 1. It falsified two live theories the hour586 it ran.5874. We fitted six theories to the verdict list instead of designing experiments588 against it. Every curve-fit died. The designed one-variable experiment solved589 it in a single upload.5905. We built a second harness alongside the official runner. It disagreed twice591 and every number it produced was retracted.5925936. **We theorised where we should have swept.** The campaign's only positive594 score came from four cheap probes that each changed one number, fired on the595 last day. Thirty rounds of theory produced -0.0019; the probes produced596 +0.0003 and corrected a wrong attribution at the same time. A parameter sweep597 against the real scorer beats a model of the real scorer. **Cycle 2 then598 inverted the error: it swept where it should have hunted**, 30 of 32 uploads599 on one construct (section 11). The sweep is a local diagnostic on a600 registered construct. It never plans the slots; the bar does.601602**The corresponding habits:** derive the gate before uploading; measure the603local-versus-official correlation as soon as four verdicts exist; fit the field604regression on day 1 and re-fit as the board fills; spend early local runs,605never slots, on a single-variable sweep of the most suspicious parameter; keep606exactly one harness, the official one.607608## 11. Cycle 2 record (Sep 1-2, 2026): what worked, what did not609610**Outcome.** 32 uploads, 20 admitted, 12 rejected, all 20 slots spent by Sep-261101:00. Best: `rs_s96_win` +0.0049 (rank 1, the sole Q1 star on day 1, server IC612-0.0001). The cycle-1 Q3 cutoff was +0.0135. Nothing we built is above the noise,613locally or on the server. Full detail: `research/SCOREBOARD.md` lines 1990-2300.614615**Worked (server-verified).**616- A trained ridge on two slow inputs (F5, F6 EWM-480 ranks), output EWM span 96,617 unit de-tilt: admitted every time at spans 24-144 and lookbacks 240-480. The618 gate passes it. It scores about zero.619- The fit-row lever: the same ridge fitted on recent rows only (recency weights,620 or the newest half) sat above every plain fit, 7/7 vs 0/13, and moved the score621 from +0.0006 to +0.0049. Still inside the noise bar.622- Local gauge-fixed concentration rank transfers to the server (+0.96). No other623 local number does.624- Pre-registered GATE lines, one variable per file: 13/13 correct on admission by625 the end. The gate is understood; every verdict was attributable.626627**Did not work.**628- Local Sharpe and IC as a ranking device, again. Local IC is anti-correlated629 with server IC inside one construct (-0.78); `win` read -0.0091 locally and630 scored best. On fit-row levers the local order is non-evidence.631- Gate-out at k=1: output span 192+, input lookback 960, any nonlinear term of632 the ranked inputs (product, squares), LightGBM on slow inputs, Huber loss. The633 one-input theory was falsified (`f6` admitted).634- Concentration alone does not pay: `rts` had the board-high conc 0.2186 at635 -0.0029.636- Sweeping one construct. 30 of 32 uploads were parameter variants of one ridge.637 All 20 admissions are now ONE pot ticket (0.75-1.0 among themselves) and every638 held build is pot-blocked.639- No new features in cycle 2. Zero. The xsec context set (cycle 1) is the only640 creative input set built, and it was never uploaded.641642**Self-audit (quant-ml-protocol Stage 5.6), written Sep-2.**643- Data: clean. Contemporaneous features, warm-up dropped, 120-row embargo.644- Statistics: the sample-size math was never written. n_indep ~207 gives645 SE(IC) ~0.015, so no achievable IC is locally detectable; the Sharpe bar646 0.0127 was known and every "win" sat inside it. Trial count: 116 family files,647 66 uploads. The local maxima are noise maxima.648- Code: one defect, fixed (`50_c2pot_corr.py` overwrote its table on subset runs).649- Process: no protocol file, no trial ledger, no process null, and the650 quant-ml-protocol skill was not used. Slots went to sweeps of a construct known651 to sit at zero. Fixed Sep-2: `docs/EXPERIMENT-PROTOCOL.md` v1 in the repo.652653**Cycle-3 rule (Keith, "B", Sep-2).** No slot below local net Sharpe 0.02 on all65431 periods, one look, pre-registered, above the process-null p95. Hunt new655constructs and new inputs, not parameters. The protocol file governs.