News Results Refresh

News & Results Refresh Loop

Keep dashboard-stock verdicts current. The detector finds material events; you refresh the dossier head window and — for an earning-power event — re-estimate IV, re-derive the verdict, and (under the re-rank guard) re-rank the LOCKED dashboard. The autonomy ladder never mutates the dashboard silently: every write goes through a sanctioned single-column writer (col N / col L) or the guarded full re-rank (82_rerank_guard.py: 3-tab backup + dry-run-diff + ticker-keyed checksum + auto-revert), and a FOREVER promotion always stops at the human gate.

Sweep universe = the FULL Action Dashboard (~241 names), derived LIVE from the dashboard each cycle (col B data rows, legend excluded), unioned with the held set so owned-but-off-dashboard names are never dropped. build_sweep_universe.py re-derives data/manifests/dashboard_universe.json from the live sheet at the start of every run, so a newly-onboarded dashboard row auto-joins the sweep next cycle with no manual step. The resolver exposes sweep_tickers() (dashboard ∪ held; degrades dashboard→held→legacy) for the sweep, and keeps held_tickers() (owned-only) for where the owned distinction matters (e.g. the AVOID-owned→SELL col-N word). The builder is READ-ONLY on the dashboard.

Golden rules (never violate)

1. Only the HEAD WINDOW changes (frontmatter snapshot + one-line thesis, first ~4,000 chars). Research PROSE is frozen. Never tail-append — 26_build_lancedb_index.py embeds one ~4,000-char head vector, so tail edits are invisible (the Repco-CBI stale-flag lesson). The structured ## One-line thesis is what col N renders: VERDICT · ✅ best · ⚠️ worst · 📰 <date>: <signal> — the 📰 segment is verdict-NEUTRAL reportage from event_index.sqlite (coln_news_segment.py), never fabricated, always after the verdict word.
2. Autonomy ladder — L0/L1 auto; L2 IV/conviction/verdict-DEMOTION/re-rank AUTO under the guard; FOREVER-promotion the SOLE gate (user override, 2026-06-25):
   • L0 — stamp last_event_checked: <today> on every swept dossier (proves "checked, nothing material" vs "never checked"). Auto.
   • L1 — a one-line catalyst / "Why" refresh, NO grade change. Auto-apply; flows to the Sheet "Why" / col N via extract_dashboard_why.py + 79_publish_col_n.py.
     • COL-N IS PUBLISHED FOR EVERY ROW (user directive 2026-06-26) — including AVOID / board-contradiction / needs_review names, analysed and filled exactly like any other stock. The binding Munger/Buffett CIO (vault) verdict is PRIMARY and is what col-N shows; the 56-director board panel is ADVISORY and NEVER vetoes col-N publication. 79 publishes whenever a VALID col-N can be synthesized (verdict-word + ✅ best + ⚠️ worst, optional 📰); the ONLY skips are verdict==NEEDS_REVIEW, a genuinely THIN dossier (BOTH best AND worst NEEDS_REVIEW), no rendered string, or head_overflow. A board contradiction / governance flag / within-tier needs_review does NOT block col-N — it is a CONTESTED GRADE that still publishes the current verdict's col-N and is RECORDED for human adjudication (78's board_contradictions + 81_audit_dashboard_consistency.py's report), not vetoed. 78 (checker) Gate 7 records the board (mis)alignment as a non-blocking signal; it never fails verification on a contradiction.
   • L2 — IV change / conviction_level change / verdict DEMOTION / within-tier flip / MoS recompute. NOW AUTO-applies AND auto-re-ranks the LOCKED dashboard via the guarded loop: an earning-power event passes coln_iv_event_gate.py → 71_refresh_orchestrator.py runs the targeted Stage-3 re-grade (binding consult + 80_estimate_iv.py + conviction gates + MoS recompute + verdict re-derive + board sanity-check) → re-freezes the head → 73_refresh_cycle.sh stage 3c invokes 82_rerank_guard.py --tag headwindow (once per cycle, after the reindex) to re-rank + publish behind a 3-tab backup + dry-run-diff + ticker-keyed checksum + auto-revert + maker≠checker. Stage 3c fires only when RERANK_AUTO=1 AND the post-reindex brain is healthy (SKIP_PUBLISH≠1) AND ≥1 L1 was applied this cycle. RERANK_AUTO is ENABLED in the LaunchAgent plist (2026-06-25) — auto-re-rank is LIVE on the scheduled loop (capability shipped PR #33, wired into 73 stage 3c, supervised first-enable done; first guarded re-rank applied: AWL→WATCH). The code default remains OFF (so a manual run without the env var dry-runs + asserts + prints the order-diff and STOPS; no live re-rank), but the scheduled cron now runs with RERANK_AUTO=1 set in the plist. 71 itself NEVER calls 82 — the re-rank is once-per-cycle in 73, not per IV event.
   • SOLE human gate (for the GRADE, not for col-N) — a FOREVER promotion (Stage-4 adversarial inversion a cron can't do), plus any needs_review / board-contradiction that would change the GRADE: routes to data/state/pending_reviews/<TICKER>.json, NEVER auto-applied. NOTE: this human gate is about the verdict/conviction GRADE only — col-N is still published for the current verdict of these contested rows (the board is advisory; see the L1 col-N rule above). A board-contradiction queues the GRADE for adjudication; it does NOT withhold the row's col-N.
   • IV = earning power only. Price / FII / MF flows / ratings / buy-calls move live MoS + Upside ONLY — coln_iv_event_gate.py returns False for them, so they NEVER trigger an IV re-grade or a re-rank.
3. The vault is BINDING. Any conviction/verdict delta must be supported by 32_consult_brain.py --corpus binding citations (real slugs only, no fabrication).
4. Maker ≠ checker. The refresh writer and the verifier are separate (Phase 4 scripts 71/72). Until those exist, run the refresh manually and verify in a separate pass.
5. Never fabricate an event. Every event row carries a dated source URL. If the detector finds nothing, that is success — stamp L0 and stop. No early-exit "empty = bug".
6. Recall canary is a hard floor. After any reindex, Recall@10 must hold ≥0.95; auto-revert the index on any drop.

Autonomy-ladder ratification. Ratified 2026-07-03 (PW-13): the daily loop auto-consults fresh (≤2d) RED+MATERIAL events, hard-capped at CYCLE_CONSULT_BUDGET=25/cycle, RED-first-oldest-first. L1 auto-applies; L2 IV/conviction/verdict changes including upgrades up through COMPOUND auto-apply under 82_rerank_guard. FOREVER promotion remains the sole permanent human gate. Backlog (>2d) events consult only under DRAIN_GO=1 (founder-gated). The value-digest reports these changes (email); it never grades.

Workflow (one cycle)

0. Build sweep universe — build_sweep_universe.py re-derives data/manifests/dashboard_universe.json from the LIVE dashboard col B (data rows only; legend excluded; aliases resolved e.g. ZOMATO→ETERNAL). READ-ONLY on the dashboard, degrade-safe (keeps the last good manifest on a failed read). Runs FIRST in 73_refresh_cycle.sh so onboarded names auto-join this cycle.
1. Detect — 70_detect_material_events.py over the sweep universe (sweep_tickers() = full dashboard ∪ held, ~241+) → data/state/event_index.sqlite. Materiality from data/config/event_taxonomy.json (RESULTS / DEAL / FILING / GOVERNANCE → INFO / MATERIAL / RED; GOVERNANCE-RED escalates). Recency window = since max(graded, last_event_checked). (--universe prints the swept set + dump coverage.)
2. Triage — for each ticker: no MATERIAL event → L0 stamp only. MATERIAL → continue.
3. Re-grade (head-window) — consult the binding brain (32_consult_brain.py --corpus binding --step conviction|margin-of-safety|verdict), reconcile with the event, decide the delta tier. Earning-power event? coln_iv_event_gate.py arms the targeted Stage-3 IV re-grade in 71 (re-run 80_estimate_iv.py price-free + conviction gates + MoS/buy_below recompute + verdict re-derive + board sanity-check).
4. Write back — L1: head-window edit via apply_head_window_grade.py (HEAD_LIMIT=4000) → col N re-published (77→79). L2 IV/conviction/verdict-DEMOTION: 71 re-freezes the head with the new grade (under --apply), then 73 stage 3c invokes 82_rerank_guard.py --tag headwindow to re-rank + publish the whole set — gated by RERANK_AUTO=1 (default-OFF → dry-run-diff + assert + STOP) AND brain-healthy post-reindex (SKIP_PUBLISH≠1) AND ≥1 L1 applied this cycle. A stale live verdict/conviction cell (which the re-rank CARRIES byte-for-byte and 82 asserts against the dossier oracle) is first reconciled by the sanctioned col-D/col-M writer sync_col_dm.py → write_col_dm.mjs (default dry-run; COL_DM_DASHBOARD_WRITE=1 to write). FOREVER-promotion / needs_review / board-contradiction: proposal JSON to pending_reviews/, NOT the dossier head, NOT the dashboard.
5. Reindex + canary — 26_build_lancedb_index.py; confirm Recall@10 ≥0.95 or auto-revert.
6. Log — append a 🔄 Refresh-Log row (Ticker · Event date · Type · Severity · Source · Status · Days stale · Proposed Δ). When the re-rank guard is armed, the LOCKED dashboard is re-ranked through 82 (guarded + auto-revert); otherwise it is untouched and the proposed Δ waits.
7. Stop condition — every swept ticker (full dashboard ∪ held) last_event_checked == today AND ingest clean AND no schema/Recall failure AND every MATERIAL event is {stamped | applied-L1 | queued-L2}.

Human gate (the sole remaining one: FOREVER promotion)

Most L2 changes (IV, conviction, verdict DEMOTION, within-tier, MoS) now auto-apply and auto-re-rank under the guard. The sole human gate is a FOREVER promotion (plus a needs_review / board-contradiction that would change the GRADE) → queued to pending_reviews/<TICKER>.json. That gate is about the GRADE only — col-N for these contested rows is still published for the current verdict (the board panel is advisory and never vetoes col-N; a board-contradiction queues the grade for adjudication, it does NOT withhold the row's col-N). The user approves a queued proposal with one apply_head_window_grade.py <approved>.json run → then a WHOLE-SET re-rank (per stock-onboarding HARD RULE #2, now wired into the loop at 73 stage 3c via 82_rerank_guard.py). Kill-switch: loop-pause-all in STATE.md → scheduled runs no-op. RERANK_AUTO=1 is ENABLED in the LaunchAgent plist (2026-06-25) — the scheduled loop's stage 3c re-ranks live (guarded + auto-revert). The code default is still OFF, so a manual 82 run without the env var dry-runs + asserts + STOPS; the plist supplies RERANK_AUTO=1 for the cron.

Scope & cadence

FULL Action Dashboard (~241 names) ∪ held, DAILY (Phase 5 LaunchAgent 10:00 IST — com.invest.news-refresh StartCalendarInterval Hour 10; the separate price-alerts agent runs later at 16:00). The universe is re-derived LIVE from the dashboard each cycle (build_sweep_universe.py → dashboard_universe.json), so onboarded names auto-join with no manual step. Held-only (off-dashboard) names stay in via the union.

See REFERENCE.md for the exact commands, SQLite schema, taxonomy, file map, and phase status.