# Radar Self Eval > Compute the radar's self-calibration: weekly funnel metrics (queue dynamics, exploration compliance, off-axis rate), a monthly hit/miss retrospective against what actually became big, and up to 3 curator proposals. Use during every weekly recalibration, after the source-strategy review; results go into the append-only `logs/calibration.md` and into the weekly report. - Skill: `neetx/radar-self-eval` (Agent Skill) - Install (CLI): `npx skillmds@latest add neetx/radar-self-eval` - Raw SKILL.md: https://api.skillmd.com/api/skills/neetx/radar-self-eval/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Docs & Writing - Author: Neetx (https://skillmd.com/u/neetx) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/neetx/radar-self-eval --- # Self-evaluation The radar must measure whether it is winning, not assume it. Three parts: metrics (every week), retrospective (monthly), curator proposals (every week). ## 1. Weekly metrics Compute from the week's daily reports (primary source — they list ledger changes) and, where needed, `git log -p -- TRENDS.md` since the previous weekly commit: - **queue funnel**: items added / promoted to trend / dropped / older than 14 days and still unverified (stale) - **ledger**: evidence items added, stage moves (up and down) - **exploration compliance**: daily runs whose `logs/source_rotation.md` line contains a venue-exploration entry ÷ daily runs executed - **off-axis rate**: share of new queue items that do NOT match any axis in `logs/strategy_notes.md` (judgment call — name them) - **discovery lag**: for each evidence item added this week, the days between its evidence-line date and the date it entered the ledger (commit date via `git log -p -- TRENDS.md`, or the daily reports). Report the median, split by channel — exploration finds vs queue promotions (backfill) — plus the backfill share of all new evidence. This is the daily-ness KPI. - **coverage** (the self-check that auto-detects the listed-but-never-swept failure class): for each "swept every run" heading in SOURCES.md (lab blogs, YouTube curators, pointer/digest blogs, discovery venues, tool-discovery channels), enumerate EVERY entry under it — INCLUDING bullets nested under a sub-label (e.g. a "non-GitHub channels:" sub-block), not just top-level list entries — and diff that full list against the week's `logs/source_rotation.md` lines, classifying every listed source as `opened`, `degraded`, or **MISSING** (in SOURCES.md but never in any log line this week). MISSING = a coverage lie — NAME them; never report "0 missing" without having run the full nested list-vs-log diff. A source MISSING or `degraded` for the whole week is a heal-or-REMOVE candidate (see Amendments): the registry must be honest about what it actually sweeps. - **routing-leak**: queue items tagged with (or clearly on) an EXISTING trend's axis yet absent from that trend's evidence — on-axis primaries hoarded in the queue instead of routed to evidence (the UltraQuant failure class). Count and name them; each is a routing miss to correct this week. - **source-discovery**: from SOURCES.md → "Discovered-source candidates" and the week's reports — candidates STAGED this week / PROMOTED to a swept source / PRUNED, plus discovery-latency = runs (or days) between an org's FIRST appearance in the staging tally and its promotion. This is the lab/vendor analog of routing-leak: a recurring on-axis org that stays un-promoted (≥2 weeks in the tally) is a coverage leak — NAME it and promote-or-drop it now. Zero candidates staged for two weeks while new evidence keeps arriving from the SAME few orgs is itself a signal the capture-the-source step isn't firing. - **capture-leak** (the notes-prose blind spot, complementary to routing-leak): scan every trend's `notes` field AND the week's reports for named primaries (arXiv ids, repo / release URLs) that appear NOWHERE as a discrete `observation_queue` item or an evidence line on any trend. Each is a capture leak — a primary the radar named but never captured as a first-class item, so it never reached study-pick selection or the convergence check (the notes-buried-item failure class). NAME them and queue them now. routing-leak only scans the queue, so an item that never reached the queue is invisible to it; this metric is what catches it. Append ONE dated line to `logs/calibration.md` (the externalized self-eval log; the `## calibration` section of TRENDS.md is now only a pointer): `- YYYY-MM-DD — W: queue +a/→p/−d/stale s · evidence +e · moves m · exploration c/r · off-axis o/a · lag expl Xd / backfill Yd (Z%) · coverage / (miss , degr ) · routing-leak · capture-leak · src-disc / (lat L)` and include the same numbers, readable, in the weekly report. Interpretation thresholds (act, don't just log): - exploration compliance < 100% → call it out; two weeks running → propose a daily prompt/skill refinement. - stale > half the queue → scanning breadth exceeds verification capacity: propose narrowing or a verification-only day. - off-axis = 0 for two consecutive weeks → anchoring warning, even if the tunnel-vision check passed. - exploration-channel median lag > 7 days → the venues are stale: rotate them. - backfill share not shrinking for two consecutive weeks (once the bootstrap queue is cleared) → the radar is doing literature review, not daily detection: propose rebalancing scan time toward exploration. ## 2. Monthly retrospective (first weekly run of the month: `date +%d` ≤ 7) Goal: ground truth — did the radar see early what later became big? 1. Pick 2–3 items that clearly became big in the past month by BROWSING venues (open the pages — evidence rules apply): HF papers trending, two major lab blog indexes, one arXiv listing. Pick what is everywhere, not what is interesting. 2. For each, find the radar's earliest mention: `git log -S"" --oneline -- TRENDS.md` plus the queue and reports. 3. Classify: **HIT-early** (tracked ≥7 days before it was everywhere), **HIT-late**, **MISS**. 4. For every MISS: add it to `observation_queue` now, and name the venue or axis that would have caught it — feed that into the proposals. Log in `logs/calibration.md`: `- YYYY-MM-DD — retro M: — HIT-early|HIT-late|MISS (first seen ), …` ## 3. Amendments (proposed → cooling period → applied) The radar runs unattended and may change its own operating instructions — under the autonomy contract in AGENTS.md. Every week, up to 3 proposed amendments: routine edits (exact replacement text), axis drops/merges/ supersessions, exploration-budget changes. Each must cite the metric or retrospective result that motivates it. The new coverage/routing metrics feed amendments directly (this is how the radar self-corrects the failure classes the curator used to catch by hand): - a source **MISSING or `degraded` all week** → first a `radar-source-heal` attempt; if it was already healed and still fails, a **heal-or-REMOVE** amendment (remove it from the SOURCES.md "swept every run" list, or escalate the secret/allowlist it needs) — a first-class amendment, because a listed-but- unswept source is a coverage lie. - a non-zero **routing-leak** → not an amendment but a this-week FIX: route the hoarded on-axis queue items to their trend's evidence now; if it recurs, amend the routing instruction. Lifecycle: - **Week N**: propose — log in `calibration` and the weekly report. - **Week N+1**: apply IF the motivating signal persists AND no dated curator veto appeared in `logs/strategy_notes.md` (silence is consent). One dedicated commit per amendment. If the signal vanished, drop the proposal and say so. - **After applying**: if calibration metrics worsen for two consecutive weeks, `git revert` the amendment and log the rollback. Scope-axis changes are recorded as "radar-adopted" dated entries in `logs/strategy_notes.md`, explicitly naming what they supersede. Curator entries are never deleted or edited — vetoes and mission input belong to the curator alone. The immutable sections listed in AGENTS.md are out of bounds: an amendment touching them is invalid. ## Boundaries - Evidence rules apply to the retrospective: cite only opened pages. - `logs/calibration.md` is append-only: never edit or delete existing lines. - If a metric shows the same procedural failure twice, trigger a skill refinement per the maintenance policy in AGENTS.md (dedicated commit, explained in the report).