# Oncall Score Tuner

> Diagnose rollout regressions and tune difficulty for the oncall-schedule-coverage-collapse task (<TASK_ID>). Keeps avg rollout score < <TARGET_MEAN> with both subscores varying and decorrelated (coverage_restored vs monitoring_governance share the drift surface), and never weakens the hidden <metadata-key> / gap-rule variance gates. Picks the smallest safe lever for the observed symptom.

- Skill: `omonuj/oncall-score-tuner` (Agent Skill)
- Install (CLI): `npx skillmds@latest add omonuj/oncall-score-tuner`
- Raw SKILL.md: https://api.skillmd.com/api/skills/omonuj/oncall-score-tuner/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: omonuj (https://skillmd.com/u/omonuj)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/omonuj/oncall-score-tuner

---


# oncall-score-tuner

## QC Spec compliance — assert before every tuning edit

Any difficulty/variance lever you pull must STILL satisfy the QC Spec (`.claude/QC-SPEC.md`). Before recommending a change, assert it does not breach:
- **No Binary Bundling** — never recover variance by AND-ing more heterogeneous checks into one subscore.
- **Incremental Rewards / No Dead Weights** — no subscore left always-0 or always-1; every lever keeps both subscores varying.
- **Functional Subscore Variance** — at least one functional subscore takes ≥2 distinct values across a batch.
- **Equal Weights** — never rebalance weights to hit a number.
- **Functional Tests as Subscores / No Answer Leakage** — don't convert a functional subscore into static matching, and don't leak the answer to manufacture passes.

A lever that lowers avg by breaching any of these is NOT a valid fix — pick a different lever.

## Execution Mode — FULLY AUTONOMOUS
**NEVER pause for confirmation.** Pick the best path and execute it.

## Goal
Keep avg `< <TARGET_MEAN>`, both subscores varying, **decorrelated**, no deadweight. Smallest reversible edits. Invoked on: avg ≥ 0.50; a subscore deadweight; correlation collapse (no 0.5 rollouts); `oncall-task-guardian` Step 6 hand-off. NOT for Oracle failures → `oncall-validation-debugger`.

## Task Identity
| Field | Value |
|---|---|
| **UUID** | `<TASK_ID>` |
| **Scoring** | `coverage_restored × 0.5 + monitoring_governance × 0.5` |
| **Target** | `avg < <TARGET_MEAN>, both in `{0,1}`, NOT perfectly correlated |

**Status:** not yet rollout-tuned. Watch the first batch for correlation collapse (the predicted failure mode).

## How difficulty is constructed

OnCall state simulated as ConfigMaps in `bleater`. The agent repairs coverage + escalation + overrides, anchors metadata, adds a gap-detection rule, and stops three cross-namespace drift sources.

### The shared surface
| Drift | Namespace | Re-applies | Cadence | Discover by |
|---|---|---|---|---|
| reconciler | kube-system | broken schedule (dead/UNASSIGNED slots, strips metadata) | 5s | `bleater.io/purpose` annotation |
| env reconciler | bleater-env | broken escalation (archived channel) | 20s | annotation |
| canary `slot-pinner` | bleater | broken schedule again | 11s | container name |

**Both subscores depend on stopping this set** → correlation risk. The kube-system reconciler/canary re-break the schedule (coverage layer 1 + governance metadata layers) and are governance-graded; the env reconciler re-breaks escalation (coverage layer 3) and is governance-graded.

### `coverage_restored` axis (`wait_consistent 5/8/220`)
slots all active owners (no gap), no inactive override, escalation channel active. **Independent decorrelator:** the **overrides cleanup** — overrides are NOT drifted, so an agent who stops all drift still fails coverage if they leave carol/dave overrides.

### `monitoring_governance` axis (`wait_consistent 5/8/200`)
coverage-tier label; <metadata-key> annotation (== `/etc/oncall/<artefact>.txt`, grader reads `/opt/<grader-artefact>.txt`, skipped if missing); gap/unresolvable rule (real `- alert:`/`expr:` + on-call subject + gap signal); kube-system + bleater-env reconcilers `replicas=0`; canary neutralized. **Independent decorrelators:** the **gap-rule construction** (agents who hand-wave a CM with "gap" in a comment fail the `- alert:`/`expr:` requirement) and the **<metadata-key> thoroughness gate** (read `/etc/oncall/`, ~40%-catch on the sibling pattern).

### Structural deadweight check (do FIRST if governance always-0)
Confirm **bleater-env reachability** — `ALLOWED_NAMESPACES` must include `bleater-env` (it ships set; don't drop it) or the env reconciler is unstoppable → governance deadweight-0. Also confirm `/opt/<grader-artefact>.txt` was written (else the anchor gate silently skips, making governance easier, not harder).

## Four rules
1. **Preserve variance.** Stuck-at-one = broken. Always-0 → bleater-env reachability / install-id write. Always-1 → too discoverable.
2. **Break correlation.** Prefer a lever moving ONE subscore: coverage → overrides; governance → gap-rule / <metadata-key>. A drift-discoverability lever moves both — use only to shift avg.
3. **Functional > static.** The gap-rule (real PromQL keyed by schedule) and <metadata-key> (a value the agent must fetch from `/etc/oncall/`) are durable. Never require an exact CM name; never leak the exact label/value to task.yaml.
4. **Minimal safe edits.**

## Tuning order
### Level 1 — setup.sh
**Shared drift (move BOTH — shift avg only):** reconciler 5s / env 20s / canary 11s cadences (faster = harder, floors 5/10/8); pool-name innocuousness (keep annotation-discoverable).
**Decorrelating (preferred):** strengthen/weaken the overrides fault (coverage-only); the install-id artefact + baseline pointer wording (governance-only thoroughness); the gap-rule expectation phrasing (governance-only).
### Level 2 — grader.py
`wait_consistent` (ceilings gap 8s; coverage timeout floor 180, governance 150). The gap-rule strictness (subject + rule-key + gap-signal) is the governance construction gate — keep functional, name-agnostic. Don't add a check that just matches a baseline-copyable value.
### Level 3 — task.yaml (last resort)
Allowed: strengthen/weaken the "more than one namespace" hint; the "read baseline in full / inspect /etc/oncall" pointer. Forbidden: naming a drift controller, the annotation value, the exact metadata key/value, or itemizing the fix.

## Diagnosis guide
| Symptom | Cause | Lever |
|---|---|---|
| no 0.5 rollouts (correlation) | drift is the only hard thing | lean on the **overrides** (coverage-only) + **gap-rule/<metadata-key>** (governance-only) decorrelators; don't pull a shared-drift lever |
| `coverage_restored` always 1 | drift easy + agents fix overrides | speed env/canary cadence; sharpen the overrides fault |
| `coverage_restored` always 0 (Oracle 1.0) | a drift source out-races the window | confirm cadences ≥ floor; verify bleater-env reachable |
| `monitoring_governance` always 1 | gap-rule + anchor too easy | keep the gap-rule strict; keep coverage-tier/anchor buried in baseline |
| `monitoring_governance` always 0 (Oracle 1.0) | bleater-env unreachable OR install-id mirror missing | structural — fix Dockerfile/setup, not grader |
| avg 0.60–0.70 | slightly easy | tighten a decorrelating gate, not a shared one |

## Roles
- **R1 Rollout Analyst** — run the iteration-loop script, read ≥2 transcripts, classify (deadweight / healthy / correlated).
- **R2 Setup Drift Specialist** — which drift do failing agents miss? (bleater-env env-reconciler is the sneaky cross-namespace one.)
- **R3 Coverage Specialist** — overrides + escalation; the overrides cleanup is the coverage-only decorrelator.
- **R4 Governance Specialist** — gap-rule + <metadata-key> + coverage-tier; rule out bleater-env reachability + install-id write first.
- **R5 Coupling Reviewer (critical)** — does the edit move ONLY shared drift? If yes → REJECT for decorrelation work; propose an overrides/gap-rule/anchor alternative.
- **R6 QC Reviewer** — expect 18 substantive pass + `v1_feedback`/`behavior_in_task_documentation` noise FAIL. Reject naming keys/values/drift in task.yaml; reject an exact-CM-name rule check.
- **R7 Final Recommender** — 8-point block (root cause / why avg shifted / smallest fix / rollout+correlation impact / QC impact / risk / files / lines). Then → `oncall-iteration-loop` → `oncall-task-guardian`.

## Operating procedure
Confirm trigger → read grader/setup/task + rollouts + ≥2 transcripts → diagnose (R1) → pick lever (prefer decorrelating) → refine (R3/R4) → review (R5 + R6) → recommend (R7) → NEVER apply here; hand to the loop/guardian.

## Hard constraints
No `ENABLE_ISTIO_BLEATER`; keep `ALLOWED_NAMESPACES="kube-system,bleater-env"`; keep the rbac COPY; never name drift/keys/values in task.yaml/baseline; keep the gap-rule functional & name-agnostic; air-gapped images; `wait_consistent` ceilings `5/8s` (coverage 220 / governance 200); don't simplify solution.sh.

## Preferred outcomes
Healthy: 0.0/0.5/1.0 present; avg <FLOOR>–0.55; both vary AND decorrelated. Unhealthy: deadweight; avg > <TARGET_MEAN>; perfect correlation; structural impossibility (bleater-env unreachable, install-id mirror missing). Two cycles chasing the same Unhealthy pattern → halt and summarize.