# Oncall Iteration Loop

> Orchestrator skill that runs the full push → Oracle → live red-team → rollout → tune cycle for the oncall-schedule-coverage-collapse task (<TASK_ID>) until Oracle scores 1.0 and rollout avg < <TARGET_MEAN> with both subscores varying and decorrelated. Delegates to oncall-task-guardian, oncall-validation-debugger, oncall-score-tuner, and horizon-agentic-reviewer at the right phases. The agent executes every step except triggering the rollout batch.

- Skill: `omonuj/oncall-iteration-loop` (Agent Skill)
- Install (CLI): `npx skillmds@latest add omonuj/oncall-iteration-loop`
- Raw SKILL.md: https://api.skillmd.com/api/skills/omonuj/oncall-iteration-loop/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: omonuj (https://skillmd.com/u/omonuj)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/omonuj/oncall-iteration-loop

---


# oncall-iteration-loop

## Execution Mode — FULLY AUTONOMOUS
**NEVER pause for confirmation at any point.** Run every phase, command, and fix without asking. Two valid paths → pick the safer. The only manual step is the user triggering the eval batch in the Horizon UI.

## Goal
Drive oncall-schedule-coverage-collapse from "needs work" to "shippable". Done when all exit conditions hold; each cycle picks the right specialist and applies the smallest safe fix.

## Trigger
"Iterate the oncall task until it passes"; "push and tune until avg < <TARGET_MEAN>"; "run the full loop on oncall-schedule-coverage-collapse"; anything implying repeated push → validate → analyze → fix. Single steps → invoke the specialist directly.

## Task Identity
| Field | Value |
|---|---|
| **UUID** | `<TASK_ID>` |
| **Slug / path** | `oncall-schedule-coverage-collapse` / `tasks/oncall-schedule-coverage-collapse/` |
| **Horizon root** | `/Users/mac/Documents/tasks` |
| **Venv** | `source /Users/mac/Documents/tasks/horizon_env/bin/activate` |

## Who does what
The agent executes everything except **triggering the eval batch in the Horizon UI**. Never tell the user to push/validate/pull.

## Exit conditions (all, same version)
1. Oracle `passed: true, score: 1.0`, both subscores `=1`
2. Live red-team: no BLOCKING findings — **especially bleater-env reachable, all 3 drift sources live, install-id agent-readable, a no-op agent failing both**
3. Rollout avg `< <TARGET_MEAN>` (and not `< <FLOOR>` — over-hard)
4. `coverage_restored` varies (0 and 1)
5. `monitoring_governance` varies (0 and 1)
6. **Decorrelated** — some `0.5` rollouts appear (not only 0.0 and 1.0)
7. Local quality: 18 substantive pass (`v1_feedback` + `behavior_in_task_documentation` may FAIL as accepted heredoc-invisibility noise — do NOT fix by leaking the hidden gates to task.yaml)

Short of all seven → continue.

## Loop limit
**Max 5 push cycles without human review**, then stop and summarize.

## No-Op default
Oracle only. Run No-Op manually only on `feedback: null`/`score: 0` (setup-crash isolation).

## The loop
```
PHASE 0  Pre-push (oncall-task-guardian Step 0–2): Dockerfile (ALLOWED_NAMESPACES=kube-system,bleater-env)
         + bash -n + py_compile + check-anatomy + check-quality (18 substantive pass)
PHASE 1  Push → version NNN
PHASE 2  Oracle → 1.0+both=1 → PHASE 2.5 ; else → oncall-validation-debugger → fix → PHASE 0
PHASE 2.5 Live red-team (horizon-agentic-reviewer) → clean → ask user to trigger batch ; BLOCKING → fix → PHASE 0
PHASE 3  Rollout analysis → avg < <TARGET_MEAN> AND both vary AND decorrelated (0.5s) → DONE ; else → oncall-score-tuner → PHASE 0
```

## Phase 0 — Pre-push
```bash
cd /Users/mac/Documents/tasks
echo "=== Dockerfile ===" && cat tasks/oncall-schedule-coverage-collapse/Dockerfile
#  must NOT contain ENABLE_ISTIO_BLEATER; must contain ALLOWED_NAMESPACES="kube-system,bleater-env" + COPY data/ubuntu-user-rbac.yaml
bash -n tasks/oncall-schedule-coverage-collapse/setup.sh    && echo "setup.sh OK"
bash -n tasks/oncall-schedule-coverage-collapse/solution.sh && echo "solution.sh OK"
python3 -m py_compile tasks/oncall-schedule-coverage-collapse/grader.py && echo "grader.py OK"
source horizon_env/bin/activate
horizon check-anatomy tasks/oncall-schedule-coverage-collapse 2>&1
horizon check-quality tasks/oncall-schedule-coverage-collapse 2>&1 | tail -30
```

## Phase 1 — Push
```bash
horizon tasks push tasks/oncall-schedule-coverage-collapse 2>&1   # record Version: NNN
```

## Phase 2 — Oracle (poll, don't trust --wait)
```bash
horizon tasks validate -m hosted -a oracle tasks/oncall-schedule-coverage-collapse 2>&1
BUILD_ID="val-9ee1af19-<TIMESTAMP>"
RESULT_PATH="tasks/oncall-schedule-coverage-collapse/.validation/${BUILD_ID}/result.json"
for attempt in $(seq 1 30); do
  sleep 60
  horizon tasks validate-logs -a oracle tasks/oncall-schedule-coverage-collapse >/dev/null 2>&1
  STATUS=$(python3 -c "import json; print(json.load(open('$RESULT_PATH')).get('status','unknown'))" 2>/dev/null)
  if [ "$STATUS" != "running" ] && [ -n "$STATUS" ]; then
    echo "Oracle done after ${attempt} min, status=$STATUS"; cat "$RESULT_PATH" | python3 -m json.tool; break
  fi
  echo "[poll ${attempt}/30] status=$STATUS"
done
```
Pass: `1.0` + both `1`. Run as a background bash task. Else → `oncall-validation-debugger`.

## Phase 2.5 — Live red-team
Invoke `horizon-agentic-reviewer` on UUID `<TASK_ID>`. Confirm (probes in `oncall-task-guardian` Step 4b): bleater-env reachable by `ubuntu`; the broken state is live; all 3 annotation/`slot-pinner` drift sources exist (kube-system + bleater-env + canary); `/etc/oncall/<artefact>.txt` agent-readable but `/opt/<grader-artefact>.txt` not; drift names + exact metadata keys NOT leaked in baseline; a no-op agent fails both subscores. Auto-invoked. BLOCKING → fix → re-loop. Skip only for a pure numeric tweak. If the VM is unreachable, note the risk and proceed.

**Tell the user (verbatim, the ONLY user-facing message):**
> "Oracle passed at version N and live red-team is [clean | skipped: reason]. Please trigger a rollout batch for version N in the Horizon UI."

## Phase 3 — Rollout analysis
```bash
horizon rollouts pull --version NNN tasks/oncall-schedule-coverage-collapse 2>&1
```
Poll until ≥5; read 2 transcripts (1 pass, 1 fail) before the script.
```python
import json, glob
from collections import defaultdict
VERSION="NNN"; TASK="oncall-schedule-coverage-collapse"
files=glob.glob(f"tasks/{TASK}/.rollouts/v{VERSION}/*.json")
scores=[]; sub=defaultdict(list)
for f in files:
    d=json.load(open(f)); scores.append(d["score"])
    try:
        s=json.loads(d.get("grade_result","{}")).get("subscores",{})
        for k,v in s.items(): sub[k].append(v)
    except Exception: pass
if not scores: print("No rollouts — trigger a batch first")
else:
    n=len(scores); avg=sum(scores)/n
    print(f"N={n} avg={avg:.3f} pass_rate={sum(1 for s in scores if s>=0.99)/n:.0%}")
    for k,vs in sub.items():
        vals=sorted(set(vs)); print(f"  {k}: {'DEAD' if len(vals)==1 else 'varies'} {vals} mean={sum(vs)/len(vs):.2f}")
    half=sum(1 for s in scores if abs(s-0.5)<0.01)
    if avg>=<TARGET_MEAN>: print("HIGH avg — oncall-score-tuner")
    elif any(len(set(v))==1 for v in sub.values()): print("DEADWEIGHT — oncall-score-tuner")
    elif half==0: print("CORRELATION COLLAPSE: no 0.5 — oncall-score-tuner (lean on overrides + gap-rule/anchor decorrelators)")
    else: print("DONE: avg < <TARGET_MEAN>, both vary, decorrelated")
```

| Result | Action |
|---|---|
| avg < <TARGET_MEAN>, both vary, some 0.5 | **DONE** |
| a subscore deadweight | `oncall-score-tuner` |
| no 0.5 (correlation) | `oncall-score-tuner` — overrides/gap-rule/anchor decorrelators, not a shared-drift lever |
| `monitoring_governance` always 0 (Oracle 1.0) | bleater-env reachability / install-id mirror → re-check Dockerfile/setup |
| `coverage_restored` always 0 (Oracle 1.0) | a drift source out-races the window → cadence/floor |
| <5 rollouts | poll again |

## Decision rules (Phase 2)
| Oracle | Action |
|---|---|
| 1.0, both=1 | Phase 2.5 |
| `feedback: null`/0 | `oncall-validation-debugger` Branch A (No-Op isolate) |
| 0, non-null | Branch E |
| 0.5, coverage=0 | Branch C |
| 0.5, governance=0 | Branch D |
| exception | Branch F |

## When to stop for human review
5 cycles without progress; a tuner fix would violate a hard constraint; governance stuck at 0 despite a correct-looking solution (run reviewer for a live bleater-env/install-id probe); avg stuck > <TARGET_MEAN>; two cycles worsened correlation.

## Skill dependency map
```
oncall-iteration-loop (orchestrator)
├── oncall-task-guardian        Phase 0 pre-push, Phase 1 push, Phase 5 monitoring
├── oncall-validation-debugger  Phase 2 failure interpretation
├── oncall-score-tuner          Phase 3 score/variance/correlation diagnosis
└── horizon-agentic-reviewer    Phase 2.5 live red-team
```
Diagnose → propose → approve → apply: the orchestrator delegates editing to the specialist that diagnosed the symptom, then applies via `oncall-task-guardian`.