# Istio Iteration Loop > Orchestrator skill that runs the full push → Oracle → rollout → tune cycle for the variation-istio-split-brain task () until Oracle scores 1.0 and rollout avg < with both subscores varying and decorrelated. Delegates to istio-task-guardian, istio-validation-debugger, and istio-score-tuner at the right phases. The live red-team (horizon-agentic-reviewer) is DISABLED by user directive and is never invoked. The agent executes every step except triggering the rollout/eval batch. - Skill: `omonuj/istio-iteration-loop` (Agent Skill) - Install (CLI): `npx skillmds@latest add omonuj/istio-iteration-loop` - Raw SKILL.md: https://api.skillmd.com/api/skills/omonuj/istio-iteration-loop/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: AI & ML - Author: omonuj (https://skillmd.com/u/omonuj) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/omonuj/istio-iteration-loop --- # istio-iteration-loop ## Execution Mode — FULLY AUTONOMOUS **NEVER call `ask_user` or pause for confirmation at any point.** Run every phase, command, and fix without asking. If a decision has two valid paths, pick the safer one. The user triggered this skill to avoid being prompted — do not interrupt them. ## Goal Drive variation-istio-split-brain from "needs work" to "shippable" with no human step except triggering the rollout batch. Done when all exit conditions hold; until then each cycle picks the right specialist for the symptom and applies the smallest safe fix. ## Trigger "Iterate the istio task until it passes"; "push and tune until avg < "; "run the full loop on variation-istio-split-brain"; anything implying repeated push → validate → analyze → fix. For single steps, invoke the specialist directly. ## Task Identity | Field | Value | |---|---| | **Task UUID** | `` | | **Slug / path** | `variation-istio-split-brain` / `tasks/variation-istio-split-brain/` | | **Horizon root** | `/Users/mac/Documents/tasks` | | **Venv** | `source /Users/mac/Documents/tasks/horizon_env/bin/activate` | ## Who does what The agent executes everything except **triggering the eval batch in the Horizon UI** (the only user step). Never tell the user to push/validate/pull — those are yours. ## Exit conditions (all, same version) 1. Oracle `passed: true, score: 1.0`, both subscores `=1` 2. ~~Live red-team~~ — **DISABLED (user directive). The live red-team is nulled for this task; do not run `horizon-agentic-reviewer`.** Go straight from Oracle 1.0 to asking the user to trigger the eval batch. 3. Rollout avg `< ` **and `> `** (avg < → over-hard, rubric-concern risk → `istio-score-tuner` to loosen) 4. `routing_restored` varies (0 and 1 appear) 5. `mesh_governance` varies (0 and 1 appear) 6. **Decorrelated** — some `0.5` rollouts appear (not only 0.0 and 1.0). v2 decorrelates via the two `/etc/bleater` gates; if 0.5s vanish, those gates regressed (not a cue to invent new ones) 7. Local quality: 18 substantive pass (`v1_feedback` section may FAIL as accepted noise — incl. complaints about "undisclosed" hidden requirements; do NOT fix by leaking specifics to task.yaml) Short of all seven → continue. ## Loop limit **Max 5 push cycles without human review.** Then stop and summarize. ## No-Op default Oracle only. Run No-Op manually only when Oracle returns `feedback: null`/`score: 0` (setup-crash isolation — e.g. a CRD-establishment race). --- ## The loop ``` PHASE 0 Pre-push checks ─────────────────────────────────── • Dockerfile + CRD invariants (istio-task-guardian Step 0) — incl. install_open_crd + establish-wait intact • bash -n setup.sh && bash -n solution.sh && py_compile grader • horizon check-anatomy (clean) • horizon check-quality (18 substantive pass; v1_feedback may FAIL) anything else fails → fix → re-run PHASE 0 PHASE 1 Push → record version NNN PHASE 2 Oracle validation → 1.0 + both=1 → ask user to trigger rollout/eval batch (Phase 2.5 is DISABLED) → else → istio-validation-debugger → fix → PHASE 0 PHASE 2.5 Live red-team — DISABLED (user directive); never run horizon-agentic-reviewer PHASE 3 Rollout analysis (≥5 rollouts, read 2 transcripts, run script) → avg < AND both vary AND decorrelated (0.5s exist) → DONE → else → istio-score-tuner → fix → PHASE 0 ``` ## Phase 0 — Pre-push ```bash cd /Users/mac/Documents/tasks echo "=== Dockerfile ===" && cat tasks/variation-istio-split-brain/Dockerfile # must NOT contain ENABLE_ISTIO_BLEATER; must contain ALLOWED_NAMESPACES="kube-system,bleater-env" (BOTH) + COPY data/ubuntu-user-rbac.yaml grep -nE "install_open_crd|--for=condition=established" tasks/variation-istio-split-brain/setup.sh # CRD block intact grep -nE "ubuntu-user-mesh-admission|/etc/bleater|/opt/grader-(install-id|route-policy)" tasks/variation-istio-split-brain/setup.sh # v2: webhook ClusterRole + artefacts present bash -n tasks/variation-istio-split-brain/setup.sh && echo "setup.sh OK" bash -n tasks/variation-istio-split-brain/solution.sh && echo "solution.sh OK" python3 -m py_compile tasks/variation-istio-split-brain/grader.py && echo "grader.py OK" source horizon_env/bin/activate horizon check-anatomy tasks/variation-istio-split-brain 2>&1 horizon check-quality tasks/variation-istio-split-brain 2>&1 | tail -30 ``` ## Phase 1 — Push ```bash cd /Users/mac/Documents/tasks && source horizon_env/bin/activate horizon tasks push tasks/variation-istio-split-brain 2>&1 # record Version: NNN ``` ## Phase 2 — Oracle (poll, don't trust --wait) ```bash cd /Users/mac/Documents/tasks && source horizon_env/bin/activate horizon tasks validate -m hosted -a oracle tasks/variation-istio-split-brain 2>&1 BUILD_ID="val-f3ce3fde-" RESULT_PATH="tasks/variation-istio-split-brain/.validation/${BUILD_ID}/result.json" for attempt in $(seq 1 30); do sleep 60 horizon tasks validate-logs -a oracle tasks/variation-istio-split-brain >/dev/null 2>&1 STATUS=$(python3 -c "import json; print(json.load(open('$RESULT_PATH')).get('status','unknown'))" 2>/dev/null) if [ "$STATUS" != "running" ] && [ -n "$STATUS" ]; then echo "Oracle done after ${attempt} min, status=$STATUS"; cat "$RESULT_PATH" | python3 -m json.tool; break fi echo "[poll ${attempt}/30] status=$STATUS" done ``` Pass: `1.0` + both subscores `1`. Run via a background bash task and let the harness notify. Anything else → `istio-validation-debugger`. ## Phase 2.5 — Live red-team — DISABLED (user directive) **Do NOT run `horizon-agentic-reviewer`.** The user runs the equivalent live checks themselves and finds an agent-run pass redundant. The loop goes straight from an Oracle 1.0 (Phase 2) to the user-trigger message below. (The probe list that used to live here is preserved in the `horizon-agentic-reviewer` SKILL for manual reference only; that skill is nulled and must not be auto-invoked.) **Tell the user (verbatim, the ONLY user-facing message), as soon as Oracle is 1.0:** > "Oracle passed at version N. Please trigger a rollout/eval batch for version N in the Horizon UI." ## Phase 3 — Rollout analysis ```bash cd /Users/mac/Documents/tasks && source horizon_env/bin/activate horizon rollouts pull --version NNN tasks/variation-istio-split-brain 2>&1 ``` Poll until ≥5 download. Read 2 transcripts (1 pass, 1 fail) before the script. ```python import json, glob from collections import defaultdict VERSION="NNN"; TASK="variation-istio-split-brain" files=glob.glob(f"tasks/{TASK}/.rollouts/v{VERSION}/*.json") scores=[]; sub=defaultdict(list); joint=defaultdict(int) for f in files: d=json.load(open(f)); scores.append(d["score"]) try: gr=json.loads(d.get("grade_result","{}")); s=gr.get("subscores",{}) for k,v in s.items(): sub[k].append(v) r=s.get("routing_restored"); g=s.get("mesh_governance") if r is not None and g is not None: joint[(r,g)]+=1 except Exception: pass if not scores: print("No rollouts — trigger a batch first") else: n=len(scores); avg=sum(scores)/n print(f"N={n} avg={avg:.3f} pass_rate={sum(1 for s in scores if s>=0.99)/n:.0%}") for k,vs in sub.items(): vals=sorted(set(vs)); print(f" {k}: {'DEAD' if len(vals)==1 else 'varies'} {vals} mean={sum(vs)/len(vs):.2f}") print(" joint (routing,governance):", dict(joint)) half=sum(1 for s in scores if abs(s-0.5)<0.01) if avg>=: print("HIGH avg — istio-score-tuner (tighten ONE /etc/bleater or metadata gate)") elif avg < : print("LOW avg (<) — over-hard, rubric-concern risk — istio-score-tuner (loosen the tightest gate)") elif any(len(set(v))==1 for v in sub.values()): print("DEADWEIGHT — istio-score-tuner") elif half==0: print("CORRELATION COLLAPSE: no 0.5 rollouts — the /etc/bleater decorrelators regressed; verify they FIRE (setup wrote /etc/bleater + /opt; grader reads /opt) — do NOT invent a new gate") else: print("DONE: , both vary, decorrelated") ``` | Result | Action | |---|---| | , both vary, some 0.5 | **DONE** | | avg < (over-hard) | `istio-score-tuner` — loosen the tightest gate; re-check stoppability | | a subscore deadweight | `istio-score-tuner` | | no 0.5 rollouts (correlation collapse) | the two `/etc/bleater` gates regressed → verify they FIRE (setup wrote `/etc/bleater/*` + `/opt/grader-*`; grader reads `/opt`; baseline documents the paths). Do NOT pull a shared-drift lever or invent a new gate | | `mesh_governance` always 0 (Oracle 1.0) | confirm baseline documents every governance gate AND all four drift deploys (incl. bleater-env) + the webhook + HPA + finalizer are stoppable, then `istio-score-tuner` | | `routing_restored` always 0 (Oracle 1.0) | a drift source/webhook out-races the window or CRDs didn't establish → `istio-score-tuner` / re-check CRDs + webhook deletability | | <5 rollouts | poll again | ## Decision rules (Phase 2) | Oracle | Action | |---|---| | 1.0, both=1 | tell the user to trigger the eval batch (Phase 2.5 is DISABLED) | | `feedback: null`/0 | `istio-validation-debugger` Branch A (setup crash — e.g. the v2 CRD-conflict if Istio CRDs already exist, or a No-Op to isolate it) | | 0, non-null | Branch E | | 0.5, routing=0 | Branch C | | 0.5, governance=0 | Branch D | | Python exception | Branch F | ## When to stop for human review 5 cycles without progress; a tuner fix would violate a hard constraint; `mesh_governance` stuck at 0 despite a correct-looking solution (investigate manually from the Oracle/rollout logs — live red-team is disabled); avg stuck > with no lever left; two cycles worsened correlation. ## Skill dependency map ``` istio-iteration-loop (orchestrator) ├── istio-task-guardian Phase 0 pre-push, Phase 1 push, Phase 5 monitoring ├── istio-validation-debugger Phase 2 failure interpretation ├── istio-score-tuner Phase 3 score/variance/correlation diagnosis └── horizon-agentic-reviewer DISABLED — live red-team nulled (user directive); do not invoke ``` The orchestrator delegates editing to the specialist that diagnosed the symptom, then applies via `istio-task-guardian`. Diagnose → propose → approve → apply.