Lane Stall Watchdog — One Iteration
One watchdog pass for a lane session (a factory lane running
under moai cc / moai glm / moai codex). The law — the ladder, the
outcome transitions, the gate inventory, the view–SSOT rule, the record
formats — lives in .claude/rules/moai/workflow/auto-semantics.md. Read
that doctrine before the first pass and whenever a step below needs its
detail; this body is the executable procedure.
0. Never prompt
The watchdog never asks the operator a question and never answers an
approval gate on anyone's behalf. Everything that would have been a question
becomes either a resolved judgment (the ladder) or an explicit wait record.
A keep-set gate — environment-impossible, operator-held, or an irreversible
external-shared operation — is never satisfied here: the recheck reads the
gate surface, and finding no change, it re-records the wait and yields.
1. Measure progress
Three channels (doctrine §3–4):
- HEAD:
git rev-parse --short HEAD in the lane's own worktree.
- evidence mtimes: the newest mtime across
.moai/reports/<card-id>/
and the active SPEC's progress.md.
- integration-window state:
moai integration status.
Compare against the previous snapshot at
.moai/state/watchdog/<card-id>.json (fields: observed_at, head_sha,
evidence_mtime_max, window_state). The lane is stalled only when ALL
THREE channels are unchanged AND the previous observation is at least N
minutes old. N defaults to 15 minutes and is a per-invocation parameter —
never a config key. No previous snapshot → record a fresh one, no stall
verdict (fail-open first observation). One channel moving is progress —
evidence mtime advancing without a commit included: end the iteration with
a one-line status — unless the lane itself is parked on a delegate (§3.5):
then the movement is the delegate's, and the deliverable read in §3.5
decides, not this rule. Write the new snapshot after the verdict.
2. Classify the cause
| Cause |
Signature |
| awaited-judgment |
the lane stopped where a judgment was needed (a gate, a verdict, a choice) and no reply arrived |
| blocked-by |
the card waits on a predecessor card; the predecessor's landing state is the open question |
| shell-error |
the lane halted on an unadjudicated command failure (a non-zero exit with no recorded disposition) |
| accidental-stop |
the session died or was interrupted mid-card — an API error (a 429) ends the turn with no further model action — and no deliberate stop was recorded |
| awaited-delegate |
the lane parked on delegated work — a spawned agent's report, a background run's completion — and the report or notice has not arrived; an available idle notice that promised a later report counts, because that agent sends nothing more unless it is messaged |
No stall / progress detected → end the iteration.
3. Apply the remedy
3.1 awaited-judgment — the decision ladder
Resolve through the ladder IN ORDER (doctrine §6–7). Availability failures
fail-open downward; negative or inconclusive verdicts never auto-proceed,
and at an authority gate they fail closed.
- Disk evidence — read the deciding artifact directly: audit verdict
files, the card's progress record, plan-audit verdicts, evidence paths.
Evidence shows proceed → resume. Evidence shows do-not-proceed → record
the wait; never proceed against evidence. Unreadable or absent → step 2.
- Decision board — run
moai decision read --scope card:<id> (the
card's rulings plus every standing ruling; doctrine §11: the append-only
board under the moai home, keyed by the project). A recorded judgment →
follow it. A judgment that says wait → explicit wait record (with an id;
doctrine §14). A record whose resolves names an open wait ends that
wait. board=absent / board=empty → step 3; never wait on the board
being filled. While a wait on the leader stays open, keep the one-shot
short recheck of doctrine §14 armed.
- Audit cross — run one
moai MCP audit tool (codex_audit,
glm_audit, claude_audit, or audit_multi) for an independent second
opinion. Positive → proceed per verdict. Negative → fail-closed: no
proceed; record + escalate to step 5. Tool absent → step 4. Inconclusive
→ record it, → step 4.
- jev_ask — only while
workflow.jev.enabled is true; a bounded
proceed / retry / wait judgment with repeatable, side-effect-free
parameters; never for completion, merge, or queue outcomes. Gated off or
unavailable → step 5 (fail-open by design).
- Lead chat — last resort. Record the explicit wait (§5 below), then
yield.
Write a decision record naming whichever step resolved the judgment (§5
below).
3.2 blocked-by
Read the blocking predecessor's on-disk evidence directly (its
.moai/reports/<card-id>/ path, its progress record). Evidence shows the
predecessor landed → resume. Otherwise record an explicit wait: reason +
predecessor id + recheck point. Where the predecessor was dropped, the wait
record names the queue relation-removal verb as the operator escape — the
lane does NOT run it (queue mutation is prohibited). The queue-level pickup
filter is another surface's decision; this remedy only reads.
3.3 shell-error
Adjudicate the error: transient (network blip, lock contention, flaky
timing) vs determinate (missing remote, bad ref, usage error).
- transient → retry at most 3 times, recording the error and the retry
count each time; retries exhausted → structured blocker report.
- determinate → ZERO retries; straight to a structured blocker report with
the verbatim error text.
Never halt indefinitely on an unadjudicated error; never loop silently.
3.4 accidental-stop
Resume by checkpoint return: use the session-handoff resume message with the
worktree-anchored Block 0 (git rev-parse --show-toplevel → this worktree),
resuming from the last recorded checkpoint in the card's progress record.
Never restart the card from zero.
3.5 awaited-delegate
Read the delegate's deliverable on disk — the artifacts it was asked to write,
its output file, its commits — against the delegation record (what was asked,
where it was to land). Never read the absence of its message as the absence of
its work.
- Deliverable present and consistent with the delegation → resume and consume
it; never wait for the report or the notice. An agent that went
available
sends nothing more unless it is messaged, so the report may never come.
- Deliverable absent or partial and the delegate evidently ended (its idle or
completion notice arrived, or its process is gone) → re-delegate with the
partial state, or resume the delegate with
SendMessage only while it is a
live teammate. Never address a teammate stopped with TaskStop by name — one
message revives it as an ownerless writer
(cross-session-messaging.md § Rules). Record which.
- Deliverable absent and the evidence still moving (the delegate is working) →
explicit wait whose recheck point is the next cron fire.
- Deliverable absent, no sign the delegate ended, and the evidence unchanged
across two consecutive fires → structured blocker naming the delegate, what
was asked of it, and the evidence read; do not wait a third time.
4. Decision board protocol
- SSOT: the disk board (doctrine §11). Append-only, one record per
line; never rewrite board history.
- Views: render state with the runner's native task tool — TaskCreate /
TaskUpdate on the Claude runner,
update_plan on codex — and read
judgments ONLY from disk. A view is never the SSOT; a long-form plan
document is never the board.
- Codex step ⑤: no session-messaging tool exists there — cross-harness
notice rides the broker (
session_msg_register / session_msg_send +
poll) and the queue-on-disk delegation channel; the wait reason is
recorded on disk either way.
5. Decision records
Every autonomous gate transition and every ladder-resolved judgment writes
ONE grep-able line on the disk board:
decision record: decided_by=<runner+role> evidence_refs=<paths+verdict-ids> ladder_path=<step>
decided_by names the deciding runner and role; evidence_refs names the
artifact paths and verdict ids the decision rests on; ladder_path names
the ladder step (①–⑤) or the gate row that resolved it. The sync audit
re-reads these records — write them so a stranger can re-verify.
6. Composition boundary
The queue-level pickup decision — the pickup filter that excludes unrelated
or blocked cards before a pick — belongs to the queue surface's owner. This
watchdog is queue-readonly: it reads evidence and state surfaces, performs
zero queue mutation, and never picks, drops, edits, or unrelates anything.
7. Output shape — escalate or record
Every iteration ends in exactly one of:
- resumed — progress detected or a judgment resolved; say which step,
and cite the decision record line.
- explicit wait — the wait record line and its recheck point, nothing
more; end the iteration (no idle-spin).
- structured blocker — determinate failure or retries exhausted: the
verbatim error, what was attempted, the evidence read, and the decision
owed. Return it to the orchestrating session; do not retry past the
ceiling; do not prompt anyone.
Boundaries (hard)
- Never prompt the operator; never answer an approval gate. Keep-set gates
(environment-impossible, operator-held, irreversible external-shared
operations) are rechecked and yielded to, never satisfied by this skill.
- Queue read-only (§6).
- One writer per tree: the watchdog writes only its own snapshot, its own
evidence records, and the disk board.
- Verification is lane-local; no background load.
- Hook-independent: awaken-tick only; no hook surface. The scheduler that
fires the awaken is the lane's standing recheck cron (doctrine §5.1), armed
at card intake — this skill is its prompt, not its owner.
1---2name: moai-lane-watchdog3description: >4 One lane stall watchdog iteration: measure lane progress on three5 channels, classify the stall cause, apply the remedy or resolve the6 judgment through the decision ladder — instead of waiting on a reply —7 then record the outcome and report. Reads the unified --auto doctrine at8 .claude/rules/moai/workflow/auto-semantics.md for the ladder, the gate9 inventory, and the record formats.10when_to_use: >11 Use when a loop iteration re-awakens a lane session (the awaken rule: the12 watchdog pass runs BEFORE card work resumes), when a lane shows no13 progress across the watch window, or when a lane needs a judgment to14 proceed and no reply has arrived.15license: Apache-2.016compatibility: Designed for Claude Code and Codex sessions17allowed-tools: Read, Grep, Glob, Bash(git rev-parse:*), Bash(git status:*), Bash(git log:*), Bash(stat:*), Bash(ls:*), Bash(test:*), Bash(moai integration status:*)18user-invocable: false19metadata:20 source: https://skillmd.com/skills/tuyv/moai-lane-watchdog21 version: "1.0.0"22 category: "workflow"23 status: "active"24 tags: "watchdog, stall, lane, ladder, decision-ladder, unattended, auto"2526# MoAI Extension: Progressive Disclosure27progressive_disclosure:28 enabled: true29---3031# Lane Stall Watchdog — One Iteration3233One watchdog pass for a lane session (a factory lane running34under `moai cc` / `moai glm` / `moai codex`). The law — the ladder, the35outcome transitions, the gate inventory, the view–SSOT rule, the record36formats — lives in `.claude/rules/moai/workflow/auto-semantics.md`. Read37that doctrine before the first pass and whenever a step below needs its38detail; this body is the executable procedure.3940## 0. Never prompt4142The watchdog never asks the operator a question and never answers an43approval gate on anyone's behalf. Everything that would have been a question44becomes either a resolved judgment (the ladder) or an explicit wait record.45A keep-set gate — environment-impossible, operator-held, or an irreversible46external-shared operation — is never satisfied here: the recheck reads the47gate surface, and finding no change, it re-records the wait and yields.4849## 1. Measure progress5051Three channels (doctrine §3–4):5253- **HEAD**: `git rev-parse --short HEAD` in the lane's own worktree.54- **evidence mtimes**: the newest mtime across `.moai/reports/<card-id>/`55 and the active SPEC's `progress.md`.56- **integration-window state**: `moai integration status`.5758Compare against the previous snapshot at59`.moai/state/watchdog/<card-id>.json` (fields: `observed_at`, `head_sha`,60`evidence_mtime_max`, `window_state`). The lane is stalled only when ALL61THREE channels are unchanged AND the previous observation is at least N62minutes old. N defaults to 15 minutes and is a per-invocation parameter —63never a config key. No previous snapshot → record a fresh one, no stall64verdict (fail-open first observation). One channel moving is progress —65evidence mtime advancing without a commit included: end the iteration with66a one-line status — unless the lane itself is parked on a delegate (§3.5):67then the movement is the delegate's, and the deliverable read in §3.568decides, not this rule. Write the new snapshot after the verdict.6970## 2. Classify the cause7172| Cause | Signature |73|---|---|74| **awaited-judgment** | the lane stopped where a judgment was needed (a gate, a verdict, a choice) and no reply arrived |75| **blocked-by** | the card waits on a predecessor card; the predecessor's landing state is the open question |76| **shell-error** | the lane halted on an unadjudicated command failure (a non-zero exit with no recorded disposition) |77| **accidental-stop** | the session died or was interrupted mid-card — an API error (a 429) ends the turn with no further model action — and no deliberate stop was recorded |78| **awaited-delegate** | the lane parked on delegated work — a spawned agent's report, a background run's completion — and the report or notice has not arrived; an `available` idle notice that promised a later report counts, because that agent sends nothing more unless it is messaged |7980No stall / progress detected → end the iteration.8182## 3. Apply the remedy8384### 3.1 awaited-judgment — the decision ladder8586Resolve through the ladder IN ORDER (doctrine §6–7). Availability failures87fail-open downward; negative or inconclusive verdicts never auto-proceed,88and at an authority gate they fail closed.89901. **Disk evidence** — read the deciding artifact directly: audit verdict91 files, the card's progress record, plan-audit verdicts, evidence paths.92 Evidence shows proceed → resume. Evidence shows do-not-proceed → record93 the wait; never proceed against evidence. Unreadable or absent → step 2.942. **Decision board** — run `moai decision read --scope card:<id>` (the95 card's rulings plus every standing ruling; doctrine §11: the append-only96 board under the moai home, keyed by the project). A recorded judgment →97 follow it. A judgment that says wait → explicit wait record (with an id;98 doctrine §14). A record whose `resolves` names an open wait ends that99 wait. `board=absent` / `board=empty` → step 3; never wait on the board100 being filled. While a wait on the leader stays open, keep the one-shot101 short recheck of doctrine §14 armed.1023. **Audit cross** — run one `moai` MCP audit tool (`codex_audit`,103 `glm_audit`, `claude_audit`, or `audit_multi`) for an independent second104 opinion. Positive → proceed per verdict. Negative → fail-closed: no105 proceed; record + escalate to step 5. Tool absent → step 4. Inconclusive106 → record it, → step 4.1074. **jev_ask** — only while `workflow.jev.enabled` is true; a bounded108 proceed / retry / wait judgment with repeatable, side-effect-free109 parameters; never for completion, merge, or queue outcomes. Gated off or110 unavailable → step 5 (fail-open by design).1115. **Lead chat** — last resort. Record the explicit wait (§5 below), then112 yield.113114Write a decision record naming whichever step resolved the judgment (§5115below).116117### 3.2 blocked-by118119Read the blocking predecessor's on-disk evidence directly (its120`.moai/reports/<card-id>/` path, its progress record). Evidence shows the121predecessor landed → resume. Otherwise record an explicit wait: reason +122predecessor id + recheck point. Where the predecessor was dropped, the wait123record names the queue relation-removal verb as the operator escape — the124lane does NOT run it (queue mutation is prohibited). The queue-level pickup125filter is another surface's decision; this remedy only reads.126127### 3.3 shell-error128129Adjudicate the error: transient (network blip, lock contention, flaky130timing) vs determinate (missing remote, bad ref, usage error).131132- transient → retry at most 3 times, recording the error and the retry133 count each time; retries exhausted → structured blocker report.134- determinate → ZERO retries; straight to a structured blocker report with135 the verbatim error text.136137Never halt indefinitely on an unadjudicated error; never loop silently.138139### 3.4 accidental-stop140141Resume by checkpoint return: use the session-handoff resume message with the142worktree-anchored Block 0 (`git rev-parse --show-toplevel` → this worktree),143resuming from the last recorded checkpoint in the card's progress record.144Never restart the card from zero.145146### 3.5 awaited-delegate147148Read the delegate's deliverable on disk — the artifacts it was asked to write,149its output file, its commits — against the delegation record (what was asked,150where it was to land). Never read the absence of its message as the absence of151its work.152153- Deliverable present and consistent with the delegation → resume and consume154 it; never wait for the report or the notice. An agent that went `available`155 sends nothing more unless it is messaged, so the report may never come.156- Deliverable absent or partial and the delegate evidently ended (its idle or157 completion notice arrived, or its process is gone) → re-delegate with the158 partial state, or resume the delegate with `SendMessage` only while it is a159 live teammate. Never address a teammate stopped with `TaskStop` by name — one160 message revives it as an ownerless writer161 (`cross-session-messaging.md` § Rules). Record which.162- Deliverable absent and the evidence still moving (the delegate is working) →163 explicit wait whose recheck point is the next cron fire.164- Deliverable absent, no sign the delegate ended, and the evidence unchanged165 across two consecutive fires → structured blocker naming the delegate, what166 was asked of it, and the evidence read; do not wait a third time.167168## 4. Decision board protocol169170- **SSOT**: the disk board (doctrine §11). Append-only, one record per171 line; never rewrite board history.172- **Views**: render state with the runner's native task tool — TaskCreate /173 TaskUpdate on the Claude runner, `update_plan` on codex — and read174 judgments ONLY from disk. A view is never the SSOT; a long-form plan175 document is never the board.176- **Codex step ⑤**: no session-messaging tool exists there — cross-harness177 notice rides the broker (`session_msg_register` / `session_msg_send` +178 poll) and the queue-on-disk delegation channel; the wait reason is179 recorded on disk either way.180181## 5. Decision records182183Every autonomous gate transition and every ladder-resolved judgment writes184ONE grep-able line on the disk board:185186```text187decision record: decided_by=<runner+role> evidence_refs=<paths+verdict-ids> ladder_path=<step>188```189190`decided_by` names the deciding runner and role; `evidence_refs` names the191artifact paths and verdict ids the decision rests on; `ladder_path` names192the ladder step (①–⑤) or the gate row that resolved it. The sync audit193re-reads these records — write them so a stranger can re-verify.194195## 6. Composition boundary196197The queue-level pickup decision — the pickup filter that excludes unrelated198or blocked cards before a pick — belongs to the queue surface's owner. This199watchdog is queue-readonly: it reads evidence and state surfaces, performs200zero queue mutation, and never picks, drops, edits, or unrelates anything.201202## 7. Output shape — escalate or record203204Every iteration ends in exactly one of:205206- **resumed** — progress detected or a judgment resolved; say which step,207 and cite the decision record line.208- **explicit wait** — the wait record line and its recheck point, nothing209 more; end the iteration (no idle-spin).210- **structured blocker** — determinate failure or retries exhausted: the211 verbatim error, what was attempted, the evidence read, and the decision212 owed. Return it to the orchestrating session; do not retry past the213 ceiling; do not prompt anyone.214215## Boundaries (hard)2162171. Never prompt the operator; never answer an approval gate. Keep-set gates218 (environment-impossible, operator-held, irreversible external-shared219 operations) are rechecked and yielded to, never satisfied by this skill.2202. Queue read-only (§6).2213. One writer per tree: the watchdog writes only its own snapshot, its own222 evidence records, and the disk board.2234. Verification is lane-local; no background load.2245. Hook-independent: awaken-tick only; no hook surface. The scheduler that225 fires the awaken is the lane's standing recheck cron (doctrine §5.1), armed226 at card intake — this skill is its prompt, not its owner.