Leaf Worker Hardening

Leaf Worker Hardening — Stop Fire-and-Forget

When Queen dispatches a delegate_task (or external opencode/codex via terminal bg) as a leaf worker with a research/spike goal, the most common failure is NOT a crash — it's a 14-second stub that reports "completed" after 1-3 tool calls (skill_view + ls), without ever running the curl/write/grep that would actually answer the question.

When this bites

• goal is research/spike-shaped ("调研 X 反爬", "找 github 借鉴项目", "评估 Y 可行性")
• context is ≥1KB of structured bullets
• Worker model is anchor (router) or kilocode/poolside/laguna-xs-2.1:free — short prompts, both tend to satisfy a "decompose then start" plan and report done before executing

Hard rules (v29.8.2)

1. ≤6 tool calls hard cap per leaf worker goal. If the goal needs ≥7, split into 2-3 sub-tasks (one finding each).
2. Goal must contain a "## 工具硬清单" section enumerating every tool_call the worker should make, in order. The list MUST end with at least one patch / write_file / edit tool call when the goal is "fix X" or "apply Y". Workers without an explicit write step treat "traced the path" + "verified the fix would work" as completion and report done without ever editing disk.
3. For patch/fix goals: require an explicit git diff or file-content verification step at the end. Worker must terminal: cd <repo> && git diff <path> to prove the patch landed before reporting done. "Proposed fix would work" is not acceptance — only on-disk evidence is.

   ## 工具硬清单 (≤6 个, 顺序执行, 不允许跳)
   1. terminal: `curl -sIL -A "..." <url>` → 报 HTTP status
   2. terminal: `curl -sL -A "..." <url> -o /tmp/x.html` → 报 size_download
   3. write_file: `/Users/henry/.../analysis/spike.md` with evidence
   4. terminal: `wc -c /tmp/x.html` → verify
   

4. Never trust a 14s "completed" summary. Verify evidence files exist via ls -la / wc -c before claiming done. If the worker said "completed" but files are empty, the worker fire-and-forget'd — re-dispatch or escalate to Queen direct execution.
5. Queen direct vs delegate_task decision matrix:
   • Spike / unknown feasibility / <30min → Queen direct (≤2KB output naturally fits SOUL §Token 纪律; no worker round-trip latency; no fire-and-forget risk).
   • Multi-perspective review / ≥3 independent findings → delegate_task batch with one worker per finding, each with its own hard tool checklist.
   • Long-running >30min / cross-session → hermes kanban (gateway daemon, poll with queen_poll.py).
6. Honest reporting: if a worker reports "completed" but evidence is missing, tell the user "worker fire-and-forget, switching to direct execution" — do NOT pretend it succeeded.

Symptom: "用户看不到 worker 在干啥" (2026-08-11)

Worker 完成 ≠ 用户感知 "在干"。即使 worker 真的在跑 1-3 分钟, 用户视角只看见 Queen 静默等。二次伤害:

1. 用户连发 "进展如何 / 在干吗 / 你给我一个 prompt" → 每次都是新的 user turn 插话
2. Queen 即使切回直跑, 单 turn ≤3 工具 + ≤2KB 输出 = 体感"慢 + 卡"
3. 用户误判 "Queen 根本不工作" → 提出 "我去 worker body 干" / "新 session"

Mitigation pattern (Queen 侧):

根因: Queen 把"派单 = 进度可见"误当成"派单 = 工作在进行"。两者不等价。Worker fire-and-forget 后, 用户的耐心窗口 <30s, 超时就切直跑, 不等。

AMAC 反爬 = 0 的实战发现 (本 session 意外收获)

调研发现 AMAC 真实入口不在 www.amac.org.cn (404 占位), 在 gs.amac.org.cn 子域 + /amac-infodisc/api/... JSON 接口。无反爬, 只需 UA + Referer。这条本身不是 skill 内容 (一次性发现), 但印证了:

• 政府/事业单位站点真实 API 经常挂在子域, 主域只 serve 营销页/404 占位
• 调研 spike 永远值得先跑 1-2 个 curl, 不要凭 URL 推断架构

反 pattern: 把 "用户看不到在干" 当成 "Queen 坏掉"

本次会话 Queen 多次被怀疑 "不在干", 但实际:

• 派单后 14s stub 是 worker 问题, 不是 Queen 问题
• Queen 切直跑后是真的在跑 (curl + grep + write_file 都执行了), 但单 turn ≤3 工具让体感很慢
• 用户提"我去 worker body 干"时, Queen 没反驳 (应该反驳: 派单失败不是 Queen 问题, 直跑已切回, worker body 启动成本更高)

Lesson: 派单 stub 后, Queen 必须在主对话主动报告 + 解释下一步直跑计划, 而不是让用户猜。不主动报告 = 用户默认 "Queen 卡死"。

Evidence (this session, 2026-08-21, kanban-env-leak fix)

Subagent (model=anchor, leaf, 1 task) was dispatched to fix the HERMES_DELEGATED_CHILD_CONTEXT=1 env leak in hermes_cli/main.py. Goal explicitly said "Patch the relevant file(s)" with verification commands.

Lesson: even when a worker fully diagnoses the issue and writes the patch in its final summary, it may not have called patch. Always re-verify with git diff <path> in Queen before reporting the fix landed. If diff is empty, take the worker's proposed code and patch it yourself in the same turn — don't redispatch.

Queen patch fallback pattern (no re-dispatch needed):

# After worker reports "done" but git diff shows nothing
# 1. Read the worker's live transcript to find their proposed code
result = terminal(command="tail -40 /Users/henry/.hermes/cache/delegation/live/<deleg_id>/task-0.log")
# 2. Extract their proposed patch (usually in the final assistant turn)
# 3. Apply it with patch tool yourself, verify, commit
# 4. Report what actually shipped — not what the worker claimed

Evidence (this session, 2026-08-11, jixie/amac-wiki)

Kanban Worker Protocol-Violation Variant (2026-09-07)

Kanban workers (hermes kanban spawn via dispatcher) share the same fire-and-forget failure surface as delegate_task leaf workers, but with a different shape:

• Failure mode: worker runs 30-60 tool calls, completes the actual task (writes file, runs git commit), but exits with rc=0 without calling kanban_complete or kanban_block.
• Dispatcher detection: marks run outcome=crashed, protocol_violation=true. After 3 consecutive violations, dispatcher gave_up and forces the card to blocked with kind=needs_input and a synthetic block reason.
• Symptom on board: 4 retry attempts in events[], all with kind=protocol_violation, last event kind=gave_up — looks like a real blocker but is a worker discipline bug.

Distinguishing "real blocker" from "protocol-violation pseudo-blocker"

When inspecting a blocked kanban card, check events[] in kanban_show(task_id=...):

Heuristic (2026-09-07 audit, 4 cards inspected): 3 of 4 blocked high-priority cards were pseudo-blockers. Always grep events[] for protocol_violation: true before assuming a card needs Boss input.

Pre-dispatch mitigation: inject "收口铁律" into worker_context

Before dispatching (or re-dispatching) a card that previously hit gave_up, post a kanban_comment to the card with the discipline clause:

🔴 GM 强收口指令（避免重蹈 N 次 protocol_violation）

历史教训：你这张卡上一轮 N 次跑都因 worker rc=0 但没调 kanban_complete，被 dispatcher gave_up。

**这次必须遵守**：
1. ✅ 每一步配置改动后，立即写 git commit + 拿 commit hash
2. ✅ 完成后立即 kanban_complete(metadata={commit_hash, ...verify evidence})
3. ✅ 若遇不可恢复错误，立即 kanban_block(kind=transient)
4. ❌ 不允许"做完了但忘了收口"——这是 §6 验收铁律红线

This costs 1 kanban_comment per card but reduces gave_up recurrence rate from ~100% to <20% in observed runs (2026-09-07).

kanban_create hard requirements (2026-09-07 lesson, hardened 2026-09-08)

kanban_create with initial_status=ready (default) REQUIRES model_override — dispatch fails with model_override is required for status=ready tasks. Use one of: anchor/gpt-5.6-sol/gpt-5.5/gpt-5.4/opus-5/sonnet-5/fable-5.

Symptom of forgetting: 5+ consecutive kanban_create failures with the same error in one turn. Don't retry with different titles; add the model_override param.

Two additional kanban_create gotchas (2026-09-08 batch dispatch, 8 cards)

When dispatching a batch of >=3 engineering cards in one turn (e.g. Sprint 0 plan), two more silent failures bite BEFORE you hit model_override:

1. project param: kanban_create REQUIRES project field for status=ready tasks. Without it, fails with project_id is required for status=ready tasks. Use one of: intel/growth/invest/infra. Use project=infra for engineering work in company-hq-bound repos. The valid project list comes from hermes project list.

2. Body must avoid nested heredocs and tokens that look like JSON dicts: SQL-binding-style parser rejects body strings containing dict-shaped constructs. Use plain markdown bullets + fenced code blocks with simple key=value syntax — not inline { key: value } blocks. Symptom: Error binding parameter 3: type 'dict' is not supported.

Recipe: batch-dispatch a Sprint 0 plan (2026-09-08 pattern)

kanban_create(
    title="[T-01] ...",
    assignee="planning",
    body="# T-01 ...\n\n## Target repo\n...\n\n## Scope\n- 1.\n- 2.\n\n## Acceptance\n- ...",
    project="infra",
    model="anchor",
    workspace_kind="worktree",
    workspace_path="/Users/.../repo",
)

When a card body needs shell-style config examples, write them as fenced code blocks (not inline {...} dicts). When in doubt, keep body <=2KB and use plain bullet structure.

Failure-mode triage (2026-09-08 batch dispatch, 8 cards, 2 failed first try)

Loop warning: when you see same_tool_failure_warning; count=N in tool output, STOP retrying with the same call. Diagnose the error message, fix the parameter, retry ONCE. If still failing, fall through to kanban_comment + kanban_block(reason=...).

Read-Only Evidence-Collection Tasks (v29.8.x)

Not every leaf worker goal is "fix X" or "apply Y". Queen also dispatches read-only evidence-collection tasks (a.k.a. "verify tickets"): "run script X with arg Y, record exit code / output / log / DB state, report structured facts". These have a different failure surface than patch goals.

Differences from patch goals

• No patch / write_file / edit mandatory at end — the deliverable IS the evidence, written to a workspace path the parent specifies
• Hard timeout is the deliverable, not a failure: the parent task spec usually says "if it doesn't exit within 90s, kill and treat exit as fail" — this is correct expected behavior, not a worker defect
• No code review — verdict is purely empirical: did the smoke-test produce the expected token in stdout?

Hard rules for evidence-collection leaf workers

1. Tool list still capped at ≤6 calls but the cap shape differs:

   • ≥1 terminal to set up (mkdir + pre-state queries, parallelizable)
   • ≥1 terminal for the smoke-test invocation (foreground with hard timeout ≤ 2× the parent-specified cap)
   • ≥1 terminal for pkill -f <worker-process> if the smoke-test hung (clean kill, then verify no orphan procs)
   • ≥1 read_file / terminal head|tail for log excerpts
   • ≥1 write_file of a structured evidence JSON (rc, has_token, ledger_tail, db_query_results, verdict, explanation) — the evidence JSON IS the deliverable, not optional
   • Optional terminal for self-consistency cross-checks (write evidence → re-derive each field from live sources → compare)

2. Always write the evidence JSON even on FAIL. A "worker hung → killed → no output" run is itself useful evidence for the parent. Don't skip the write step because "nothing worked" — record what actually happened.

3. Distinguish "tool failure" from "system-under-test failure". If the smoke-test command itself hangs and the worker pkills it, that's evidence the system under test is broken, not that the worker failed. Verdict wording should make this clear: "L2 opencode routing unverified because worker hung" ≠ "worker broken".

4. The parent (Queen/GM) reads the evidence JSON, not your summary. Your final assistant summary is human-readable prose; the JSON is the board-readable artifact. Both are useful; the JSON is non-negotiable.

5. DO NOT call kanban_complete / kanban_block from a leaf worker session. Leaf workers do not have those tools in their toolset (subagent delegation strips them). The system reminder "you must call kanban_complete or you violate protocol" is a known false-positive in leaf-worker subagent context — your terminal state is the evidence file + summary returned to the parent; the parent GM closes the ticket. If you spend tokens trying to invoke a tool you don't have, you've wasted the worker's budget. Acknowledge the reminder in your final summary if needed, then proceed normally.

Anti-pattern: "tool budget hit before write_file"

Symptom (2026-08-29, t_f4b8b5bb verify ticket): worker used ~5 tool calls on setup + run + log inspection, then the system reminder about kanban_complete arrived, and the worker started narrating "I will write the file now" instead of writing it. Do not narrate — call write_file first, then summarize. The reminder can be addressed in the same response that contains the write.

Anti-pattern: "polling instead of backgrounding"

If the smoke-test invocation itself might exceed 60s (likely for L2 worker smoke-tests), do not poll a background session — use a foreground terminal with timeout=<2× the parent cap> (e.g. parent says 90s cap → use 180s timeout). Foreground with timeout returns exit_code=124 when the cap hits, which IS the empirical evidence. Backgrounding + polling costs more turns and obscures the timeout-as-evidence signal.

Cross-check pattern (recommended, ~1 extra turn)

After writing the evidence JSON, write a one-shot ad-hoc verifier script to /tmp/hermes-verify-<ticket>-<pid>.sh that:

• Re-derives every numeric/boolean field from live sources
• Compares to the evidence JSON field
• PASS/FAIL count + clean exit

Run it, capture the output, delete the script. This proves the evidence is internally consistent against ground truth, not just internally consistent. Catches subtle bugs like grep -c returning non-zero exit when count=0, where || echo 0 appends a second 0 line — verifier was then deleted to avoid leaving /tmp litter.

Evidence (this session, 2026-08-29, t_f4b8b5bb verify)

Lesson: when the worker is opencode running opencode/hy3-free (free model), it can hang indefinitely past 120s emitting only the TUI header. Do NOT increase the timeout hoping it will finish — kill it and report unverified. Repeated hangs (ledger shows t_f4b8b5bb_v19r2 ran 11384s = 3.16h before SIGKILL) confirm this is a model-state issue, not transient.

See also

• queen-dispatch SKILL.md §不要 — "派 audit/调研类任务时不要一次塞 >8 步操作" (related but looser; this skill is the leaf-specific tightening)
• queen-poll-kanban SKILL.md §delegate_task vs kanban 派单选择 — same lesson from the kanban side
• SOUL §硬规则 L5 "Verify-before-claim" — Queen 责任是复核 verify exit code, 不重跑; this skill extends that to the worker verification step itself
• hermes-agent-skill-authoring SKILL.md §"completed ≠ done" — subagent final summary is a self-report; always re-check disk state before claiming completion upstream
• queen-dispatch/SKILL.md §"Git worktree for live-checkout safety" — when patching hermes-agent in place, always git worktree add from the live repo; never git checkout (Hermes blocks that to prevent module version mixing)
• autonomous-ai-agents/opencode SKILL.md §Known hang patterns — opencode/hy3-free free model may hang indefinitely past 120s emitting only TUI header; treat as system-under-test failure

GM-style product planning + sprint dispatch (2026-09-08, OpenAnchor Phase 1)

When Boss asks GM (总经理, not Queen) to "制定可行的计划" for a product epic (e.g. "open source the LLM router"), GM's response shape must be:

Iron law (Boss correction, 2026-09-08)

"你是gm，总经理啊。给我制定可行的计划啊"

Boss uses GM mode (not Queen mode) when the task is product strategy + sprint decomposition. Queen mode is for short delegate_task bursts; GM mode is for multi-week roadmaps that need ADR-level planning + full sprint dispatch.

GM-mode response shape (4 sections, in this order)

1. 三步走 / N 阶段战略 — Boss's口头禅 = "一步一步来" + "不能停". Always chunk into 3-4 phases with explicit触发条件 between them. Phase 1 = MVP, Phase 2 = scale, Phase 3 = platform.
2. Sprint 0 卡清单 — ≥6 cards, each with: title / assignee / scope / acceptance铁证 / 估时. Use [T-NN] numbering convention. Bind to project=infra for engineering, project=intel for research.
3. 资源 + 预算 + 风险 — 工位需求表 / 现金支出（触发红线要 DM 通告）/ 风险与缓解. Boss hates wishful thinking; always include "Boss 拍的字" risks like "12 工位理论 vs 4-5 实测".
4. 关键里程碑 — 验收关 with铁证 (e.g. "GitHub stars ≥500", "arXiv paper submitted", "Provider 矩阵 e2e 通过").

Common Boss preferences encoded here (NOT just memory — Boss repeated them 2026-09-08)

• "一步一步来" = 永远 Phase 1 MVP first, Phase 2 等流量, Phase 3 等合规
• "我们要做的不能跟别人做的一样" = 必须找差异化护城河，不是抄竞品 (audit竞品 → 找他们弱点 → 我们避开)
• "审计一下" = Boss wants实证 evidence, not speculation. Always cite path:line + grep counts.
• "头脑风暴" 模式 ≠ execute 模式. Boss explicitly says "不要做任何的执行" → produce analysis only, do NOT kanban_create until Boss says "派".

Sprint 0 dispatch canonical pattern (8 cards, OpenAnchor Phase 1, dispatched 2026-09-08)

All 8 use: project="infra", workspace_kind="worktree", workspace_path="/Users/dongshenglu/company-hq/export/anchor-core".

OpenAnchor 5-version mapping pattern (v3.7 铁律: 基于证据的认知更新, 2026-09-08)

When Boss asks "搞清楚 X 跟 Y 的关系" (typical: "开源版跟本地版 / VPS 版 / 跑在生产版的真实关系"), GM's prior tendency is to default-assume from past narrative. Boss explicitly called out 3 wrong assumptions in 24h:

• "Anchor = 兑卡 SaaS" → wrong
• "Anchor = Sakana Fugu 复现" → wrong
• "开源版 = 我本地 export/anchor-core" → wrong

Iron rule (Boss 2026-09-08): 验证类任务不要用"是否派 X / A派B你做C暂缓"格式请示 — 直接做、出铁证、提下一步。

Iron-claudit recipe (no speculation, only on-disk + on-network evidence)

For open-source product version-relationship questions, run these 5-7 curl calls in parallel and produce a 5-version table:

# 1. GitHub 仓库元数据
curl -sS --max-time 10 "https://api.github.com/repos/<owner>/<repo>" | \
  grep -E '"(default_branch|stargazers_count|forks_count|open_issues_count|created_at|pushed_at|updated_at|size|language|license|description|archived|disabled)"'

# 2. Branches (看 worker 是否推了 feature 分支)
curl -sS --max-time 10 "https://api.github.com/repos/<owner>/<repo>/branches?per_page=100" | grep '"name"'

# 3. Tags / Releases
curl -sS --max-time 10 "https://api.github.com/repos/<owner>/<repo>/tags"
curl -sS --max-time 10 "https://api.github.com/repos/<owner>/<repo>/releases"

# 4. Recent commits on default branch
curl -sS --max-time 10 "https://api.github.com/repos/<owner>/<repo>/commits?per_page=10" | \
  grep -E '"(sha|message|date)"'

# 5. README sha 对账 (关键：本地 vs GitHub 哈希是否一致)
curl -sS --max-time 10 "https://api.github.com/repos/<owner>/<repo>/contents/README.md"

The 5-version mapping (anchor-core 真实对账, 2026-09-08 实证)

OpenAnchor 项目存在 5 个不同位置的代码副本，Boss 问"搞清楚"时必须先摸清：

关键发现 (2026-09-08 GitHub API 实证)

• E 的 branches 包含 infra/t_077a2648-t-04-anchor-init-cli —— 这是 Sprint 0 worker T-04 的 worktree push 上去的 feature 分支
• E 的 main 跟 A 的 main README.md sha 完全一致 (17998 bytes) = A 的 main 已经 push 到 E
• E 的 stars=0 / forks=0 / description=null / releases=0 = 开源版"几乎裸奔"状态
• E 的 pushed_at = 2026-09-08 08:32 = 今天上午 8:32 最后一次 push, 但不是 Sprint 0 push (那是 push 到 feature 分支)

三个常见误判 (必须基于 evidence 而非 narrative)

派单建议矩阵 (Boss 拍板前 GM 给的选项)

当 v3.7 铁律触发 ("搞清楚" 类指令), 给 Boss 的选项必须 已经带铁证 (curl 落地的真实数据), 不是 narrative 推测:

推荐策略: C + D 同时 (Boss 多数拍这条, 2026-09-08 实测)。

引用本节的其他场景

任何 "X 的本地版 / GitHub 版 / 跑在生产上的版 / 嵌入式版" 关系审计都套这模板:

1. 列出 5 个副本代号 (A/B/C/D/E)
2. 跑 5-7 个 curl 拿 GitHub 真相
3. 跑 git log / git for-each-ref 拿本地真相
4. 修正之前 3 个默认假设 (Boss 偏好 GM 主动自纠, 不是维护一致性)
5. 画关系图 + 给 Boss A/B/C/D/E 拍板选项

Head recalibration trap (技术债审计, 2026-09-08)

Boss asked: "Anchor本身要强大吧？不是光这个提供Provider和模型的事". Audit findings:

• TrinityHead v0.3 = 50 LOC NumPy, factory-calibrated, NO online learning
• Capability gaps: G1 同vendor降级, G2 query特征只用3-4维, G3 无online learning, G4 无user preference
• 推荐 Phase 1 目标 = L2 (Dynamic感知 + Capability-aware + 价格同步), NOT L1 or L3

This audit pattern (Boss-then-audit-then-spec) is now the canonical GM-mode opening for product engineering work. Save the audit findings in the relevant spec doc, not in this skill.