Workflow Rig

Workflow Rig

Use this skill as the canonical front door for agent-driven work in joelclaw.

The user should not have to choose between agent-workloads, restate-workflows, queue internals, or runtime trivia.

If the user says the magic words —

• "run this through the workflow rig"
• "dogfood this with the workflow rig"
• "kick this off"
• "start a canary"
• "run this as a workload"

— load this skill first.

What this skill owns

• workload shaping (serial, parallel, chained)
• choosing between inline work, durable runtime, or a pure handoff
• joelclaw workload plan / dispatch / run
• explicit stage DAGs via --stages-from
• honest runtime truth: what the restate-worker can do today, and what it still cannot do
• verification contracts before runtime fan-out
• canary/dogfood posture for real workload proofs

Core rule

Intent first, substrate second.

The caller describes the work. The workflow rig chooses the narrowest honest execution path.

Verification first, loops second. Before planning durable or parallel runtime work, ask: can an agent prove this worked without Joel staring at it? If not, shape a verification/tooling stage first. Do not turn unverified vibes into high-throughput slop.

Do not push Redis vs Restate vs sandbox trivia back onto the caller unless that tradeoff is the actual decision.

Current proven state (as of 2026-03-17, session StubbornFerret)

• joelclaw workload plan, joelclaw workload dispatch, and joelclaw workload run are real.
• Proven durable path: joelclaw workload plan → Redis queue → Restate dagOrchestrator → dagWorker → execution.
• Multi-stage DAGs with dependsOn are proven across 3-5 stage pipelines. Downstream stages can consume earlier outputs via {{nodeId}} interpolation.
• --stages-from stages.json is proven: duplicate ids, unknown deps, self-deps, and cycles are rejected before runtime admission; critical path and phase grouping are calculated.
• shell handler ✅ runs commands in the k8s restate-worker pod. Git clone, pi agent file writes, git commit, and git push are proven.
• infer handler ✅ full pi agent with tools — read, edit, write, bash. pi -p mode enables all tools. It can read files, edit them, run commands, and produce working code changes. Not just text generation — full agent-style codegen.
• microvm handler ⏸️ code works (one-shot exec model proven via bun test in pod) but ADR-0230 is paused — Colima nestedVirtualization crashes the VM under load. Don't use.
• The restate-worker image is a full agent environment: pi 0.58.4, 76 skills, GitHub push auth from k8s secret, pi auth mounted from host (stays fresh).
• Autonomous codegen proven: infer handler reads files + edits them via pi tools. Shell handler handles git clone/commit/push. Chain them in a DAG for full autonomous coding loops.
• Pre-cloned repo cache at /app/repo-cache (~200ms copy vs ~3s clone). dagWorker uses 15m inactivity timeout and 30m hard abort.
• Retry caps: dagWorker maxAttempts=5, dagOrchestrator maxAttempts=3. No more infinite retry poisoning.

What does not work yet

• microvm handler paused (ADR-0230) — needs dedicated Linux hardware with native KVM
• DAG completion notifications to the gateway not wired (operators poll or check OTEL)
• large-file pi agent edits can be slow (3-5 minutes) but succeed within the 15m timeout

Canonical operator flow

1. Shape the work with joelclaw workload plan, including the verification contract for every stage.
2. Present the shaped workload and verification path. Use existing authorization for a scoped run request; ask only for missing consequential choices or additional scope.
3. Within the authorized scope:
   • execute inline if it is bounded, local, and reversible
   • use joelclaw workload run for real durable execution
   • use joelclaw workload dispatch only for a real baton pass
4. If you enqueue runtime work, poll for progress with joelclaw runs, joelclaw run <run-id>, or OTEL. There is no automatic completion ping yet.
5. Report outcome tersely: changed, verified, remaining, next move

If the proof path is missing, stop before runtime admission and author the reusable verifier skill/tool/check first. That is the leverage move.

Magic words → canonical commands

Plan only

joelclaw workload plan "<intent>" --repo <repo> [--paths a,b,c] [--stages-from <path>] [--write-plan <path>]

Real runtime canary / dogfood

joelclaw workload run <plan-artifact> \
  [--stage <stage-id>] \
  [--tool pi|codex|claude] \
  [--timeout <seconds>] \
  [--model <model>] \
  [--execution-mode auto|host|sandbox] \
  [--sandbox-backend local|k8s] \
  [--sandbox-mode minimal|full] \
  [--repo-url <git-url>] \
  [--dry-run] \
  [--skip-dep-check]

Handoff, not execution

joelclaw workload dispatch <plan-artifact> \
  [--stage <stage-id>] \
  [--project <mail-project>] \
  [--from <agent>] \
  [--to <agent>] \
  [--send-mail] \
  [--write-dispatch <path>]

Sandbox mode guidance

Use --sandbox-mode full when the proof needs real runtime surfaces:

• service/network lifecycle
• full environment materialization
• cleanup evidence
• anything where a minimal local sandbox would hide the real failure mode

Use --sandbox-mode minimal for cheap code/doc/test slices where full runtime provisioning is overkill.

Use --stages-from when you already have a real stage DAG. The planner preserves per-stage acceptance, validates dependencies/cycles, calculates critical path metadata, and keeps the DAG instead of collapsing it into template stages.

Use --skip-dep-check only for deliberate manual recovery. Normal joelclaw workload run blocks a stage until its explicit dependencies have terminal truth.

Do not use microvm — ADR-0230 is paused. Colima nestedVirtualization is unstable. Use shell + infer for all work.

Current runtime truth

• joelclaw workload run is the real bridge from workload artifacts to runtime admission.
• The durable path is Redis queue admission → Restate dagOrchestrator → dagWorker.
• dagOrchestrator resolves dependency waves correctly for chained multi-stage DAGs.
• {{nodeId}} interpolation is proven for passing upstream outputs into downstream stages.
• The shell handler is the proven path for git operations (clone, commit, push) and arbitrary CLI work.
• The infer handler is a full pi agent — it reads, edits, and writes files via tools. Use it for codegen, analysis, and any task that benefits from LLM + file access.
• Chain infer (edit files) → shell (git commit + push) for autonomous coding loops.
• The microvm handler is paused (ADR-0230). Do not use.
• Completion is poll-based for now. No gateway finish event is emitted when a DAG lands.

Real chained example

This is an honest four-stage shape the rig can run today:

[
  {
    "id": "research",
    "name": "Research current state",
    "acceptance": ["Facts gathered"],
    "executionMode": "manual"
  },
  {
    "id": "plan",
    "name": "Turn research into an execution plan",
    "dependsOn": ["research"],
    "acceptance": ["Implementation plan written"],
    "executionMode": "manual"
  },
  {
    "id": "implement",
    "name": "Apply the change in the worker",
    "dependsOn": ["plan"],
    "acceptance": ["Requested files updated", "Commit pushed"],
    "executionMode": "pi",
    "notes": "Use {{plan}} as the downstream input."
  },
  {
    "id": "verify",
    "name": "Verify and summarize",
    "dependsOn": ["implement"],
    "acceptance": ["Exact proof command/artifact captured", "Quality bar checked", "Closeout ready"],
    "executionMode": "manual",
    "notes": "Use {{implement}} for verification context. If this cannot be proven by an agent, split out a verifier skill/tool stage before implementation."
  }
]

Run it through the front door:

joelclaw workload plan "Research, plan, implement, then verify the change" \
  --repo ~/Code/joelhooks/joelclaw \
  --stages-from stages.json \
  --write-plan plan.json

When to reach for compatibility skills

• agent-workloads — only when an older prompt already names it; treat it as a compatibility alias
• restate-workflows — only when the work is specifically about external-repo runtime bridging or low-level substrate contracts

Dogfood posture

When proving runtime work:

• prefer a canary first
• use the real front door (joelclaw workload run), not hand-rolled system/agent.requested, unless you are debugging below the rig
• capture honest evidence from queue admission, Restate, dagWorker, and resulting git/verification artifacts
• poll the run yourself; there is no completion event to the gateway yet
• inside a sandboxed stage run, do not launch another workflow-rig canary
• if the rig is broken, say the rig is broken; do not blame sandboxes or the gateway for a queue/worker failure
• if the task needs large-file agent edits, budget minutes, not seconds

Rules

• do not invent new workload vocabulary when docs/workloads.md already defines it
• do not force the operator to choose queue vs Restate vs sandbox when joelclaw workload run is the right bridge
• do not use microvm — ADR-0230 paused
• do not claim a dogfood proof succeeded unless the real runtime path moved and produced evidence
• do not imply automatic completion notifications exist when they do not