Pipeline State

pipeline-state — the run-wide handoff bus

Durable, file-backed state for the ENTIRE pipeline. One JSON file persists the run's variables + per-phase outputs + .ai_log/ evidence-path references across phases, so a downstream agent (or the orchestrator) recovers exactly what it needs by LOOKUP instead of receiving a large inlined dump — and nothing is lost to context compression on long runs.

The standing pattern: evidence -> .ai_log/; references + key fields -> pipeline-state; handoffs = state lookups, not inlined dumps.

• the orchestrator INITIALIZES the state at session start (alongside creating the session folder), passes its path (state_file) to every dispatched agent, and after each phase READS that phase's output fields + .ai_log/ evidence paths FROM the state to validate the gate and to build the NEXT phase's prompt — passing state keys + paths, never re-dumping large content.
• every agent, on START, READS the state to pull the inputs this phase needs (key fields + .ai_log/ evidence paths) — it does NOT expect large inlined content; on COMPLETION, WRITES its outputs — key result fields + PATHS to evidence offloaded in .ai_log/ (never the blobs).
• It COEXISTS with the docs/-native reports (the human-readable record) and the .ai_log/ path-handoff. The state REFERENCES the docs/ + .ai_log/ artifacts; it does not replace them.

State file (PER-SESSION, .ai_log/, git-ignored)

.ai_log/session-<session-id>-<name>.json

ONE file per run, living under .ai_log/ — e.g. .ai_log/session-2026-06-30-member-blog-gating.json.

Naming convention. <session-id> is the run's disambiguating id — commonly the session date <YYYY-MM-DD> (it separates repeat/concurrent runs); <name> is the epic slug. Together <session-id>-<name> is the session LABEL <YYYY-MM-DD>-<epic-slug> (the same slug as session_dir). The script DERIVES the path from that label (session-<label>.json).

The orchestrator derives + creates it at init and passes its exact PATH as state_file to every dispatched agent. Agents read/write the path they were given — they do NOT reconstruct or hardcode a constant path, because it varies per session. Use the one convention consistently across the skill, the orchestrating command, and any workflow definition.

Git behavior. Because it lives under .ai_log/, it is git-IGNORED (never committed) by the /.ai_log/* + !/.ai_log/.gitkeep rule — no .gitignore change needed. It is the ONE structured handoff JSON in .ai_log/; ordinary evidence there stays throwaway. It is ephemeral, per-run, and regenerated fresh each session — NOT a durable record. Durable records still live in committed docs/, which the state only references: the state is a machine handoff bus, never durable content parked in a git-ignored folder.

Script

scripts/pipeline-state.sh          # ships with this skill; call it from wherever the skill is installed

Commands

# Start a new run — writes .ai_log/session-<session_id>.json AND cleans stale .ai_log/ artifacts
pipeline-state.sh init   <session_id>

# Store a string/scalar field
pipeline-state.sh set    <session_id> <field> <value>

# Append a value to a list field (creates the list if absent) — e.g. an .ai_log/ evidence path
pipeline-state.sh append <session_id> <field> <value>

# Read a single field (or the full JSON if field is omitted) — the recovery / handoff lookup
pipeline-state.sh get    <session_id> [field]

# Delete the state file
pipeline-state.sh clear  <session_id>

<session_id> is the run label on init (convention: <YYYY-MM-DD>-<epic-slug>, the same slug as session_dir); the state file PATH is DERIVED from it (.ai_log/session-<session_id>.json), so every later command passes the SAME <session_id> and resolves the SAME per-session file. The orchestrator captures the path init prints and passes it as state_file; agents use that exact path.

State schema + read/write protocol

See references/state-schema.md — the per-phase entry shape (agent, status, key output fields, .ai_log/ evidence paths, work-item links) + the read-on-START / write-on-COMPLETION protocol and the field conventions (scalar via set, list via append).

init cleanup

On init the script clears stale .ai_log/ artifacts from prior runs — everything under .ai_log/ except the tracked .ai_log/.gitkeep — so a fresh run starts clean. This matches the temporary-only rule for that folder and the folder-tracked / contents-ignored convention (/.ai_log/* + !/.ai_log/.gitkeep).

Usage

# --- the orchestrator, at SESSION START (alongside creating docs/sessions/<session>/) ---
SESSION="2026-06-30-member-blog-gating"          # = the session_dir slug
bash scripts/pipeline-state.sh init "$SESSION"
bash scripts/pipeline-state.sh set "$SESSION" session_dir "docs/sessions/$SESSION/"
bash scripts/pipeline-state.sh set "$SESSION" user_request "$USER_REQUEST"
# init derived .ai_log/session-$SESSION.json — the orchestrator passes state_file=.ai_log/session-$SESSION.json to EVERY dispatched agent

# --- any phase, on COMPLETION: write outputs = key fields + .ai_log/ evidence PATHS (never blobs) ---
bash scripts/pipeline-state.sh set    "$SESSION" prd_document "docs/prd/member-blog-gating.md"
bash scripts/pipeline-state.sh append "$SESSION" phase-0-evidence ".ai_log/phase-0-pm-prd.md"

# --- next phase, on START: read the inputs it needs (fields + .ai_log/ paths), not a big dump ---
PRD=$(bash scripts/pipeline-state.sh get "$SESSION" prd_document)

# --- the orchestrator, at the GATE: read the phase's outputs + evidence paths to validate + build next prompt ---
bash scripts/pipeline-state.sh get "$SESSION"        # full state (recovery)

Out-of-pipeline flows use it too. Any standalone multi-step run (a test-healing pass, a migration sweep) uses the SAME script and file — init at the start, set/append for its per-step variables (the item under work, file paths, evidence paths, a PR number), and get to restore state after context compression. The schema in references/state-schema.md covers both shapes.

VERSION 2.0.0

Made by HasBrains — https://hasbrains.com/