# Flowing

> Runs a multi-step procedure as a Python DAG, so ordering, branching, retries and rollback are enforced by the runner rather than described in prose a model can generate past, and the whole procedure costs one tool call instead of a model turn per step. Use for "edit these files, run the tests, and restore everything if they fail", "retry the flaky step until the check passes", "fetch, validate, then skip the upload when nothing changed", "make sure these steps cannot be skipped", "resume from where it broke instead of redoing the expensive stages", "run these independent calls at once and merge the results", or any procedure of 3+ steps with branches, input contracts, or side effects. Primitives are depends_on, when=, validate=, retry_until=, undo=, detached= and journal_path=. Not for a single call, for steps needing reasoning between them that no predicate captures, or for async and distributed work. To audit whether one check can go red, use gating. To fan work out across subagents, use a dynamic workflow.

- Skill: `oaustegard/flowing-2` (Agent Skill, multi-file: 6 files)
- Install (CLI): `npx get-skill add oaustegard/flowing-2`
- Raw SKILL.md: https://api.skillmd.com/api/skills/oaustegard/flowing-2/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: oaustegard (https://skillmd.com/u/oaustegard)
- Updated: 2026-11-16
- Page: https://skillmd.com/skills/oaustegard/flowing-2

---


## NOT SUPERSEDED BY DYNAMIC WORKFLOWS — read first

Claude Code's dynamic workflows orchestrate **subagents** (separate contexts,
fan-out to 16-concurrent / 1000-agent). This skill is a **different primitive**:
single-context control flow over YOUR OWN tool calls, with durable side-effects
and checkpoint resume. The workflows runtime explicitly cannot touch the
filesystem or shell directly — its agents do the work and the script only
coordinates them. Flowing is the inverse: the script does the work.

Use flowing for an in-context pipeline (3+ steps, branches, retries, validation,
detached side-effects). Use a workflow when you need many subagents. They compose;
they do not compete. Do not abandon flowing for a workflow — you would lose the
durable side-effects and the cross-session checkpoint that hub-spoke depends on.

# Flowing — Control Flow in Code, Not Prose

When a procedure needs 3+ steps with branches, retries, or contracts, encode it as a DAG of Python tasks instead of prose imperatives. Prose like "first X, then Y, then if Z retry 3×" is read and generated past. A `@task` graph is structural: a step physically cannot run until its inputs are bound, and gates that fire on bad inputs can't be skipped.

The runner owns control flow — branching, retrying, validating, propagating failures, parallelizing. You provide judgment at the leaves. Runner: `scripts/flowing.py`.

## Quick Start

```python
from flowing import task, Flow

@task
def fetch_data():
    return {"items": [1, 2, 3]}

@task(depends_on=[fetch_data])
def process(fetch_data):          # param name must match the dep's name
    return sum(fetch_data["items"])

@task(depends_on=[process])
def store(process):
    print(f"Result: {process}")

Flow(store).run()                 # topo-sorts, runs each layer, parallel within a layer
```

Each task receives its dependencies as kwargs named after them. Independent tasks in the same layer run in parallel.

## Control-Flow Primitives

Encode branches and contracts as graph structure, not `if` statements inside task bodies.

### `when=` — conditional gate

Run the task only if the predicate (over gathered dep values) is truthy. Falsy → SKIPPED, and the skip propagates to dependents.

```python
@task(depends_on=[fetch], when=lambda fetch: fetch["needs_processing"])
def process(fetch):
    return transform(fetch["payload"])
```

### `validate=` — edge contract

Check gathered dep values before the body runs. Raise → FAILED with **no retry** (bad inputs don't fix themselves). Pass → proceed.

```python
def must_have_items(fetch):
    if not fetch.get("items"):
        raise ValueError("fetch returned empty payload")

@task(depends_on=[fetch], validate=must_have_items)
def process(fetch):
    return sum(fetch["items"])
```

### `retry_until=` — predicate-driven loop

Run the body, then call `retry_until(value)`. True → done. False → retry, consuming the `retry=` budget. Use for self-correcting LLM steps: generate, check, regenerate.

```python
@task(retry=4, retry_until=lambda r: r["valid"])
def generate_until_valid():
    candidate = llm_call(...)
    return {"valid": passes_schema(candidate), "candidate": candidate}
```

Distinct from `retry=` alone, which only retries on a raised exception.

### `undo=` — rollback on a later failure

When a main task fails, every task that succeeded in this run and declared `undo=` has `undo(value)` called with its own return value, in reverse completion order. Undone tasks and everything downstream of them end as `ROLLED_BACK`, so `resume()` re-runs them and the journal forgets them. Detached dependents of a rolled-back task skip, so no notification fires for work that was undone.

The common case is edit, test, and restore on red. `snapshot(paths)` records byte-exact contents (a missing file counts, and is deleted again on restore); `restore` is its undo:

```python
from flowing import task, Flow, snapshot, restore

@task(undo=restore)
def before():
    return snapshot(["src/tax.py", "src/invoice.py"])

@task(depends_on=[before])
def edit(before):
    rename_everywhere("calculate_tax", "compute_tax")

@task(depends_on=[edit])
def verify(edit):
    r = subprocess.run(["pytest", "-q"], capture_output=True, text=True)
    if r.returncode:
        raise RuntimeError(r.stdout[-3000:])

flow = Flow(verify)
flow.run()   # red tests: both files are back as they were; flow.results["verify"].error holds the output
```

Put `undo=` on the step *before* the risky one. `undo` only runs for tasks that succeeded, so an `edit` that fails halfway through its files would never undo itself; the snapshot step that preceded it does.

Limits: an `undo` that raises is logged, collected in `flow.rollback_failures`, and leaves its task `SUCCEEDED` while the rest of the walk continues. Results replayed from the journal are never undone, because their effects belong to an earlier run. Side effects of the failing step itself (a migrated database, a sent message) are only undone if some step's `undo` covers them. `Flow(..., rollback=False)` keeps the failed state on disk for inspection. `undo=` is refused on a `detached=True` task.

## Other capabilities

- **Parallel execution** — independent tasks in a layer run on a thread pool (`max_workers=`).
- **`detached=True`** — side-effect tasks (memory writes, notifications) that run after the main DAG and never block it on failure.
- **In-process resume** — `flow.run()` → fix → `flow.resume()` re-runs from the failure point, keeping succeeded tasks cached **in memory** (same process only). `flow.override(task, value)` injects a corrected result.
- **Durable journal (`journal_path=`)** — opt-in content-addressed replay that survives container death. `Flow(term, journal_path="/path/run.jsonl").run()` appends each succeeded task's result to an append-only JSONL keyed by a `step_key` = SHA-256 over the task's bytecode + its `when`/`validate`/`retry_until` bodies + its dependencies' keys (chained, so an upstream change propagates downstream). A later `run()` — even in a fresh container — replays the unchanged prefix from the journal and only executes tasks whose key is absent; editing a task body busts its key and re-runs it and its dependents, while cosmetic knobs (`retry=`, `timeout_s=`, `name`) do not. This is the cross-session checkpoint hub-spoke work relies on. Caveat: results are pickled, so non-picklable return values simply re-run; closure-captured values are not part of the key (only the task body's own code is).
- **`timeout_s=`**, **`retry=`** with exponential backoff, **`fail_fast=`**. Under `fail_fast`, parallel siblings that were already running still have their results recorded, so rollback can undo them.

Read [references/reference.md](references/reference.md) before using anything beyond the quick start and the primitives above — it covers every `@task` parameter, the `Flow` methods, resume/override, detached auto-discovery, and the `validate=`/`when=` signature-matching gotcha.

## When to use

- A procedure has branches that matter → `when=` makes them structural.
- Steps have input contracts → `validate=` makes them enforceable.
- An LLM step needs to converge → `retry_until=` puts the check in the loop.
- A change spans several files and must pass a check, or not land at all → `snapshot`/`restore` as `undo=`.
- 3+ independent operations that can parallelize.
- Multi-step pipelines where late failures shouldn't waste early work.
- Side-effects that shouldn't block the critical path → `detached=True`.

## When NOT to use

- A single sequential operation — just call the function.
- The next step needs *reasoning* about the prior result that can't be a predicate — use a think loop.
- Async or distributed workflows — this is single-container, thread-pool based.

## Authoring discipline

If you find yourself writing prose like *"first call X, validate Y, then if Z retry up to 3 times"*, that is a flowing graph. Refactor before shipping. Prose imperatives don't enforce; `@task` graphs do.

The same holds for your own tool calls. Search, edit and test as three model turns, each waiting on the last, is one Bash call running one flow. Each turn saved is a full wait for the model plus the tokens to restate the plan. When the next step does not need judgment about the last result, put the steps in a graph instead of in turns.

