# Golem Parallel Workers Go

> Fan out work to multiple parallel agents and collect results in a Go Golem project. Use when the user asks about parallel execution, fan-out/fan-in, spawning worker agents for parallel work, or aggregating results from multiple agents in a Go Golem project.

- Skill: `golemcloud/golem-parallel-workers-go` (Agent Skill)
- Install (CLI): `npx get-skill add golemcloud/golem-parallel-workers-go`
- Raw SKILL.md: https://api.skillmd.com/api/skills/golemcloud/golem-parallel-workers-go/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: golemcloud (https://skillmd.com/u/golemcloud)
- Updated: 2026-11-12
- Page: https://skillmd.com/skills/golemcloud/golem-parallel-workers-go

---


# Parallel Workers — Fan-Out / Fan-In (Go)

## Overview

A single Golem agent admits invocations **sequentially**. Within one invocation, goroutines can overlap I/O by interleaving at yield points; this does not promise CPU-thread parallelism. Distribute work across **multiple agent instances** for independent invocation execution. The Go SDK gives you three building blocks:

1. **`CallAsync` + `Future.Get`** — start several cross-agent calls at once, then await each. `Call` / `MustCall` await one call at a time **per goroutine**; calls in separate goroutines can overlap at yield points. `CallAsync` returns immediately, so several calls are in flight together, and a goroutine blocked in `Future.Get` yields to the component-model event loop while the others proceed.
2. **`Trigger` + promises** — fire-and-forget each worker, hand each a promise ID, then await all the promises. Best for long-running work where you don't want a call in flight the whole time.
3. **`golem.ForkSelf`** — copy the running agent, state included, at the current point; the copy continues under a new phantom id.

> **Platform contract:** a single target instance handles one invocation at a time. Independent invocation execution comes from fanning out to **different** target instances (distinct full identities), not from calling the same instance repeatedly. This does not prohibit goroutine concurrency within one invocation.

## Approach 1: async fan-out with `CallAsync` / `Future.Get`

Start every worker call with `CallAsync`, collect the `*golem.Future[Out]` values in a slice, then `Get()` (or `MustGet()`) each. All calls are in flight while you await.

### Worker definition (`agents/worker/worker.go`)

```go
package worker

import "github.com/golemcloud/golem/sdks/go/golem"

// ID identifies a worker instance — distinct IDs run in parallel.
type ID struct{ Index int64 }

type ProcessIn struct{ Item string }

type RunReportIn struct {
    Region  string
    Promise golem.PromiseID
}

var Agent = golem.DefineAgent[ID](golem.Spec{
    Name:        "WorkerAgent",
    Description: "Processes one item",
})

var Process = Agent.Method[ProcessIn, string]("process", golem.Description("Process a single item and return the result"))
var RunReport = Agent.Method[RunReportIn, golem.Unit]("runReport")
```

### Coordinator fan-out (`agents/coordinator/impl/impl.go`)

```go
package impl

import (
    "myapp/agents/coordinator"
    "myapp/agents/worker"

    "github.com/golemcloud/golem/sdks/go/golem"
)

type state struct{}

var agent = coordinator.Agent.Implement(func(coordinator.ID) *state { return &state{} })

func init() {
    agent.Handle(coordinator.FanOut, func(_ *golem.Context[state], in coordinator.FanOutIn) []string {
        // 1. Start every call — each returns immediately with a future.
        futures := make([]*golem.Future[string], len(in.Items))
        for i, item := range in.Items {
            // A distinct worker instance per item → they run concurrently.
            c := worker.Agent.Get(worker.ID{Index: int64(i)})
            futures[i] = worker.Process.CallAsync(c, worker.ProcessIn{Item: item})
        }

        // 2. Await each. All calls were already in flight, so this fans in.
        results := make([]string, len(futures))
        for i, f := range futures {
            results[i] = f.MustGet() // MustGet() blocks (and yields) for this one result
        }
        return results
    })
}
```

`Future.Get` returns `(Out, error)`; the error is a `*golem.RemoteCallError` when the worker failed or could not be reached. `MustGet` panics with it instead, and the panic traps the component, as in Rust. To keep going when one worker fails, use `Get` and collect the errors. Model *expected* per-item outcomes as a `golem.Result` in the worker method's output, then unwrap or inspect each result:

```go
// worker.Process returns golem.Result[string, string]
res, err := futures[i].Get()     // golem.Result[string, string]
switch {
case err != nil:
    failures = append(failures, err.Error())
case res.IsErr():
    failures = append(failures, res.Err())
default:
    successes = append(successes, res.Ok())
}
```

## Approach 2: fire-and-forget with promise collection

For long-running work, `Trigger` each worker (fire-and-forget, no call held open) and pass each a promise ID; the worker completes its promise when done, and the coordinator awaits them all.

### Coordinator

```go
agent.Handle(coordinator.DispatchAndCollect, func(ctx *golem.Context[state], in coordinator.RegionsIn) []string {
    // One promise per worker.
    promises := make([]*golem.Promise[string], len(in.Regions))
    for i := range in.Regions {
        promises[i] = golem.NewPromise[string]()
    }

    // Fire-and-forget: hand each worker its region and its promise id.
    for i, region := range in.Regions {
        c := worker.Agent.Get(worker.ID{Index: int64(i)})
        worker.RunReport.MustTrigger(c, worker.RunReportIn{Region: region, Promise: promises[i].ID()})
    }

    // Await all promises (each durably suspends until its worker completes it).
    results := make([]string, len(promises))
    for i, p := range promises {
        results[i] = p.MustAwait()
    }
    return results
})
```

### Worker completes its promise

```go
agent.Handle(worker.RunReport, func(_ *golem.Context[state], in worker.RunReportIn) golem.Unit {
    report := "Report for " + in.Region + ": OK"
    golem.CompletePromise(in.Promise, report)
    return golem.Unit{}
})
```

`Trigger` returns a `golem.InvocationID` (and an error if the invocation could not be enqueued) and does not wait; a failure after the invocation is accepted is not reported at the trigger site — the worker signals completion (or failure) through its promise.

## Approach 3: `golem.ForkSelf`

`golem.ForkSelf` copies the running agent at the current point. Both copies continue from there: `forked` tells them apart, and `phantomID` is the copy's id on both sides. Synchronize them with a promise:

```go
agent.Handle(coordinator.Compute, func(ctx *golem.Context[state], _ golem.Unit) string {
	result := golem.NewPromise[string]()

	if forked, _ := golem.MustForkSelf(); forked {
		// The copy does its share of the work and hands the result over.
		golem.CompletePromise(result.ID(), "forked-result")
		return "forked done" // seen only by the copy's invocation
	}

	// The original waits for the copy.
	return "combined: original + " + result.MustAwait()
})
```

Fork in a loop for N-way fan-out: each iteration's copy takes slice `i` and returns, while the original awaits every promise. `ForkSelf` returns an error (a `*golem.AgentOperationError`) where the host refuses; `MustForkSelf` panics instead.

## When to use which

| Criteria | `CallAsync` / `Future` | `Trigger` + promises |
|----------|------------------------|----------------------|
| Short calls, want results inline | ✅ Best fit | Works but heavier |
| Long-running work, don't hold a call open | ⚠️ Call stays in flight | ✅ Best fit |
| Need the invocation identity for other work | `Future.ID` | `Trigger` returns `InvocationID` |
| Per-item error as a value | Return `golem.Result` in the method | Encode outcome in the promise payload |
| Workers need the current state | Pass it as arguments | Pass it as arguments — or use `golem.ForkSelf`, whose copy inherits it |

## Key Constraints

- **Sequential invocation admission is not a synchronization guarantee.** Goroutines in one invocation can interleave at RPC, HTTP, stream and promise waits; protect shared mutable state as needed. Use distinct worker instances for independent invocation execution; fanning out to the *same* durable instance serializes.
- Don't hold a `*golem.Future` across invocations — start it and collect or cancel it within the same handler. A second `Get` returns the same result. To abandon a result, call `Cancel` **before collecting it**: a later `Get` returns `golem.ErrCallCancelled`. Cancellation is best effort; the remote invocation may already have executed. `Cancel` after collection does nothing.
- `Future.Get` returns a remote failure as an error, and `Promise.Await` returns an error only for a payload that does not decode; their `Must` forms panic. Model expected outcomes as a `golem.Result` in the method output.
- Avoid a **synchronous** cycle: two agents each blocking in `Call` / `MustCall` on the other deadlocks. Break cycles with `Trigger` / promises.
- Worker instances persist after the coordinator finishes (they are durable by default); give them stable IDs you can address again, or make them ephemeral (see `golem-stateless-agent-go`).

### Related Skills

| Skill | When to Load |
|-------|--------------|
| `golem-call-another-agent-go` | The synchronous / async call mechanics (`Call`, `CallAsync`) |
| `golem-fire-and-forget-go` | `Trigger` details for fire-and-forget dispatch |
| `golem-wait-for-external-input-go` | Promises used to collect worker results |
| `golem-multi-instance-agent-go` | Address many worker instances by `ID` (and phantoms) |

