Golem Custom Snapshot Scala

Custom Snapshots in Scala

Golem agents can implement snapshotting to support manual (snapshot-based) updates and snapshot-based recovery. The Scala SDK provides two approaches: automatic JSON-based snapshotting via Snapshotted[S] and custom binary hooks.

When to Use Snapshotting

Snapshotting solves two distinct problems:

1. Manual / snapshot-based component updates — required when updating agents between incompatible component versions.
2. Fast recovery and oplog compaction — for long-running agents whose oplog grows over time (heartbeats, polling loops, recurring tasks, agents with frequent state changes). Without snapshotting, every recovery replays the full oplog from the beginning, which becomes increasingly expensive. With periodic snapshotting (every(N) or periodic(...)), recovery starts from the latest snapshot and replays only the entries after it.

You cannot opt out of oplog writes for a durable agent. If you are worried about oplog volume or replay cost, do not try to skip persistence — enable snapshot-based recovery here instead.

Enabling Snapshotting

Snapshotting must be enabled in the @agentDefinition annotation. Without it, no snapshot exports are generated:

@agentDefinition(snapshotting = "every(1)")
trait MyAgent extends BaseAgent {
  class Id(val value: String)
  def doSomething(): Future[String]
}

Snapshotting Modes

The snapshotting parameter accepts these values:

@agentDefinition(snapshotting = "periodic(30s)")
trait PeriodicAgent extends BaseAgent { ... }

@agentDefinition(snapshotting = "every(10)")
trait BatchAgent extends BaseAgent { ... }

Automatic JSON Snapshotting with Snapshotted[S]

The recommended approach. Bundle all mutable state into a case class with a Schema instance, then mix Snapshotted[S] into the implementation class:

1. Define the state type:

final case class CounterState(value: Int)
object CounterState {
  implicit val schema: Schema[CounterState] = Schema.derived
}

2. Enable snapshotting on the agent definition:

@agentDefinition(snapshotting = "every(1)")
@description("A counter with automatic JSON-based state persistence.")
trait AutoSnapshotCounter extends BaseAgent {
  class Id(val value: String)
  def increment(): Future[Int]
}

3. Mix in Snapshotted[S] on the implementation:

@agentImplementation()
final class AutoSnapshotCounterImpl(private val name: String)
    extends AutoSnapshotCounter
    with Snapshotted[CounterState] {

  var state: CounterState = CounterState(0)

  override def increment(): Future[Int] =
    Future.successful {
      state = state.copy(value = state.value + 1)
      state.value
    }
}

object AutoSnapshotCounterImpl {
  def loadSnapshot(
    state: CounterState,
    context: SnapshotRestoreContext
  ): Future[AutoSnapshotCounterImpl] =
    Future.successful {
      val instance = new AutoSnapshotCounterImpl(context.identity[String](0))
      instance.state = state
      instance
    }
}

The macro detects Snapshotted[S], summons Schema[S] at compile time, and generates snapshot handlers that serialize/deserialize state as JSON using zio-schema. Deserialization is automatic, while the companion loadSnapshot factory controls how that state and the restore context construct the complete implementation instance.

Requirements for Snapshotted[S]

• The implementation must have a var state: S field.
• S must be a case class with a Schema instance.
• The companion object must define loadSnapshot(state: S, context: SnapshotRestoreContext): Future[Impl].

Custom Snapshot Hooks

For custom binary serialization, define saveSnapshot() on the implementation class and the restoration factory loadSnapshot(...) on its companion object:

@agentDefinition(snapshotting = "every(1)")
trait SnapshotCounter extends BaseAgent {
  class Id(val value: String)
  def increment(): Future[Int]
}

@agentImplementation()
final class SnapshotCounterImpl(private val name: String) extends SnapshotCounter {
  private var value: Int = 0

  def saveSnapshot(): Future[Array[Byte]] =
    Future.successful(encodeU32(value))

  override def increment(): Future[Int] =
    Future.successful {
      value += 1
      value
    }

  private def encodeU32(i: Int): Array[Byte] =
    Array(
      ((i >>> 24) & 0xff).toByte,
      ((i >>> 16) & 0xff).toByte,
      ((i >>> 8) & 0xff).toByte,
      (i & 0xff).toByte
    )

}

object SnapshotCounterImpl {
  def loadSnapshot(
    bytes: Array[Byte],
    context: SnapshotRestoreContext
  ): Future[SnapshotCounterImpl] = Future.successful {
    val value =
      ((bytes(0) & 0xff) << 24) |
        ((bytes(1) & 0xff) << 16) |
        ((bytes(2) & 0xff) << 8) |
        (bytes(3) & 0xff)

    val instance = new SnapshotCounterImpl(context.identity[String](0))
    instance.value = value
    instance
  }
}

Method Signatures

// Save: serialize the agent's current state to bytes
def saveSnapshot(): Future[Array[Byte]]

// Companion factory: construct a complete agent from bytes and restore context
def loadSnapshot(
  bytes: Array[Byte],
  context: SnapshotRestoreContext
): Future[Implementation]

The macro detects these convention methods and wires them into the snapshot exports automatically. The restore context provides identity fields, the full agent ID, restored principal, phantom ID, and fresh configuration.

Restoration Is Read-Only

loadSnapshot is a specially supported SDK lifecycle operation, not an agent method. Normal construction is skipped, and the factory must return the complete fresh instance. Golem installs it only after the returned Future succeeds.

Snapshot loading runs in read-only mode and writes nothing to the oplog. Decoding, local computation, config reads, and other permitted reads can be used to construct the instance. Mutating host operations and outgoing HTTP or agent RPC calls are rejected before they take effect.

If the factory fails, partial state is discarded. A manual update remains on the previous component version. During automatic recovery, Golem recreates the component and replays without the failed automatic snapshot; it does not try an older automatic snapshot.

Best Practices

1. Prefer Snapshotted[S] — automatic JSON serialization via zio-schema is simpler and less error-prone.
2. Keep state in one case class — bundle all mutable state into a single var state: S for clean persistence.
3. Keep snapshots small — large snapshots impact recovery and update time.
4. Use custom hooks for binary formats — when you need compact encoding or compatibility with non-Scala components.
5. Test round-trips — verify that save → load produces equivalent state without ordinary initialization.
6. Handle migration — when the state schema changes between versions, loadSnapshot should handle snapshots from older versions.