Agent Substrate Overview

agent-substrate-overview

Agent Substrate is an Apache-2.0 Kubernetes runtime for agent-like workloads that multiplexes many stateful actors onto fewer worker pods, preserving in-memory + disk state via gVisor checkpoint/restore. It is explicitly alpha (v0.0.0, README quote: "VERY early development… APIs are almost guaranteed to change"). This skill is a short reference so you can evaluate it against k8s-agent-sandbox, not an operational guide.

If the operational decision is already "use Substrate," read the upstream README directly — this skill is intentionally not a step-by-step install guide because Substrate is at v0.0.0 with API churn between commits, so pinning a recipe here would go stale.

What Substrate is

From the upstream README: "Agent substrate is a system built on top of Kubernetes which manages agent-like workloads to achieve higher scale and efficiency than Kubernetes alone can offer, with lower latency."

Architectural diff vs kubernetes-sigs/agent-sandbox:

Per Google's 2026-05-20 blog: "Substrate is the open-source ultra-scale tier sharing Sandbox's core secure runtime and snapshotting capabilities with a minimal control plane designed to bypass some of the limitations of Kubernetes for millions of sub-second tool calls."

Components

• ateapi — gRPC control plane.
• atelet — node-level DaemonSet supervising workers and snapshots.
• atecontroller — k8s controller reconciling Substrate CRDs.
• atenet — networking controller: DNS, Envoy routing, proxy sidecars.
• ateom-gvisor — interior helper running runsc checkpoint/restore.
• podcertcontroller — Pod Certificate signer polyfill.
• kubectl-ate — operator CLI plugin.

External deps: gVisor, Valkey (Redis-compatible), RustFS.

CRDs

• WorkerPool — N pods that will host actors. Configures pod sizing and oversubscription.
• ActorTemplate — describes a workload container image + boot/snapshot config; many actors instantiate from one template.

SessionIdentity (not a CRD; a service exposed by ateapi) exchanges ephemeral kubelet credentials for stable JWT/cert identities that persist across worker migrations.

Claude Code on Substrate

Substrate's demos/claude-code-multiplex/ ships a working example:

• A WorkerPool of 2 pods is oversubscribed by 3 ActorTemplates (luna, mars, orion).
• Each agent is a container built from workload/ (Dockerfile + Python wrapper around Claude Code) and referenced by sha256 digest in claude-code-multiplex.yaml.tmpl.
• A Go dashboard in ui/ calls the ateapi gRPC + Kubernetes API.

Example: WorkerPool + ActorTemplate field semantics

Trimmed from demos/claude-code-multiplex/claude-code-multiplex.yaml.tmpl upstream (Apache-2.0) — the WorkerPool plus one of its three ActorTemplates (mars and orion are structurally identical to luna, differing only in metadata.name and the ACTOR_NAME/TASK env values):

# 2 worker replicas for 3 actors — the multiplex pressure that makes the
# substrate suspend/resume behavior visible.
apiVersion: ate.dev/v1alpha1
kind: WorkerPool
metadata:
  name: claude-workerpool
  namespace: claude-multiplex-demo
  labels:
    workload: claude-multiplex
spec:
  replicas: 2
  ateomImage: ko://github.com/agent-substrate/substrate/cmd/ateom-gvisor

---

apiVersion: ate.dev/v1alpha1
kind: ActorTemplate
metadata:
  name: agent-luna
  namespace: claude-multiplex-demo
spec:
  pauseImage: "registry.k8s.io/pause:3.10.2@sha256:f548e0e8e3dc1896ca956272154dde3314e8cc4fde0a57577ee9fa1c63f5baf4"
  containers:
  - name: claude
    image: ${WORKLOAD_IMAGE}
    env:
    - name: ACTOR_NAME
      value: "luna"
    - name: TASK
      value: "Tell me one short, surprising fact about the Moon. One sentence."
    - name: ANTHROPIC_API_KEY
      valueFrom:
        secretKeyRef:
          name: anthropic-api-key
          key: api-key
  workerSelector:
    matchLabels:
      workload: claude-multiplex
  snapshotsConfig:
    location: gs://${BUCKET_NAME}/claude-multiplex-demo/

Field semantics not obvious from the names alone:

• WorkerPool.spec.replicas — pod count, independent of how many ActorTemplates target it. Here, 2 pods host 3 actors, so Substrate must suspend at least one actor at any moment.
• WorkerPool.spec.ateomImage — the ateom-gvisor interior helper image each pod runs to perform checkpoint/restore; not the workload image itself.
• ActorTemplate.spec.workerSelector.matchLabels — restricts which WorkerPools this template's actors may schedule onto, by matching pool metadata.labels. Typed *metav1.LabelSelector (the Deployment/NetworkPolicy idiom, not a Service's bare spec.selector map) and optional — upstream docs it as a gate: "if nil, all pools are eligible," and it can only narrow the eligible set, never expand it.
• ActorTemplate.spec.snapshotsConfig.location — where suspended-actor state is persisted (an object-storage URI here) so a later resume can restore it, potentially onto a different pod.
• ActorTemplate.spec.pauseImage — the sandbox/pause container each actor's pod slot boots from; unrelated to the workload container image.

Substrate is not an MCP server — it's a workload host. Claude Code packaged as an OCI container becomes an actor like any other Substrate workload.

Install summary (do not paste verbatim — read upstream first)

The upstream uses shell scripts under hack/, not Helm or Kustomize. Examples:

• Kind: hack/create-kind-cluster.sh, hack/install-ate-kind.sh --deploy-ate-system
• GKE: go run ./tools/setup-gcp --all, hack/install-ate.sh --deploy-ate-system

The exact scripts change frequently in alpha. Read the README at https://github.com/agent-substrate/substrate before running any of them.

Which to pick

• Use kubernetes-sigs/agent-sandbox for: per-session Claude Code pods on a laptop or GKE; production workloads now; integration with the broader k8s ecosystem (NetworkPolicy, StorageClass, etc.).
• Use agent-substrate for: ultra-high-density actor workloads where standard k8s scheduling is the bottleneck; experimentation with checkpoint/restore migration; research environments where alpha APIs are acceptable.

For this plugin's Claude Code-on-sandbox use case (one operator, one session at a time, microVM isolation): stick with k8s-agent-sandbox. Substrate's multiplexing is overkill and its alpha status means upgrade churn.

See also

• k8s-agent-sandbox — the primary path this plugin recommends.
• Upstream Substrate: https://github.com/agent-substrate/substrate
• 2026-05-20 GCP blog: https://cloud.google.com/blog/products/containers-kubernetes/bringing-you-agent-sandbox-on-gke-and-agent-substrate

Anti-fabrication

• Substrate is at v0.0.0; do not claim it is production-ready.

• The kubectl-ate plugin and ateapi are different things from agent-sandbox's kubectl + Python SDK — do not paste agent-sandbox CRDs into a Substrate cluster or vice versa.
• The "shares snapshot primitives" relationship is documented in Google's blog post; the actual code-level overlap is not enumerated upstream. Don't claim deeper code-level equivalence than that.
• Substrate's install scripts change frequently. Read the current README before invoking them.