K8S Local

A project's stack on a local Kubernetes cluster

Docker Compose stops being enough once the app needs ingress hostnames, subdomain routing, or the same manifests it runs in production. A local Kubernetes cluster gives you that, and adds three traps that cost an afternoon each: an image the kubelet refuses to see, a hostname that routes nowhere while everything reports healthy, and a login that fails only over plain http.

klocal drives the stack; the project's shape lives in a committed .k8s-local/project.json, so the same tool works in every repo.

The rule that matters most

Never touch a cluster without checking the kubecontext first. One stale context is the whole difference between a local bring-up and a production deploy, and kubectl will not ask. Two independent checks, because neither is enough alone:

1. The name must be rancher-desktop, docker-desktop, minikube, colima, kind-* or k3d-*. A whitelist, not a heuristic — "contains the word local" would accept a production cluster called localstack-prod.
2. The API server address must be loopback or RFC1918, and the URL is parsed, not string-matched. Names are chosen by whoever created the cluster, so a remote cluster can be called kind-prod; the address is the part that naming cannot fake — provided you strip the userinfo first, since https://127.0.0.1:x@prod.example.com:6443 is a legal URL whose host is prod.example.com.

No context at all is refused the same way a remote one is. Every command that reaches the cluster is guarded — including logs and psql, which read and write through a pod — and the verified context is pinned with --context on every call, so switching the kubeconfig mid-build cannot redirect the rest.

Pinning carries the name forward, not the cluster it named, so every write path (up, rebuild, down) re-reads the server URL immediately before it writes. The windows are real: a build takes minutes, and down waits at its confirmation prompt for as long as you take to answer.

If a legitimate local context is refused, add its exact name to kl_context_name_is_local and a case to the test suite; never loosen the pattern.

Do not

• Push a local image to a registry. Build it straight into the cluster's image store. Keep imagePullPolicy at IfNotPresent — it is the default for an ordinary tag, but Always for :latest or no tag, and under Always the kubelet ignores the local image and tries to pull it. Clear any inherited imagePullSecrets too; that does not affect whether the local image is used, it just removes a reference to a registry credential this cluster has not got. And on kind, k3d or minikube the build must additionally be imported into the node's own store, or the pod keeps running the previous image.
• Pick the build engine by which binary exists. Rancher Desktop ships nerdctl even when the engine is moby, where it cannot reach the k3s containerd socket. Probe the socket: nerdctl --namespace k8s.io info, then docker info. And do not stop at the socket either — with Rancher Desktop on containerd and Docker Desktop installed, both probes answer. Decide by the context: a docker-desktop cluster reads the docker store no matter what else replies.
• Report a read you could not do as a finding. "Not set on that Deployment" and "I was not allowed to look" are different answers, and only the first is about the manifest. klocal status says which one it has.
• Let the app connect to Postgres as the container's superuser. It bypasses every FORCE ROW LEVEL SECURITY policy, so tenant isolation is silently off while all tests still pass. Create a NOSUPERUSER NOBYPASSRLS role, and assert it before declaring the stack up.
• Regenerate a password or signing key on a re-run. Redis keeps the --requirepass it booted with, so the app crash-loops on WRONGPASS; a rotated signing key breaks an already-running dev server. Read the existing value first, generate only when it is absent.
• Declare an env var twice. It is malformed input to a merge-keyed list, and what happens next depends on the path: a kustomize strategic-merge patch collapses it and keeps the first, silently, before kubectl sees anything; a plain manifest applied client-side keeps both, warns, and the kubelet uses the last; server-side apply rejects it outright. Do not memorise a winner — the point is that the manifest cannot tell you and only the live object can. klocal status reports the duplicate next to the value actually in effect.
• Give the local datastores a volume. They are meant to be throwaway; say so in the project's docs rather than making local state precious.

Commands

Common workflows

Set a project up. scaffold writes the manifests and config; every file carries PROJECT placeholders and comments explaining what to decide.

klocal scaffold                       # from the project root
$EDITOR .k8s-local/project.json       # then the PROJECT placeholders in deploy/
klocal up

Get secrets in without writing them to disk. klocal never handles secret values. Point secret.hook at a command that creates the Secret, and it runs from the project root with KL_NAMESPACE, KL_SECRET, KL_SECRET_ENV_FILE and KL_KUBECTL exported. Use $KL_KUBECTL inside the hook — it carries the pinned --context; a bare kubectl there is unpinned and follows the kubeconfig. Piping a vault straight into kubectl keeps values out of both the filesystem and the transcript — see the onepassword skill for opgate.

"secret": { "hook": "opgate run -f api/.env.op -- ./deploy/scripts/local-secret.sh" }

Check what is actually running. Never report the manifest's intent as fact. klocal status prints the live pods, services and ingress, whether the datastores are ephemeral, and any env var a manifest declares twice alongside the value that actually won.

Before you say it works

A green rollout means the pods started, not that the stack works. Confirm the hostname answers, and confirm the answer came from the app rather than the ingress controller — a 404 from the app and a 404 from Traefik look identical until you read the body.

Further reading

• references/bring-up.md — the procedure step by step, and why it is ordered so
• references/engines.md — Rancher Desktop, kind, k3d, minikube: what differs
• references/troubleshooting.md — symptom, cause, fix
• references/example-cme.md — a real multi-tenant stack, worked end to end