# Zitadel Helm

> Use when deploying, configuring, or upgrading ZITADEL on Kubernetes via Helm. Covers HelmRelease values, CNPG database, Gateway API routing, caches, masterkey, SMTP, and production patterns. NOT for API-level operations (orgs/apps/users).

- Skill: `aidas-dev/zitadel-helm` (Agent Skill)
- Install (CLI): `npx skillmds@latest add aidas-dev/zitadel-helm`
- Raw SKILL.md: https://api.skillmd.com/api/skills/aidas-dev/zitadel-helm/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: Aidas-dev (https://skillmd.com/u/aidas-dev)
- Updated: 2026-09-21
- Page: https://skillmd.com/skills/aidas-dev/zitadel-helm

---


# ZITADEL Helm

## Overview

ZITADEL deployed via HelmRelease (Flux) with external PostgreSQL (CNPG), Gateway API routing (Cilium Gateway), memory caching. Two containers: main ZITADEL API (Go) and Login UI (Next.js). Init + setup jobs bootstrap the FirstInstance on install.

**Latest stable:** chart 10.0.4 → ZITADEL v4.15.3. See [artifacthub](https://artifacthub.io/packages/helm/zitadel/zitadel) for latest version.

## Architecture

```
Gateway API (e.g., Cilium, Istio, Envoy Gateway)
  ├── HTTPRoute "zitadel"         → zitadel:8080 (h2c)
  ├── HTTPRoute "zitadel-login"   → zitadel-login:3000  (/ui/v2/login)
  └── GRPCRoute (optional)        → zitadel:8080 (gRPC)

ZITADEL Deployment (zitadel:8080)
  ├── Init job (schema bootstrap)
  ├── Setup job (FirstInstance creation)
  └── Env vars → DB credentials from K8s secrets

ZITADEL Login Deployment (login:3000)
  └── Authenticates to ZITADEL via X.509 RSA keypair (auto-generated by Helm chart)

CNPG Cluster (zitadel-db, PostgreSQL 18)
  └── zitadel-db-rw:5432 → ZITADEL connects via DSN
```

## Key Configuration

### Database (CNPG — External)

Disabled bundled PostgreSQL subchart. CNPG cluster created separately:

```yaml
# HelmRelease values
postgresql:
  enabled: false

zitadel:
  configmapConfig:
    Database:
      Postgres:
        Host: zitadel-db-rw.zitadel.svc.cluster.local
        Port: 5432
        Database: zitadel
        User:
          Username: zitadel
          SSL:
            Mode: disable   # or require/verify-full with dbSslCaCrt
        Admin:
          Username: zitadel
          SSL:
            Mode: disable
  env:
    - name: ZITADEL_DATABASE_POSTGRES_USER_PASSWORD
      valueFrom:
        secretKeyRef:
          name: zitadel-db-app
          key: password
    - name: ZITADEL_DATABASE_POSTGRES_ADMIN_PASSWORD
      valueFrom:
        secretKeyRef:
          name: zitadel-db-app
          key: password
```

Alternative DSN-based config (replaces `configmapConfig.Database.Postgres`):

```yaml
zitadel:
  env:
    - name: ZITADEL_DATABASE_POSTGRES_DSN
      valueFrom:
        secretKeyRef:
          name: zitadel-db-credentials
          key: dsn
```

### Secrets Management

Sensitive values via `valuesFrom` in HelmRelease (not plaintext):

```yaml
# release.yaml
valuesFrom:
  - kind: Secret
    name: zitadel-secrets
    valuesKey: masterkey
    targetPath: zitadel.masterkey
  - kind: Secret
    name: zitadel-db-credentials
    valuesKey: password
    targetPath: zitadel.configmapConfig.Database.Postgres.User.Password
  - kind: Secret
    name: zitadel-db-credentials
    valuesKey: password
    targetPath: zitadel.configmapConfig.Database.Postgres.Admin.Password
  - kind: Secret
    name: zitadel-secrets
    valuesKey: smtpPassword
    targetPath: zitadel.configmapConfig.Notifications.SMTP.Password
```

> **Chart 10.x change:** `global.sharedSecret` was removed. Login UI auth switched from PAT (chart 9.x) to X.509 RSA keypair. Helm auto-generates the keypair on install and reuses it across upgrades. Set `login.loginServiceKeySecretName` to use an existing secret instead.

**Masterkey:** Must be exactly 32 bytes printable ASCII. Generate:
```bash
tr -dc A-Za-z0-9 </dev/urandom | head -c 32
```
Set via `zitadel.masterkey` or `zitadel.masterkeySecretName`. Loss = data loss.

### Gateway API Routing

```yaml
ingress:
  enabled: false   # Use Gateway API instead

gateway:
  grpcRoute:
    enabled: true
    parentRefs:
      - kind: Gateway
        name: my-gateway
        namespace: gateway-ns
        sectionName: https
```

Separate HTTPRoute resources for main API and login:

| Route | Host | Backend | Purpose |
|---|---|---|---|
| `zitadel` (HTTPRoute) | auth.example.com | zitadel:8080 | Main API, console |
| `zitadel-login` (HTTPRoute) | auth.example.com | zitadel-login:3000 | Login UI at /ui/v2/login |
| `zitadel-grpc` (GRPCRoute, auto) | auth.example.com | zitadel:8080 | gRPC (if gateway.grpcRoute.enabled) |

Both HTTPRoutes match the same hostname with path-based splitting.

**Important:** ZITADEL requires end-to-end HTTP/2 (h2c). The Gateway must support h2c backend connections (Cilium does natively).

### Caching

Memory cache for small deployments:

```yaml
zitadel:
  configmapConfig:
    Caches:
      Connectors:
        Memory:
          Enabled: true
      Instance:
        Connector: memory
        MaxAge: 1h
      Organization:
        Connector: memory
        MaxAge: 1h
```

Production (Redis/Valkey):

```yaml
Caches:
  Connectors:
    Redis:
      Enabled: true
      Addr: redis-cluster:6379
  Instance:
    Connector: redis
    MaxAge: 10m
```

### FirstInstance / Bootstrapping

Configured in `configmapConfig.FirstInstance`. Helm chart auto-generates Machine Keys + PAT secrets:

```yaml
zitadel:
  configmapConfig:
    FirstInstance:
      Org:
        Human:
          UserName: admin
          Password: "Admin123!"
          FirstName: Admin
          LastName: User
          Email: admin@example.com
          PasswordChangeRequired: false
        Machine:
          Machine:
            Username: iam-admin
          Pat:
            ExpirationDate: "2029-01-01T00:00:00Z"
```

After install, secrets created:
- `iam-admin` — JWT machine key
- `iam-admin-pat` — Personal Access Token (for admin API access)
- `{release}-login-service-key` — X.509 RSA keypair for Login UI auth (chart 10.x+)
- `login-client` — Login UI credential (chart 9.x: PAT; removed in 10.x, replaced by X.509 keypair)

### Node Selection

```yaml
nodeSelector:
  kubernetes.io/hostname: dedicated-node

login:
  nodeSelector:
    kubernetes.io/hostname: dedicated-node
```

### SMTP / Notifications

```yaml
zitadel:
  configmapConfig:
    Notifications:
      SMTP:
        Host: smtp.example.com
        Port: 587
        User: noreply@example.com
        Password: ""     # via valuesFrom secret
        SenderAddress: noreply@example.com
        SenderName: Zitadel Auth
```

## Upgrade

### General

1. Update `version:` in HelmRelease
2. ZITADEL handles its own DB migrations via init + setup jobs
3. Helm hooks orchestrate order: init → setup → deployment
4. No manual migration steps needed for patch/minor versions
5. **Check release notes** for breaking changes between major chart versions
6. ZITADEL supports PG 14–18. PG 18 requires ZITADEL v4.11.0+
7. Masterkey never changes after initial install
8. After upgrade, verify console login, OIDC app auth, and DB migration logs

### Chart 9.x → 10.x Migration

**Breaking changes:**

| Change | Impact |
|--------|--------|
| Login UI auth: PAT → X.509 RSA keypair | Helm auto-generates keypair. Set `login.loginServiceKeySecretName` to use an existing secret |
| `global.domain` removed | Set `zitadel.configmapConfig.ExternalDomain` instead |
| `initJob.command` removed | Template handles the command internally |
| `global.sharedSecret` removed | No longer needed — X.509 cert replaces cookie-based login auth |
| `FirstInstance.MachineKeyPath`/`PatPath` auto-managed | Helm manages these internally; setting manually causes failures |

**Required CNPG changes:** The init job in chart 10.x tries to CREATE DATABASE on upgrade. Your PostgreSQL user needs both `LOGIN` and `CREATEDB` privileges. For CNPG clusters, add a `managed.roles` section:

```yaml
# In your CNPG Cluster spec
managed:
  roles:
    - name: zitadel        # or your DB user
      login: true          # required — CNPG defaults to login: false
      createdb: true       # required for init job
      ensure: present
```

Without `login: true`, the init job fails with: `FATAL: role "zitadel" is not permitted to log in (SQLSTATE 28000)`.
Without `createdb: true`, the init job fails with: `permission denied to create database (SQLSTATE 42501)`.

**Upgrade procedure:**
1. Apply CNPG `managed.roles` first (before chart upgrade)
2. Update chart version from 9.x to 10.x
3. Delete these deprecated values (if present): `global.domain`, `initJob.command`, `global.sharedSecret` valuesFrom
4. Let Helm run init + setup jobs (DB migration)
5. Verify console login and OIDC app auth

## Quick Reference

| Need | Config path |
|---|---|
| External domain | `zitadel.configmapConfig.ExternalDomain` |
| TLS termination | At Gateway, set `TLS.Enabled: false` |
| Database host | `zitadel.configmapConfig.Database.Postgres.Host` |
| DB SSL mode | `Database.Postgres.User.SSL.Mode` (disable/require/verify-full) |
| Masterkey | `zitadel.masterkey` or `zitadel.masterkeySecretName` |
| Admin bootstrap | `zitadel.configmapConfig.FirstInstance.Org.Human` |
| SMTP | `zitadel.configmapConfig.Notifications.SMTP.*` |
| Memory cache | `zitadel.configmapConfig.Caches.Connectors.Memory.Enabled: true` |
| Node pinning | `nodeSelector`, `login.nodeSelector` |
| Disable bundled PG | `postgresql.enabled: false` |
| Disable Ingress | `ingress.enabled: false` (use Gateway API instead) |
| Enable gRPC route | `gateway.grpcRoute.enabled: true` |

## Common Mistakes

- **Masterkey loss = permanent data loss.** Cannot recover. Store in SealedSecret or external vault.
- **Bundled PostgreSQL subchart NOT for production.** Always disable (`postgresql.enabled: false`) and use external CNPG cluster.
- **Gateway must support h2c backend.** Without h2c, ZITADEL console fails. Cilium Gateway supports h2c natively.
- **Login route must precede main route** on the same hostname, otherwise /ui/v2/login matches the catch-all `/` rule and hits the main API instead of the login UI.
- **env vars take precedence over configmapConfig.** If both `ZITADEL_DATABASE_POSTGRES_DSN` env var and `configmapConfig.Database.Postgres` are set, the env var wins.
- **PasswordChangeRequired: true** on FirstInstance.Human will block initial console login until admin changes password.
- **SMTP password** must be via secret, never in values.yaml. Use `valuesFrom` in HelmRelease.
- **FirstInstance fields are install-only.** Changing them post-install has no effect. Modify via API or DB directly.
- **Chart 10.x init job needs CREATEDB.** If using CNPG, add `managed.roles` with `login: true` and `createdb: true`. Without `login: true`, init job fails with `FATAL: role "x" is not permitted to log in`.
- **Chart 10.x removed `global.domain`.** Use `zitadel.configmapConfig.ExternalDomain` instead. Also removed: `initJob.command`, `global.sharedSecret`.
- **Chart 10.x login auth changed.** Login UI now uses X.509 RSA keypair (auto-generated) instead of PAT. Chart preserves existing keypair across upgrades via `lookup`.
- **Login UI key rotation.** To force new keypair, delete the `{release}-login-service-key` secret and trigger Helm upgrade. New keypair is auto-generated.