DevOps GitOps and Packaging
Agent Workflow (MANDATORY)
Before ANY chart, overlay, Application or Flux object, spawn in parallel, one spawn_agent call each:
- explore-codebase - find existing
Chart.yaml, kustomization.yaml, Argo CD or Flux manifests and the repo layout
- research-expert - verify fields and flags for the installed tool version at helm.sh, kubectl.docs.kubernetes.io, argo-cd.readthedocs.io, fluxcd.io
- mcp__context7__query-docs -
/websites/helm_sh, /helm/helm, /kubernetes-sigs/kustomize, /websites/argo-cd_readthedocs_io_en_stable, /websites/fluxcd_io_flux
Then read the target read-only (helm version, helm list -A, argocd version, flux check, kubectl get crd | grep -E 'argoproj|fluxcd'). After changes, run sniper.
Versions (checked 2026-11-10)
Helm 4.3.1 (https://helm.sh/docs/overview/), Kustomize 5.8.3 (https://github.com/kubernetes-sigs/kustomize/releases), Argo CD 3.6.1 (https://github.com/argoproj/argo-cd/releases/tag/v3.6.1), Flux 2.10.0 (https://github.com/fluxcd/flux2/releases/tag/v2.10.0). API versions used here: argoproj.io/v1alpha1, source.toolkit.fluxcd.io/v1, kustomize.toolkit.fluxcd.io/v1, helm.toolkit.fluxcd.io/v2, image.toolkit.fluxcd.io/v1, kustomize.config.k8s.io/v1beta1. Read the installed version first (helm version, argocd version, flux version); the CRDs on the cluster win over this file.
Pinned versions:
helm: "4.3.1 (helm.sh docs 'Version: 4.3.1')"
kustomize: "5.8.3"
argo-cd: "3.6.1 (released 2026-11-04)"
flux: "2.10.0 (released 2026-10-26)"
Overview
| Area |
Decision in one line |
| Packaging |
Helm for third-party or parameterised apps; Kustomize for your own plain manifests; both only with a stated reason (helm-vs-kustomize.md) |
| Helm 4 |
Server-side apply default for new releases, --atomic renamed --rollback-on-failure, --force renamed --force-replace, post-renderers are plugins (helm-4-changes.md) |
| Controller |
Argo CD when a UI, multi-cluster and per-app RBAC matter; Flux when everything should be CRDs reconciled in-cluster; one controller per cluster, never both on the same objects |
| Delivery |
Git is the source of truth; change Git, not the cluster; pin charts and images by version or digest |
| Pruning |
Off by default or confirm; a prune is a delete (see Critical Rules) |
| Promotion |
One artifact, environment-specific config only; promote by PR, not by rebuilding (repo-layout-promotion.md) |
Critical Rules
- Version first - read the tool versions and the CRDs on the cluster before writing a field; an apiVersion not in the cluster is not usable
- Render, then diff, then apply -
helm template / kustomize build, then helm diff upgrade or kubectl diff, argocd app diff, flux diff kustomization; never sync blind
- Destructive verbs are proposed, not run -
helm uninstall, helm rollback, Argo CD sync with prune, argocd app delete --cascade, flux delete, flux suspend, kustomize build | kubectl delete -f -: exact ACTION / EFFECT / PROOF, then WAIT-FOR-GO (helm-release-ops.md)
- Never hand-edit a GitOps-owned object - the controller reverts it; the fix is a commit
- No secret value in Git or in output - names and keys only, values via devops-secrets (SOPS, External Secrets);
helm template/--dry-run print Secrets, so use --hide-secret
- Pin everything - chart version,
targetRevision as tag or SHA, image tag plus digest; no HEAD, latest or floating range in production
- Never invent a field or flag - cite the official page; unknown = "not verified"
Architecture
gitops/
├── charts/<app>/ # Helm chart (Chart.yaml, values.yaml, values.schema.json, templates/)
├── apps/<app>/base/ # Kustomize base
├── apps/<app>/overlays/{staging,prod}/
├── clusters/{staging,prod}/ # per-cluster entry: Flux Kustomizations or Argo CD Applications
└── infrastructure/ # controllers and shared config, reconciled before apps
Reference Guide
Concepts
| Topic |
Reference |
When to Consult |
| Helm 4 changes |
helm-4-changes.md |
Moving from Helm 3, CI flags, SSA, plugins |
| Chart authoring |
helm-chart-authoring.md |
Structure, values, schema, dependencies, hooks, tests |
| Release operations |
helm-release-ops.md |
OCI push/pull, diff, upgrade, rollback, uninstall |
| Kustomize |
kustomize.md |
Bases, overlays, patches, generators, components |
| Helm vs Kustomize |
helm-vs-kustomize.md |
Choosing, combining, helmCharts limits |
| Argo CD |
argocd-applications.md |
Application, AppProject, sync policy, waves, hooks, deletion |
| ApplicationSet |
argocd-applicationset-appofapps.md |
Many apps or clusters, app-of-apps |
| Flux |
flux.md |
Sources, Kustomization, HelmRelease, suspend/delete |
| Flux image automation |
flux-image-automation.md |
Tag scanning and commits back to Git |
| Layout and promotion |
repo-layout-promotion.md |
Environments, promotion, drift, rollback, secrets pointer |
Templates
| Template |
When to Use |
| chart/ |
Minimal chart with values.schema.json (Helm templates are not parseable YAML) |
| kustomize/ |
Base + prod overlay + component |
| argocd/appproject.yaml |
Project limiting repos, destinations, resource kinds |
| argocd/application.yaml |
Application with manual-prune safe sync policy |
| argocd/applicationset.yaml |
One app per environment directory |
| flux/gitrepository.yaml |
Git source |
| flux/kustomization.yaml |
Reconcile a path with ordering and health checks |
| flux/helmrelease.yaml |
OCI chart, remediation, drift detection |
| flux/image-automation.yaml |
ImageRepository, ImagePolicy, ImageUpdateAutomation |
| gitops-repo-layout.md |
Repository tree and promotion flow |
Other files of the directory templates above: chart/values.yaml, chart/values.schema.json, chart/templates/_helpers.tpl, chart/templates/deployment.yaml, kustomize/base/deployment.yaml, kustomize/overlays/prod/kustomization.yaml, kustomize/components/ingress-tls/kustomization.yaml.
Quick Reference
CTX=prod-eu; NS=shop # always explicit
helm template web ./charts/web -f values-prod.yaml --hide-secret >/dev/null # render only
helm --kube-context "$CTX" -n "$NS" diff upgrade web ./charts/web -f values-prod.yaml # helm-diff plugin
helm --kube-context "$CTX" -n "$NS" upgrade web ./charts/web -f values-prod.yaml --dry-run=server --hide-secret
kustomize build apps/web/overlays/prod | kubectl --context "$CTX" diff -f - # exit 1 = differences
argocd app diff web --refresh # exit 1 = differences
flux diff kustomization web --path ./apps/web/overlays/prod # exit 1 = differences
flux reconcile kustomization web --with-source # after the user's go
Best Practices
DO
- Add
values.schema.json to every chart you own so bad values fail at render time
- Keep one Application or Kustomization per deployable unit, with
dependsOn or sync waves for ordering
- Give Argo CD
AppProject or Flux service accounts the narrowest repos, namespaces and kinds
- Record the rollback target (chart version, Git SHA, Helm revision) in the change proposal
DON'T
- Enable automated prune or
selfHeal on a production Application without the owner's go
- Run
helm upgrade --reuse-values across chart versions (stale values hide new defaults)
- Let Argo CD and Flux, or Helm CLI and a controller, manage the same release
- Template secrets into values files committed in clear text
Related skills
- devops-kubernetes
- devops-k8s-cluster
- devops-secrets
- devops-cicd
- devops-docker
- devops-observability
- devops-sre-ops
1---2name: devops-gitops-packaging-23description: Use when packaging or delivering workloads to Kubernetes - Helm 4 charts (values, values.schema.json, dependencies, OCI registries, upgrade/rollback/test), Kustomize bases, overlays, patches and components, choosing Helm vs Kustomize vs both, Argo CD (Application, ApplicationSet, AppProject, sync policy, waves, hooks, app-of-apps), Flux (GitRepository, OCIRepository, Kustomization, HelmRelease, image automation), GitOps repository layout, promotion between environments, drift and rollback. Not for the Kubernetes resources themselves (devops-kubernetes), cluster install or upgrade (devops-k8s-cluster), CI pipelines that build images (devops-cicd) or secret stores (devops-secrets).4metadata:5 source: https://skillmd.com/skills/fusengine/devops-gitops-packaging-26---78<objective>9Covers turning manifests into a versioned, reviewable, reversible delivery:10Helm 4 charts and releases, Kustomize overlays, the Helm vs Kustomize choice,11Argo CD and Flux controllers, repository layout, promotion, drift and rollback.12Does not cover the Kubernetes objects themselves (devops-kubernetes), cluster13build and upgrade (devops-k8s-cluster), the CI that builds and signs images14(devops-cicd, devops-docker) or secret stores (devops-secrets).15</objective>1617# DevOps GitOps and Packaging1819## Agent Workflow (MANDATORY)2021Before ANY chart, overlay, Application or Flux object, spawn in parallel, one `spawn_agent` call each:22231. **explore-codebase** - find existing `Chart.yaml`, `kustomization.yaml`, Argo CD or Flux manifests and the repo layout242. **research-expert** - verify fields and flags for the installed tool version at helm.sh, kubectl.docs.kubernetes.io, argo-cd.readthedocs.io, fluxcd.io253. **mcp__context7__query-docs** - `/websites/helm_sh`, `/helm/helm`, `/kubernetes-sigs/kustomize`, `/websites/argo-cd_readthedocs_io_en_stable`, `/websites/fluxcd_io_flux`2627Then read the target read-only (`helm version`, `helm list -A`, `argocd version`, `flux check`, `kubectl get crd | grep -E 'argoproj|fluxcd'`). After changes, run **sniper**.2829## Versions (checked 2026-11-10)3031Helm 4.3.1 (https://helm.sh/docs/overview/), Kustomize 5.8.3 (https://github.com/kubernetes-sigs/kustomize/releases), Argo CD 3.6.1 (https://github.com/argoproj/argo-cd/releases/tag/v3.6.1), Flux 2.10.0 (https://github.com/fluxcd/flux2/releases/tag/v2.10.0). API versions used here: `argoproj.io/v1alpha1`, `source.toolkit.fluxcd.io/v1`, `kustomize.toolkit.fluxcd.io/v1`, `helm.toolkit.fluxcd.io/v2`, `image.toolkit.fluxcd.io/v1`, `kustomize.config.k8s.io/v1beta1`. Read the installed version first (`helm version`, `argocd version`, `flux version`); the CRDs on the cluster win over this file.3233Pinned versions:3435```yaml36helm: "4.3.1 (helm.sh docs 'Version: 4.3.1')"37kustomize: "5.8.3"38argo-cd: "3.6.1 (released 2026-11-04)"39flux: "2.10.0 (released 2026-10-26)"40```4142## Overview4344| Area | Decision in one line |45|------|----------------------|46| **Packaging** | Helm for third-party or parameterised apps; Kustomize for your own plain manifests; both only with a stated reason ([helm-vs-kustomize.md](references/helm-vs-kustomize.md)) |47| **Helm 4** | Server-side apply default for new releases, `--atomic` renamed `--rollback-on-failure`, `--force` renamed `--force-replace`, post-renderers are plugins ([helm-4-changes.md](references/helm-4-changes.md)) |48| **Controller** | Argo CD when a UI, multi-cluster and per-app RBAC matter; Flux when everything should be CRDs reconciled in-cluster; one controller per cluster, never both on the same objects |49| **Delivery** | Git is the source of truth; change Git, not the cluster; pin charts and images by version or digest |50| **Pruning** | Off by default or `confirm`; a prune is a delete (see Critical Rules) |51| **Promotion** | One artifact, environment-specific config only; promote by PR, not by rebuilding ([repo-layout-promotion.md](references/repo-layout-promotion.md)) |5253## Critical Rules54551. **Version first** - read the tool versions and the CRDs on the cluster before writing a field; an apiVersion not in the cluster is not usable562. **Render, then diff, then apply** - `helm template` / `kustomize build`, then `helm diff upgrade` or `kubectl diff`, `argocd app diff`, `flux diff kustomization`; never sync blind573. **Destructive verbs are proposed, not run** - `helm uninstall`, `helm rollback`, Argo CD sync with prune, `argocd app delete --cascade`, `flux delete`, `flux suspend`, `kustomize build | kubectl delete -f -`: exact ACTION / EFFECT / PROOF, then WAIT-FOR-GO ([helm-release-ops.md](references/helm-release-ops.md))584. **Never hand-edit a GitOps-owned object** - the controller reverts it; the fix is a commit595. **No secret value in Git or in output** - names and keys only, values via devops-secrets (SOPS, External Secrets); `helm template`/`--dry-run` print Secrets, so use `--hide-secret`606. **Pin everything** - chart version, `targetRevision` as tag or SHA, image tag plus digest; no `HEAD`, `latest` or floating range in production617. **Never invent a field or flag** - cite the official page; unknown = "not verified"6263## Architecture6465```66gitops/67├── charts/<app>/ # Helm chart (Chart.yaml, values.yaml, values.schema.json, templates/)68├── apps/<app>/base/ # Kustomize base69├── apps/<app>/overlays/{staging,prod}/70├── clusters/{staging,prod}/ # per-cluster entry: Flux Kustomizations or Argo CD Applications71└── infrastructure/ # controllers and shared config, reconciled before apps72```7374## Reference Guide7576### Concepts7778| Topic | Reference | When to Consult |79|-------|-----------|-----------------|80| **Helm 4 changes** | [helm-4-changes.md](references/helm-4-changes.md) | Moving from Helm 3, CI flags, SSA, plugins |81| **Chart authoring** | [helm-chart-authoring.md](references/helm-chart-authoring.md) | Structure, values, schema, dependencies, hooks, tests |82| **Release operations** | [helm-release-ops.md](references/helm-release-ops.md) | OCI push/pull, diff, upgrade, rollback, uninstall |83| **Kustomize** | [kustomize.md](references/kustomize.md) | Bases, overlays, patches, generators, components |84| **Helm vs Kustomize** | [helm-vs-kustomize.md](references/helm-vs-kustomize.md) | Choosing, combining, `helmCharts` limits |85| **Argo CD** | [argocd-applications.md](references/argocd-applications.md) | Application, AppProject, sync policy, waves, hooks, deletion |86| **ApplicationSet** | [argocd-applicationset-appofapps.md](references/argocd-applicationset-appofapps.md) | Many apps or clusters, app-of-apps |87| **Flux** | [flux.md](references/flux.md) | Sources, Kustomization, HelmRelease, suspend/delete |88| **Flux image automation** | [flux-image-automation.md](references/flux-image-automation.md) | Tag scanning and commits back to Git |89| **Layout and promotion** | [repo-layout-promotion.md](references/repo-layout-promotion.md) | Environments, promotion, drift, rollback, secrets pointer |9091### Templates9293| Template | When to Use |94|----------|-------------|95| [chart/](references/templates/chart/Chart.yaml) | Minimal chart with `values.schema.json` (Helm templates are not parseable YAML) |96| [kustomize/](references/templates/kustomize/base/kustomization.yaml) | Base + prod overlay + component |97| [argocd/appproject.yaml](references/templates/argocd/appproject.yaml) | Project limiting repos, destinations, resource kinds |98| [argocd/application.yaml](references/templates/argocd/application.yaml) | Application with manual-prune safe sync policy |99| [argocd/applicationset.yaml](references/templates/argocd/applicationset.yaml) | One app per environment directory |100| [flux/gitrepository.yaml](references/templates/flux/gitrepository.yaml) | Git source |101| [flux/kustomization.yaml](references/templates/flux/kustomization.yaml) | Reconcile a path with ordering and health checks |102| [flux/helmrelease.yaml](references/templates/flux/helmrelease.yaml) | OCI chart, remediation, drift detection |103| [flux/image-automation.yaml](references/templates/flux/image-automation.yaml) | ImageRepository, ImagePolicy, ImageUpdateAutomation |104| [gitops-repo-layout.md](references/templates/gitops-repo-layout.md) | Repository tree and promotion flow |105106Other files of the directory templates above: [chart/values.yaml](references/templates/chart/values.yaml), [chart/values.schema.json](references/templates/chart/values.schema.json), [chart/templates/_helpers.tpl](references/templates/chart/templates/_helpers.tpl), [chart/templates/deployment.yaml](references/templates/chart/templates/deployment.yaml), [kustomize/base/deployment.yaml](references/templates/kustomize/base/deployment.yaml), [kustomize/overlays/prod/kustomization.yaml](references/templates/kustomize/overlays/prod/kustomization.yaml), [kustomize/components/ingress-tls/kustomization.yaml](references/templates/kustomize/components/ingress-tls/kustomization.yaml).107108## Quick Reference109110```bash111CTX=prod-eu; NS=shop # always explicit112helm template web ./charts/web -f values-prod.yaml --hide-secret >/dev/null # render only113helm --kube-context "$CTX" -n "$NS" diff upgrade web ./charts/web -f values-prod.yaml # helm-diff plugin114helm --kube-context "$CTX" -n "$NS" upgrade web ./charts/web -f values-prod.yaml --dry-run=server --hide-secret115kustomize build apps/web/overlays/prod | kubectl --context "$CTX" diff -f - # exit 1 = differences116argocd app diff web --refresh # exit 1 = differences117flux diff kustomization web --path ./apps/web/overlays/prod # exit 1 = differences118flux reconcile kustomization web --with-source # after the user's go119```120121## Best Practices122123### DO124- Add `values.schema.json` to every chart you own so bad values fail at render time125- Keep one Application or Kustomization per deployable unit, with `dependsOn` or sync waves for ordering126- Give Argo CD `AppProject` or Flux service accounts the narrowest repos, namespaces and kinds127- Record the rollback target (chart version, Git SHA, Helm revision) in the change proposal128129### DON'T130- Enable automated prune or `selfHeal` on a production Application without the owner's go131- Run `helm upgrade --reuse-values` across chart versions (stale values hide new defaults)132- Let Argo CD and Flux, or Helm CLI and a controller, manage the same release133- Template secrets into values files committed in clear text134135---136137## Related skills138139- devops-kubernetes140- devops-k8s-cluster141- devops-secrets142- devops-cicd143- devops-docker144- devops-observability145- devops-sre-ops