OpenChoreo Platform-Engineer GitOps Guide
Git is the source of truth; the cluster is its reflection. This skill writes OpenChoreo platform resources to Git, lets Flux CD reconcile them, and reads cluster state with occ to verify.
This skill is scoped to platform-engineer GitOps work — initial scaffold + ongoing platform authoring. Application-level GitOps (Project / Component / Workload / ComponentRelease / ReleaseBinding) and non-GitOps cluster operations (Helm install, direct CRD edits) are out of scope; tell the user when a task crosses that boundary.
Step 0 — Prerequisites
occ must be configured against the target cluster:
command -v occ && occ config context list # occ configured? which control plane?
occ namespace list # cluster reachable + logged in?
If occ points at the wrong control plane, or occ login / occ namespace list fails, configure it for the target before proceeding. Details: CLI Configuration and CLI Installation → Login.
- Repoint the shipped
default — every install ships a default control plane + context; if you only need occ on this one cluster, just update its URL and log in:occ config controlplane update default --url https://api.<cp-domain>
occ login
- Add a second control plane alongside an existing one (e.g. keep a local k3d entry and add a remote cluster, switching between them with
occ config context use) — context add requires both --controlplane and --credentials, each created first:occ config controlplane add <cp> --url https://api.<cp-domain>
occ config credentials add <cp>
occ config context add <cp> --controlplane <cp> --credentials <cp> --namespace <ns>
occ config context use <cp> && occ login
- Self-signed certs (
x509: certificate is not trusted on login) — trust the CA in your OS trust store (extract, then import per OS — see the CLI Installation doc):kubectl get secret openchoreo-ca-secret -n cert-manager \
-o jsonpath='{.data.ca\.crt}' | base64 -d > openchoreo-ca.crt
# macOS: sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain openchoreo-ca.crt
occ login is interactive (browser) — hand it to the user if you can't complete it.
Step 1 — Load concepts (MANDATORY)
Read references/concepts.md in full before anything else. Not optional, even if the task looks simple — resource model / sync ordering / immutability / verification ladder / drift recovery are all easy to get wrong from memory. Load once per session; if you catch yourself acting without it, stop and load now.
Load other references on-demand as the task needs them:
references/authoring.md — where CRD shapes come from (live cluster / scripts/extract-resources.sh for defaults + GitOps workflows / scripts/fetch-page.sh for docs / ecosystem catalogs for community extensions), repo path conventions, the cluster↔namespace scope swap, the CI-vs-GitOps workflow gotcha, git workflow, DCO.
scripts/fetch-page.sh — fetch any OpenChoreo docs page by title (resolves against llms.txt, picks a stable version). Use this for full CRD schemas with optional fields; --section "API Reference" scopes matching to CRD-reference pages, --list dumps the index.
scripts/extract-resources.sh — extract default platform resources from samples/getting-started/all.yaml (defaults --kind X [--name Y]), or GitOps build-and-release workflows discovered via the ecosystem catalog (gitops-workflows --list / --name <slug>). Refuses vanilla CI workflows by default.
references/cel.md — only when writing or reviewing CEL inside ComponentType / Trait / Workflow templates.
Detect the mode
{ ls flux 2>/dev/null || ls clusters 2>/dev/null; } \
&& { ls platform-shared 2>/dev/null || ls namespaces 2>/dev/null; }
- Scaffolding mode — cwd is empty or not yet a GitOps repo. Go to
recipes/scaffold.md. Covers pristine / platform-only / active-cluster paths.
- Operating mode — cwd has a Flux entrypoint (
flux/ for single-cluster, clusters/<name>/ for multi-cluster) plus at least one of platform-shared/ or namespaces/. Skip to the relevant author / install recipe.
If the heuristic doesn't fit the layout (per the docs' Flexible Repository Structures — repo-per-project, separate-releasebindings-repo, etc.), ask the user.
What this skill can do
What this skill cannot do
- Application-level GitOps —
Project / ProjectReleaseBinding / Component / Workload / ComponentRelease / ReleaseBinding / Resource / ResourceReleaseBinding / workload-descriptor authoring. Out of scope; tell the user when the task crosses into application territory. (PE-side: (Cluster)ProjectType and (Cluster)ResourceType templates belong here; the dev-side Project instances + their ProjectReleaseBindings / ResourceReleaseBindings belong in the developer-gitops skill. One exception: the default project's per-env ProjectReleaseBindings materialised during install-defaults / bootstrap are a one-time platform convenience so the default cell exists out of the box; any other project's bindings are developer-owned.)
- Helm install of the OpenChoreo control plane / planes. Assumes a running control plane.
- Plane management in Git —
DataPlane / ClusterDataPlane / WorkflowPlane / ClusterWorkflowPlane / ObservabilityPlane / ClusterObservabilityPlane are one-time install-side setups with cert management. Out of scope by default; brief note in recipes/author-other-resources.md for users who insist.
- Imperative ops — triggering a
WorkflowRun, kubectl exec, runtime log tail, direct CRD edits against the API server. WorkflowRun does not belong in Git; trigger via the UI, webhook, or occ component workflow run.
- External-system admin — Git provider webhooks, IdP / SSO, Vault / AWS Secrets Manager backend setup. The skill wires only the OpenChoreo-side
SecretReference resources, not the upstream store.
- CD tools other than Flux CD.
Working style
- Git is the source of truth. GitOps-managed resources change only through Git.
occ apply -f and kubectl apply -f are reserved for pre-Flux bootstrap and out-of-Git cluster reads.
- Flux prunes on delete. If a resource was committed and is then removed from Git, the next reconcile deletes it from the cluster. Useful for retiring resources cleanly; dangerous if you commit accidentally.
- Always
git commit -s (DCO). Every change is a feature branch + PR.
occ over kubectl for OpenChoreo CRDs. When reading / writing Project, Component, Workload, ComponentRelease, ReleaseBinding, Resource, ResourceRelease, ResourceReleaseBinding, Environment, DeploymentPipeline, ComponentType, ResourceType, Trait, Workflow, SecretReference, AuthzRole, plane CRDs — use occ <kind> get/list/delete. Reach for kubectl only for non-OpenChoreo resources (Flux, ESO, Argo CRDs, raw K8s).
- Verify, don't assume. Reconciliation is interval-based (
GitRepository: 1m, Kustomization: 5m); read the result back with occ <kind> get after merge.
- Don't open a PR or push without explicit user confirmation. Both are remote-visible.
Stable guardrails
- Sync ordering —
platform-shared/ before namespaces/<ns>/platform/ before namespaces/<ns>/projects/, via Flux dependsOn.
- No plaintext secrets in Git — use
SecretReference resources backed by a ClusterSecretStore.
- Protect
platform-shared/ with CODEOWNERS — cluster-scoped changes affect every namespace. Sample at assets/codeowners-platform-shared.
- Cluster ↔ namespace scope is interchangeable for ComponentType / ResourceType / Trait / Workflow. Convert by swapping the
kind: and adding/removing metadata.namespace:. Update referrers' allowedWorkflows[].kind accordingly; for ResourceTypes, update consumer Resource.spec.type.kind. See authoring.md.
- Vanilla CI workflows aren't GitOps-compatible. The four
ClusterWorkflows shipped by the platform install — dockerfile-builder / paketo-buildpacks-builder / gcp-buildpacks-builder / ballerina-buildpack-builder — write the Workload CR directly to the cluster API, so Flux reverts them. extract-resources.sh defaults refuses to extract them by default; use extract-resources.sh gitops-workflows for the GitOps equivalents (docker-gitops-release etc.). When in doubt about any other Workflow, inspect it (occ clusterworkflow get <name> / occ workflow get <name> -n <ns>) and check whether the pipeline ends with git-commit-push-pr (GitOps-compatible) or generate-workload-cr (not). Full write-up in authoring.md Vanilla CI workflows aren't GitOps-compatible.
Anti-patterns
- Scaffolding without confirming the
occ + kubectl context — silently seeding the wrong cluster's resources into Git.
- Wiring Flux before the user has confirmed the remote URL — Flux will start pulling from somewhere unexpected.
- Pushing or opening a PR before the user has reviewed the commit list.
- Hand-authoring large CRDs (ComponentTypes / Workflows) from memory instead of fetching the shape from upstream or templating from a live
occ <kind> get.
- Carrying vanilla CI workflows into a GitOps repo — they write directly to the cluster and conflict with Flux.
- Inventing tooling the user didn't ask for (kustomize overlays, custom controllers, helper scripts).
- Treating cluster reads as authoritative after GitOps is wired — once Flux is reconciling, Git is the source of truth.
1---2name: openchoreo-platform-engineer-gitops3description: Platform-engineer GitOps work for OpenChoreo — scaffolding a GitOps repo (pristine, platform-only, or active cluster), wiring Flux CD, and authoring platform CRDs (ProjectTypes, ComponentTypes, ResourceTypes, Traits, Workflows, Environments, DeploymentPipelines, SecretReferences, AuthzRoles, alert rules, notification channels) via Git. Use when the user says 'set up GitOps for this cluster', 'move this cluster to GitOps', 'wire Flux', 'add a ProjectType / ComponentType / ResourceType / Trait / Workflow via Git', or operates a platform change inside a scaffolded GitOps repo.4---56# OpenChoreo Platform-Engineer GitOps Guide78Git is the source of truth; the cluster is its reflection. This skill writes OpenChoreo **platform** resources to Git, lets Flux CD reconcile them, and reads cluster state with `occ` to verify.910This skill is scoped to platform-engineer GitOps work — initial scaffold + ongoing platform authoring. Application-level GitOps (Project / Component / Workload / ComponentRelease / ReleaseBinding) and non-GitOps cluster operations (Helm install, direct CRD edits) are out of scope; tell the user when a task crosses that boundary.1112## Step 0 — Prerequisites1314`occ` must be configured against the target cluster:1516```bash17command -v occ && occ config context list # occ configured? which control plane?18occ namespace list # cluster reachable + logged in?19```2021**If `occ` points at the wrong control plane, or `occ login` / `occ namespace list` fails**, configure it for the target before proceeding. Details: [CLI Configuration](https://openchoreo.dev/docs/platform-engineer-guide/cli-configuration) and [CLI Installation → Login](https://openchoreo.dev/docs/getting-started/cli-installation).2223- **Repoint the shipped `default`** — every install ships a `default` control plane + context; if you only need occ on this one cluster, just update its URL and log in:24 ```bash25 occ config controlplane update default --url https://api.<cp-domain>26 occ login27 ```28- **Add a second control plane alongside an existing one** (e.g. keep a local k3d entry and add a remote cluster, switching between them with `occ config context use`) — `context add` requires **both** `--controlplane` and `--credentials`, each created first:29 ```bash30 occ config controlplane add <cp> --url https://api.<cp-domain>31 occ config credentials add <cp>32 occ config context add <cp> --controlplane <cp> --credentials <cp> --namespace <ns>33 occ config context use <cp> && occ login34 ```35- **Self-signed certs** (`x509: certificate is not trusted` on login) — trust the CA in your OS trust store (extract, then import per OS — see the CLI Installation doc):36 ```bash37 kubectl get secret openchoreo-ca-secret -n cert-manager \38 -o jsonpath='{.data.ca\.crt}' | base64 -d > openchoreo-ca.crt39 # macOS: sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain openchoreo-ca.crt40 ```4142`occ login` is interactive (browser) — hand it to the user if you can't complete it.4344## Step 1 — Load concepts (MANDATORY)4546**Read [`references/concepts.md`](./references/concepts.md) in full before anything else.** Not optional, even if the task looks simple — resource model / sync ordering / immutability / verification ladder / drift recovery are all easy to get wrong from memory. Load once per session; if you catch yourself acting without it, stop and load now.4748Load other references **on-demand** as the task needs them:4950- [`references/authoring.md`](./references/authoring.md) — where CRD shapes come from (live cluster / `scripts/extract-resources.sh` for defaults + GitOps workflows / `scripts/fetch-page.sh` for docs / ecosystem catalogs for community extensions), repo path conventions, the cluster↔namespace scope swap, the CI-vs-GitOps workflow gotcha, git workflow, DCO.51- [`scripts/fetch-page.sh`](./scripts/fetch-page.sh) — fetch any OpenChoreo docs page by title (resolves against `llms.txt`, picks a stable version). Use this for full CRD schemas with optional fields; `--section "API Reference"` scopes matching to CRD-reference pages, `--list` dumps the index.52- [`scripts/extract-resources.sh`](./scripts/extract-resources.sh) — extract default platform resources from `samples/getting-started/all.yaml` (`defaults --kind X [--name Y]`), or GitOps build-and-release workflows discovered via the ecosystem catalog (`gitops-workflows --list` / `--name <slug>`). Refuses vanilla CI workflows by default.53- [`references/cel.md`](./references/cel.md) — only when writing or reviewing CEL inside ComponentType / Trait / Workflow templates.5455## Detect the mode5657```bash58{ ls flux 2>/dev/null || ls clusters 2>/dev/null; } \59 && { ls platform-shared 2>/dev/null || ls namespaces 2>/dev/null; }60```6162- **Scaffolding mode** — cwd is empty or not yet a GitOps repo. Go to [`recipes/scaffold.md`](./references/recipes/scaffold.md). Covers pristine / platform-only / active-cluster paths.63- **Operating mode** — cwd has a Flux entrypoint (`flux/` for single-cluster, `clusters/<name>/` for multi-cluster) plus at least one of `platform-shared/` or `namespaces/`. Skip to the relevant author / install recipe.6465If the heuristic doesn't fit the layout (per the docs' *Flexible Repository Structures* — repo-per-project, separate-releasebindings-repo, etc.), ask the user.6667## What this skill can do6869- **Scaffold a GitOps repo + wire Flux** — pristine, platform-only, or active cluster → [`recipes/scaffold.md`](./references/recipes/scaffold.md)70- **Install the default platform resources** — project / environments / pipeline / ComponentTypes / Traits + GitOps-mode build-and-release Workflows + Argo `ClusterWorkflowTemplate`s → [`recipes/install-defaults.md`](./references/recipes/install-defaults.md)71- **Install Flux + `git-token` / `gitops-token` secrets** when scaffolding finds them missing → [`recipes/install-flux-and-secrets.md`](./references/recipes/install-flux-and-secrets.md)72- **Author platform resources via Git** — pick the right recipe up-front:73 - `(Cluster)ProjectType` → [`recipes/author-projecttype.md`](./references/recipes/author-projecttype.md)74 - `(Cluster)ComponentType` → [`recipes/author-componenttype.md`](./references/recipes/author-componenttype.md)75 - `(Cluster)ResourceType` → [`recipes/author-resourcetype.md`](./references/recipes/author-resourcetype.md)76 - `(Cluster)Trait` → [`recipes/author-trait.md`](./references/recipes/author-trait.md)77 - `(Cluster)Workflow` → [`recipes/author-workflow.md`](./references/recipes/author-workflow.md)78 - `Environment` + `DeploymentPipeline` (+ default `Project`) → [`recipes/author-environment-pipeline.md`](./references/recipes/author-environment-pipeline.md)79 - `SecretReference` → [`recipes/author-secret-reference.md`](./references/recipes/author-secret-reference.md)80 - `AuthzRole` / `ClusterAuthzRole` + bindings → [`recipes/author-authz.md`](./references/recipes/author-authz.md)81 - `ObservabilityAlertRule`, `NotificationChannel`, anything else → [`recipes/author-other-resources.md`](./references/recipes/author-other-resources.md)82- **Verify reconciliation; recover from drift** → [`recipes/verify-and-recover-drift.md`](./references/recipes/verify-and-recover-drift.md)8384## What this skill cannot do8586- **Application-level GitOps** — `Project` / `ProjectReleaseBinding` / `Component` / `Workload` / `ComponentRelease` / `ReleaseBinding` / `Resource` / `ResourceReleaseBinding` / workload-descriptor authoring. Out of scope; tell the user when the task crosses into application territory. (PE-side: `(Cluster)ProjectType` and `(Cluster)ResourceType` *templates* belong here; the dev-side `Project` instances + their `ProjectReleaseBinding`s / `ResourceReleaseBinding`s belong in the developer-gitops skill. **One exception:** the *default* project's per-env `ProjectReleaseBinding`s materialised during `install-defaults` / bootstrap are a one-time platform convenience so the default cell exists out of the box; any other project's bindings are developer-owned.)87- **Helm install of the OpenChoreo control plane / planes.** Assumes a running control plane.88- **Plane management in Git** — `DataPlane` / `ClusterDataPlane` / `WorkflowPlane` / `ClusterWorkflowPlane` / `ObservabilityPlane` / `ClusterObservabilityPlane` are one-time install-side setups with cert management. Out of scope by default; brief note in [`recipes/author-other-resources.md`](./references/recipes/author-other-resources.md) for users who insist.89- **Imperative ops** — triggering a `WorkflowRun`, `kubectl exec`, runtime log tail, direct CRD edits against the API server. `WorkflowRun` does **not** belong in Git; trigger via the UI, webhook, or `occ component workflow run`.90- **External-system admin** — Git provider webhooks, IdP / SSO, Vault / AWS Secrets Manager backend setup. The skill wires only the OpenChoreo-side `SecretReference` resources, not the upstream store.91- **CD tools other than Flux CD.**9293## Working style9495- **Git is the source of truth.** GitOps-managed resources change only through Git. `occ apply -f` and `kubectl apply -f` are reserved for pre-Flux bootstrap and out-of-Git cluster reads.96- **Flux prunes on delete.** If a resource was committed and is then removed from Git, the next reconcile deletes it from the cluster. Useful for retiring resources cleanly; dangerous if you commit accidentally.97- **Always `git commit -s`** (DCO). Every change is a feature branch + PR.98- **`occ` over `kubectl` for OpenChoreo CRDs.** When reading / writing Project, Component, Workload, ComponentRelease, ReleaseBinding, Resource, ResourceRelease, ResourceReleaseBinding, Environment, DeploymentPipeline, ComponentType, ResourceType, Trait, Workflow, SecretReference, AuthzRole, plane CRDs — use `occ <kind> get/list/delete`. Reach for `kubectl` only for non-OpenChoreo resources (Flux, ESO, Argo CRDs, raw K8s).99- **Verify, don't assume.** Reconciliation is interval-based (`GitRepository: 1m`, `Kustomization: 5m`); read the result back with `occ <kind> get` after merge.100- **Don't open a PR or push without explicit user confirmation.** Both are remote-visible.101102## Stable guardrails103104- **Sync ordering** — `platform-shared/` before `namespaces/<ns>/platform/` before `namespaces/<ns>/projects/`, via Flux `dependsOn`.105- **No plaintext secrets in Git** — use `SecretReference` resources backed by a `ClusterSecretStore`.106- **Protect `platform-shared/` with CODEOWNERS** — cluster-scoped changes affect every namespace. Sample at [`assets/codeowners-platform-shared`](./assets/codeowners-platform-shared).107- **Cluster ↔ namespace scope is interchangeable** for ComponentType / ResourceType / Trait / Workflow. Convert by swapping the `kind:` and adding/removing `metadata.namespace:`. Update referrers' `allowedWorkflows[].kind` accordingly; for ResourceTypes, update consumer `Resource.spec.type.kind`. See [`authoring.md`](./references/authoring.md).108- **Vanilla CI workflows aren't GitOps-compatible.** The four `ClusterWorkflow`s shipped by the platform install — `dockerfile-builder` / `paketo-buildpacks-builder` / `gcp-buildpacks-builder` / `ballerina-buildpack-builder` — write the `Workload` CR directly to the cluster API, so Flux reverts them. `extract-resources.sh defaults` refuses to extract them by default; use `extract-resources.sh gitops-workflows` for the GitOps equivalents (`docker-gitops-release` etc.). When in doubt about any other Workflow, inspect it (`occ clusterworkflow get <name>` / `occ workflow get <name> -n <ns>`) and check whether the pipeline ends with `git-commit-push-pr` (GitOps-compatible) or `generate-workload-cr` (not). Full write-up in [`authoring.md`](./references/authoring.md) *Vanilla CI workflows aren't GitOps-compatible*.109110## Anti-patterns111112- Scaffolding without confirming the `occ` + `kubectl` context — silently seeding the wrong cluster's resources into Git.113- Wiring Flux before the user has confirmed the remote URL — Flux will start pulling from somewhere unexpected.114- Pushing or opening a PR before the user has reviewed the commit list.115- Hand-authoring large CRDs (ComponentTypes / Workflows) from memory instead of fetching the shape from upstream or templating from a live `occ <kind> get`.116- Carrying vanilla CI workflows into a GitOps repo — they write directly to the cluster and conflict with Flux.117- Inventing tooling the user didn't ask for (kustomize overlays, custom controllers, helper scripts).118- Treating cluster reads as authoritative *after* GitOps is wired — once Flux is reconciling, Git is the source of truth.