Generate Sh Checker

Generate.sh Checker

This skill helps maintain the generate.sh pattern across all apis/ subdirectories in the Config Connector codebase.

Workflow

1. Scan for missing scripts: Find service directories in apis/ that do not have a generate.sh file but contain KRM types files.

   find apis -maxdepth 1 -type d ! -name "apis" ! -name "refs" | while read dir; do if [ ! -f "$dir/generate.sh" ] && find "$dir" -name "*_types.go" | grep -q .; then echo "$dir"; fi; done
   

   (Note: apis/refs is a special folder and does not correspond to a GCP service. Since it lacks *_types.go files, the above command naturally skips it, which is correct.)

2. Gather Resource Information: For each identified directory, read api_types.go and groupversion_info.go to extract:

   • PROTO_SERVICE: Look for // +kcc:spec:proto= or // +kcc:proto= markers in api_types.go.
   • GROUP: Look for // +groupName= in groupversion_info.go.
   • VERSION: The directory name (e.g., v1beta1).
   • RESOURCE_MAPPINGS: Mapping of Kind:ProtoMessage from // +kcc:spec:proto= markers.
   • SERVICE_NAME: The parent directory name in apis/ (e.g., apigateway).

   Note: If the directory does not contain any *_type*.go file (e.g., it only contains reference types like service_reference.go), there are no types or mappers to generate. In this case, generate.sh is not required.

3. Create generate.sh: Create a generate.sh file in the directory. Ensure the year in the copyright header is current (2026).

   Template:

   #!/bin/bash
   # Copyright 2026 Google LLC
   #
   # Licensed under the Apache License, Version 2.0 (the "License");
   # you may not use this file except in compliance with the License.
   # You may obtain a copy of the License at
   #
   #      http://www.apache.org/licenses/LICENSE-2.0
   #
   # Unless required by applicable law or agreed to in writing, software
   # distributed under the License is distributed on an "AS IS" BASIS,
   # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   # See the License for the specific language governing permissions and
   # limitations under the License.
   
   set -o errexit
   set -o nounset
   set -o pipefail
   
   REPO_ROOT="$(git rev-parse --show-toplevel)"
   CONTROLLERBUILDER="${CONTROLLERBUILDER:-}"
   if [[ -z "${CONTROLLERBUILDER}" ]]; then
     if [[ -x "${REPO_ROOT}/bin/controllerbuilder" ]]; then
       CONTROLLERBUILDER="${REPO_ROOT}/bin/controllerbuilder"
     else
       CONTROLLERBUILDER="go run ${REPO_ROOT}/dev/tools/controllerbuilder"
     fi
   fi
   source "${REPO_ROOT}/dev/tools/goimports.sh"
   cd ${REPO_ROOT}/dev/tools/controllerbuilder
   
   ./generate-proto.sh
   
   # --- v1alpha1 (if applicable) ---
   ${CONTROLLERBUILDER} generate-types \
     --service <PROTO_SERVICE> \
     --api-version <GROUP>/v1alpha1 \
     --include-skipped-output \
     --resource <KIND1>:<PROTO_MESSAGE1>
   
   # --- v1beta1 (if applicable) ---
   ${CONTROLLERBUILDER} generate-types \
     --service <PROTO_SERVICE> \
     --api-version <GROUP>/v1beta1 \
     --include-skipped-output \
     --resource <KIND2>:<PROTO_MESSAGE2>
   
   # --- mappers (run exactly once at the end with --multiversion) ---
   ${CONTROLLERBUILDER} generate-mapper \
     --service <PROTO_SERVICE> \
     --api-version <GROUP>/<LATEST_VERSION> \
     --include-skipped-output \
     --multiversion
   
   cd ${REPO_ROOT}
   dev/tasks/generate-crds
   
   go run -mod=readonly golang.org/x/tools/cmd/goimports@${GOLANG_X_TOOLS_VERSION} -w  pkg/controller/direct/<SERVICE_NAME>/
   

4. Special Handling (Multi-version & Promotion/Consolidation):

   • File Naming: generate-types expects the main types file to be named <lowercase_proto_message_name>_types.go. If the existing file has a different name (e.g., cluster_types.go instead of attachedcluster_types.go), rename it before running the generator.
   • Hand-written types.generated.go: If a types.generated.go already exists but lacks the // Code generated by ... DO NOT EDIT. header, it was hand-written. Rename it to types.go to prevent it from being overwritten.
   • Pointer Types: When preserving hand-written structs that correspond to proto messages, ensure their fields use pointers (e.g., *string with ,omitempty instead of string) where the proto fields are optional. Otherwise, mapper.generated.go will fail to compile with type assignment errors (e.g., cannot use direct.LazyPtr(in.GetName()) ... as string value in assignment).
   • Multi-version resources: When promoting or maintaining a resource in multiple versions (e.g. v1alpha1 and v1beta1), we should use // +kubebuilder:metadata:labels="internal.cloud.google.com/additional-versions=v1alpha1" on the v1beta1 resource struct to generate v1alpha1 from v1beta1.
   • Consolidated generate.sh: All version generation tasks must be consolidated into a single service-level script (apis/<service>/generate.sh). We run generate-types for each version in sequence.
   • Single generate-mapper with --multiversion: Only call generate-mapper once at the very end of the script (targeting the highest/most stable version) and always pass the --multiversion flag by default (even if the service currently has only one version). This prevents duplicate function clashing and keeps the codebase extensible as new versions are introduced. When using --multiversion, check existing handwritten controller code to ensure they call the updated version-suffixed mapper functions (e.g. KMSImportJobSpec_v1beta1_FromProto). If there are custom manual mapper functions in *_mappings.go or *_mappers.go, rename them to match the new version suffix (e.g., _v1beta1_FromProto or _v1alpha1_FromProto) so generate-mapper recognizes them and skips generating duplicates.
   • Different Proto Packages/Versions: If a resource (e.g. an alpha/beta resource like ComputeFutureReservation) belongs to a different proto package/version (e.g. google.cloud.compute.v1beta) than the default --service package (e.g. google.cloud.compute.v1), you can specify its fully-qualified name in the --resource flag (e.g. --resource ComputeFutureReservation:google.cloud.compute.v1beta.FutureReservation). This allows generate-types to locate the correct message descriptors across different packages when compiling multiple API levels.

5. Execute and Verify:

   • Make generate.sh executable: chmod +x apis/<SERVICE>/generate.sh.
   • Run it: ./apis/<SERVICE>/generate.sh.
   • Verify that types.generated.go is created in the API directory.
   • Verify that pkg/controller/direct/<SERVICE>/mapper.generated.go is updated.
   • Verify that CRDs in config/crds/resources/ are updated.
   • Ref fields with acronyms: If you encounter // MISSING: [Acronym]... (like MISSING: KMSKey or MISSING: CAPool) in the generated mapper.generated.go, it might be because the generate-mapper tool expects the field in the KRM struct to use the fully capitalized acronym (e.g. KMSKeyRef instead of KmsKeyRef, CAPoolRef instead of CaPoolRef). Rename the Go struct field to match the acronym (this won't break the yaml if the json tag is unchanged), and update any references in mapper.go or [service]_controller.go. The generator should then automatically map the Ref field properly.

6. Commit and PR: Create a branch, commit the changes, and propose a PR with a descriptive title like chore: apis/<SERVICE> should follow generate.sh pattern.

Troubleshooting

See notes.md for troubleshooting uncommon edge cases.

• Deepcopy-gen errors (invalid slice element type: invalid type): This typically happens if generate-types outputs a struct name with a different capitalization than what is currently manually written in the *_types.go file (e.g. PSCConfig vs PscConfig). To fix, rename the type and all its usages in the *_types.go and pkg/controller/direct/<SERVICE>/mapper.go files to match the generated capitalization, then run ./generate.sh again.