Add List Resource Workflow

add-list-resource-workflow

Note to AI Agents: You MUST read the YAML frontmatter above first. Only read the rest of this file if the description matches your current roadblock or required task.

This workflow produces a change scoped to one product that flips generate_list_resource: true on every eligible MMv1 resource in that product, generates the downstream code, and runs the generated list-query tests. Do one product per change, with as many eligible resources as pass.

Consult .agents/knowledge/index.md for the topics this task touches and open the relevant sources.

Prerequisites

• You are in the magic-modules root directory.
• $GOPATH is set and terraform-provider-google is checked out at $GOPATH/src/github.com/hashicorp/terraform-provider-google (or another known path — confirm with the user).
• Remote repositories are configured (e.g. upstream pointing to GoogleCloudPlatform/magic-modules and personal fork remote such as origin).
• The user has named one target product (e.g. compute). Resources will be selected automatically by the eligibility scan.

Eligibility Check

A resource is eligible when the generated list-query test can run unattended. In practice that means:

1. The resource must not be excluded from identity or read generation (exclude_identity_generation: true or exclude_read: true) — the generator hard-fails on these.
2. The first example must not have exclude_test: true, since the generated query test reuses its config.
3. Every scope parameter in the list URL (path params other than project/region/zone/location — e.g. disk, instance, parent) needs special handling, because the query test template currently only auto-injects project/region/zone into the test context map. Resources whose list URL contains other scope params are not eligible until the template is extended.

Required body fields (set at create time) do not affect list eligibility. Only path scope params on the list/collection URL matter.

Run the eligibility scan across the whole product and produce a candidate list. Report it back to the user before editing any YAML.

PRODUCT=<product>   # e.g. compute

python3 - "$PRODUCT" <<'PY'
import sys, glob, yaml, os, re
product = sys.argv[1]
AUTO_SCOPES = {"project", "region", "zone", "location"}
candidates, skipped = [], []
for f in sorted(glob.glob(f"mmv1/products/{product}/*.yaml")):
    if f.endswith("/product.yaml"):
        continue
    try:
        d = yaml.safe_load(open(f).read())
    except Exception as e:
        skipped.append((f, f"yaml parse error: {e}")); continue
    if not isinstance(d, dict):
        continue
    name = d.get("name") or os.path.basename(f)
    if d.get("exclude") or d.get("exclude_resource"):
        skipped.append((name, "excluded resource")); continue
    if d.get("exclude_identity_generation") or d.get("exclude_read"):
        skipped.append((name, "exclude_identity_generation or exclude_read")); continue
    if d.get("generate_list_resource"):
        skipped.append((name, "already opted in")); continue
    ex = d.get("examples") or d.get("samples") or []
    if not ex or not isinstance(ex[0], dict):
        skipped.append((name, "no examples/samples")); continue
    first = ex[0]
    if first.get("exclude_test"):
        skipped.append((name, "first example has exclude_test")); continue
    # Inspect list URL scope params. Prefer collection_url_key construction over base_url,
    # but base_url is a reliable proxy for the list collection URL.
    list_url = d.get("base_url") or ""
    scope_params = re.findall(r"{{\s*(\w+)\s*}}", list_url)
    bad_scope = [s for s in scope_params if s not in AUTO_SCOPES]
    if bad_scope:
        skipped.append((name, f"list URL has unsupported scope param(s): {bad_scope}")); continue
    candidates.append((name, f))

print("CANDIDATES:")
for n, f in candidates:
    print(f"  - {n}  ({f})")
print("\nSKIPPED:")
for n, r in skipped:
    print(f"  - {n}: {r}")
PY

Stop and present the candidate list to the user before making any edits. Do not attempt to remove required: true from properties or to remove exclude_identity_generation to force eligibility.

Execution Steps

1. Sync and branch

git fetch upstream main
BRANCH="add-${PRODUCT}-list-resources"   # e.g. add-compute-list-resources
git checkout -b "$BRANCH" upstream/main

If the working tree is dirty, stash before checkout and warn the user.

2. Edit each eligible resource's YAML

For every resource the user approved from the eligibility scan, insert generate_list_resource: true as a top-level key in its YAML. Place it adjacent to other top-level booleans such as immutable: or has_self_link: for readability. Do not touch any other fields.

# Example placement (manual edit, repeated per resource)
# ...
# has_self_link: true
# immutable: true
# generate_list_resource: true
# timeouts:
# ...

3. Generate the downstream provider

PROVIDER_PATH="$GOPATH/src/github.com/hashicorp/terraform-provider-google"

# Stop if downstream has uncommitted work
( cd "$PROVIDER_PATH" && git status --porcelain ) && echo "Confirm clean before continuing"

make provider VERSION=ga OUTPUT_PATH="$PROVIDER_PATH"

Expected new files in the downstream per opted-in resource:

• google/services/<product>/list_<resource>.go
• google/services/<product>/list_<resource>_generated_test.go
• website/docs/list-resources/<terraform_name>.html.markdown

4. Build and test

cd "$PROVIDER_PATH"
go build ./...

# Run every generated list-query test for the product in one go.
# Test name format: TestAcc<ResourceName>ListQuery_generated
TF_ACC=1 go test -v -timeout 120m \
  ./google/services/<product> \
  -run 'ListQuery_generated$' | tee /tmp/list_query_test.out

Required environment for a live run: GOOGLE_PROJECT, GOOGLE_REGION, GOOGLE_ZONE, GOOGLE_CREDENTIALS (or ADC). Confirm with the user before consuming GCP resources.

If any test fails, do not patch the generator or the YAML to suppress the failure. Report the failing resources to the user. The user decides whether to (a) drop those resources and re-generate, or (b) abort entirely. Never silently ship a change that has failing list-query tests.

Handoff & Guardrails

• One product per change. Bundle every eligible resource in the product into a single change. Do not split a product across multiple changes unless the user explicitly asks.
• Never edit the generator (mmv1/api/, mmv1/provider/, mmv1/templates/terraform/list_resource*) from this workflow. If the generator misbehaves, stop and escalate to the user.
• Never commit downstream provider files to the magic-modules branch.
• On any failure during generate/build, abort and report the exact failing command and output. On test failures, drop the failing resource(s) (or abort) per the user's choice — never ship a change with failing list-query tests.