Bump Serverless Env Version

Bump the serverless environment version

Read .agents/rules/template-versions.md first for compatibility and synchronization policy. Never hand-edit generated acceptance output.

1. Resolve a compatible version set

Use the requested environment version, or the newest published version when none was specified. Confirm its runtime Python version from the environment release notes and databricks/environments.

Apply .agents/rules/template-versions.md to the environment, Python, and DB Connect pins. Check Python compatibility in the DB Connect requirements. Advance conservative_db_connect_version_spec when compatibility requires it or its DBR line has six months or less of support remaining. Use the supported-LTS table in the Databricks Runtime release notes to choose the oldest compatible replacement with more than six months of support remaining. Do not advance it merely to match the environment version.

2. Update every template source

Find both macro definitions and hardcoded literals; trust the search results:

grep -rn 'environment_version\|environment-version\|requires-python\|default_python_version' libs/template/templates/

Inspect every hit and change only values that pin a version. Re-run the command after editing and confirm every version-bearing template hit uses the target. If all sources and coupled pins already match the resolved set and the worktree has no partial bump, report that no changes are needed and stop.

Current sources include:

• serverless_environment_version definitions in default/library/versions.tmpl and dbt-sql/library/versions.tmpl;
• a hardcoded value in the default-scala job template;
• three hardcoded values in lakeflow-integrations, including its --environment-version argument.

In default/library/versions.tmpl, update python_version_spec and default_python_version when the runtime Python version changes. Update conservative_db_connect_version_spec only under the compatibility and support rule above. Keep the version-specific compatibility comments accurate even when their pin is unchanged. Do not synchronize unrelated DBR or SQL-template DB Connect macros.

Update version-specific examples in .agents/rules/template-versions.md so its policy remains accurate; do not change the policy itself as part of the bump.

Also inspect defaultServerlessVersion in libs/localenv/envkey.go. It is the product-spec fallback for databricks environments setup-local, not a template pin; do not infer that it should move with the templates. Change it only when the user or current product specification also requires the fallback to move, and first confirm python/serverless/serverless-v{N}/pyproject.toml exists in databricks/environments. Then update default-version help, error, and test expectations under libs/localenv and cmd/environments; verify with go test ./libs/localenv ./cmd/environments, then update and verify with:

go test ./acceptance -run '^TestAccept/localenv' -update -timeout=60m
go test ./acceptance -run '^TestAccept/localenv' -timeout=60m

Otherwise leave it unchanged and record why. Do not change the intentionally older SSH fixtures in acceptance/ssh/connect-serverless-*.

3. Regenerate and verify goldens

Update and verify the acceptance suite:

./task test-update
go test ./acceptance -run '^TestAccept$' -timeout=60m

The non-update command must pass. Update mode selects covering EnvMatrix variants; the non-update run verifies every variant against the regenerated goldens.

4. Add the changelog fragment

Add .nextchanges/bundles/serverless-environment-version-v{N}.md. Follow the pr-checklist skill's changelog conventions. Describe a benefit stated in the target version's release notes, link the actual version page, and mention the DB Connect version only if it changed.

Cross-check the final source and generated-output footprint against the prior template bumps in PR #3897 and PR #6378. Explain material differences in the final handoff or PR description. Prior PRs are not sources of truth: revalidate their compatibility decisions, wording, URLs, and file lists.

5. Finish only when requested

Required sub-skill: use pr-checklist, run its checks, and inspect the final diff. If formatting or linting changes generated acceptance files, fix the source and regenerate them.

Commit, push, or create/update a PR only when the user explicitly requests that operation. When requested, follow pr-checklist rather than duplicating its commit and PR-body instructions here.

When creating a PR for the bump, suggest lennartkats-db as a reviewer.