Datadog Rules Go Orchestrion Upstream Migration

Datadog rules_go Orchestrion upstream migration

Use this skill when you need to move the vendored Orchestrion-enabled rules_go fork in this repository from the currently pinned upstream rules_go base to another upstream tag or commit.

This skill is intentionally repository-local: it is stored as a Codex-compatible skill, but any agent can read it as a normal implementation guide.

The manifest-driven Go/Python sync API does not change the fork migration boundary. A migration must preserve disabled empty-repository behavior, Bazel-managed Go SDK bootstrap, and the existing public Orchestrion aliases; it must not add target discovery, service naming, or manifest policy to rules_go.

Non-negotiable contract

Preserve the maintained fork contract:

• Migrate the target upstream's base tree first.
• Treat base as the clean, generic Orchestrion integration.
• Maintain both the patch series under third_party/rules_go_orchestrion/patches/<upstream>/ and the materialized trees selected by third_party/rules_go_orchestrion/registry.json.
• Maintain public consumer patch profiles under third_party/rules_go_orchestrion/profiles/.
• Do not reintroduce a public complete variant or public consumer-specific patch bundles.
• Do not edit generated *.CHANGED_FILES.md reports by hand.
• Do not update *.METADATA.json to a new upstream tag or commit until the vendored tree actually reflects that upstream base plus the Datadog delta.
• Do not call the migration complete until materialization checks, profile verification, changed-file regeneration, and relevant smoke or integration validation have run.
• Preserve deterministic action outputs for GoStdlib, GoSyntheticTestmainHelpers, synthetic GoCompilePkg (~testmain.a), and GoLink. An ordinary GoStdlib action must not publish or consume the instrumented stdlib cache. A Test Optimization build must publish stable woven archives across two isolated output roots, including CGO mode.

First actions

1. Confirm the requested target upstream:
   • exact upstream rules_go tag or commit
   • whether the work is exploratory or intended for a pull request
   • whether the new upstream must be included in public consumer patch profile generation immediately
2. Inspect the current repository state before editing:
   • git status --short
   • third_party/rules_go_orchestrion/registry.json
   • the target upstream's *.METADATA.json
   • the target upstream's *.CHANGED_FILES.md
   • third_party/rules_go_orchestrion/profiles/workspace_runtime.json
3. Read the maintainer docs that define the current fork model:
   • docs/rules_go_orchestrion_support_selection.md
   • docs/rules_go_variant_maintenance_guide.md
   • docs/go_orchestrion_maintainer_state.md
   • docs/go_orchestrion_bazel_deep_dive.md
   • CONTRIBUTING.md
4. Inspect the current patch/tree relationship before changing the fork:
   • python3 tools/dev/generate_rules_go_fork_maps.py --check
   • python3 tools/dev/materialize_rules_go_fork.py check --all
   • python3 tools/dev/verify_rules_go_profiles.py --public-denylist tools/dev/private_leak_public_denylist.txt
5. Install zstd before running the local generated-profile verifier; it reads Bazel's compact execution logs.

Implementation path

• Migration workflow: follow migration-workflow.md.
• Validation: follow validation-checklist.md.
• Troubleshooting: follow troubleshooting.md.

Sensitive surfaces

Expect conflicts or semantic drift around these paths first:

• go/orchestrion_workspace.bzl
• go/private/orchestrion/*
• go/private/context.bzl
• go/private/actions/compilepkg.bzl
• go/private/actions/stdlib.bzl
• go/private/actions/link.bzl
• go/private/actions/archive.bzl
• go/private/rules/library.bzl
• go/private/rules/test.bzl
• go/private/rules/stdlib.bzl
• go/private/rules/transition.bzl
• go/tools/builders/builder.go
• go/tools/builders/compilepkg.go
• go/tools/builders/env.go
• go/tools/builders/env_orchestrion.go
• go/tools/builders/importcfg.go
• go/tools/builders/link.go
• go/tools/builders/module_proxy.go
• go/tools/builders/orchestrion*.go
• go/tools/builders/stdlib.go
• go/tools/builders/tool_version.go

Pay special attention to:

• Orchestrion toolexec wiring
• synthetic testmain handling
• stdlib weaving and stdlib archive/cache persistence
• offline module proxy inputs
• dd_trace_go_versions.json
• tool-version validation
• Bzlmod and WORKSPACE extension entrypoints
• Bazel transitions that must preserve Orchestrion settings

Done criteria

A migration is done only when all of these are true:

• base points at the requested upstream base plus the Datadog Orchestrion integration.
• The target upstream's *.METADATA.json file names the upstream repository, commit, and tag when a tag exists.
• The target upstream's *.CHANGED_FILES.md report was regenerated by tools/dev/diff_rules_go_fork.py.
• python3 tools/dev/materialize_rules_go_fork.py check --all passes.
• python3 tools/dev/verify_rules_go_profiles.py --public-denylist tools/dev/private_leak_public_denylist.txt passes.
• Profile verification executes two isolated builds for both ordinary and Test Optimization modes and compares the four required action families. The ordinary declared stdlib cache is empty; the instrumented CGO stdlib contains Test Optimization markers and produces identical bytes.
• The selected validation lanes in validation-checklist.md pass, or every skipped lane has a concrete reason.
• Any fixture or consumer validation that was required for the changed surface has passed.
• The final report names the target upstream, changed-path counts, validation results, and any remaining external blockers.

Test Optimization alias contract

When validating a consumer that uses Test Optimization, preserve the stable alias contract from the vendored base tree:

common:test-optimization --repo_env=DD_TEST_OPTIMIZATION_ENABLED=1

--config=test-optimization is the only user-facing metadata switch. Optimized targets enable Orchestrion through their transition. Omitting the config must leave metadata bootstrap disabled for public Go extension repositories and for low-level repositories explicitly configured with enabled_by_env = True.

Stop conditions

Stop and escalate instead of guessing when:

• The requested target upstream tag or commit is ambiguous.
• Upstream rules_go changed a build pipeline surface enough that the Orchestrion behavior cannot be mapped without a design decision.
• base needs behavior that appears specific to one Datadog monorepo consumer.
• The patch series applies but the materialized tree check differs from the checked-in tree and the difference is not intentionally regenerated.
• Smoke validation passes but runtime validation shows no CI Visibility startup or no payload files.
• The only available fix requires a public consumer-specific patch bundle.
• Required validation needs credentials or environment state that is not available locally.