Run Uat

Run UAT

Purpose

Execute end-to-end User Acceptance Testing by posting a feature-specific artifact and the comprehensive demo to a real PR on GitHub or Azure DevOps, then validating the rendered output.

Hard Rules

Must

• Provide at least one --report/--instructions pair (feature-specific report and test instructions).
• The comprehensive demo is appended automatically — do NOT add it as a --report.
• Artifacts must be up-to-date (generated from current branch code). Run scripts/generate-demo-artifacts.sh if needed.
• Leave the UAT PRs open when you finish creating them. Completing the UAT Tester's stage is what opens the Maintainer's approval gate, and the gate asks them to look at those PRs — closing them first makes the gate unanswerable.
• Cleanup happens after the decision is recorded, and belongs to the Release Manager, not the UAT Tester.
• Report platform-specific rendering issues clearly.

Must Not

• Post a minimal or simulation artifact (reject files containing "minimal" or "simulation" in the name).
• Pass the comprehensive demo as a --report argument (added automatically).
• Leave UAT PRs open after testing completes.
• Modify any source code during UAT.

Pre-requisites

• Git submodules initialized: git submodule update --init --recursive
• GitHub UAT: Authentication is automatic for coding agents (uses GH_UAT_TOKEN). For local dev: gh auth login.
• Azure DevOps UAT: Authentication is automatic for coding agents (uses AZDO_UAT_TOKEN → AZURE_DEVOPS_EXT_PAT). For local dev: set AZURE_DEVOPS_EXT_PAT.

Recommended: Single Wrapper Command

# Minimum: one feature-specific report with test instructions
scripts/uat-run.sh \
  --report artifacts/<feature-specific-report>.md \
  --instructions "In azurerm_key_vault_secret.audit_policy, verify key_vault_id displays as 'Key Vault \`kv-name\` in resource group \`rg-name\`' instead of full /subscriptions/ path"

# Multiple feature-specific reports (all with instructions):
scripts/uat-run.sh \
  --report artifacts/feature-a.md \
  --instructions "Verify feature A renders correctly" \
  --report artifacts/feature-b.md \
  --instructions "Verify feature B renders correctly"

# GitHub only:
scripts/uat-run.sh \
  --report artifacts/<feature>.md \
  --instructions "..." \
  --platform github

# Create PRs without polling (for manual review workflow):
scripts/uat-run.sh \
  --report artifacts/<feature>.md \
  --instructions "..." \
  --create-only
# After the Maintainer records the gate decision, the Release Manager cleans up:
scripts/uat-run.sh --cleanup-last

What the Script Does

1. Validates that all provided artifacts are up-to-date (built from current branch code)

2. Auto-configures GitHub/AzDO authentication for coding agent environments

3. Creates a unique UAT branch in the UAT repos

4. Creates UAT PR(s) on GitHub and/or Azure DevOps

5. Posts each feature report as a PR comment with the format:

   ## Test Instructions
   <instructions you provided>
   
   ## Report
   <artifact content>
   

6. Automatically appends artifacts/comprehensive-demo-simple-diff.md (GitHub) and artifacts/comprehensive-demo.md (AzDO) as the final regression test comment

7. Polls for approval (unless --create-only)

8. Cleans up PRs and branches on approval

0. Rebase on Latest Main

Before running UAT, ensure your branch is up to date to avoid testing against stale base changes. Use the git-rebase-main skill.

1. Generate Fresh Artifacts (if needed)

# If script fails with "Artifact is outdated":
scripts/generate-demo-artifacts.sh

# For feature-specific artifacts:
dotnet run --project src/Oocx.TfPlan2Md/Oocx.TfPlan2Md.csproj -- \
  [your args] --output artifacts/<feature-specific>.md

Troubleshooting