Azure Devops Pr Review

review-pr

Code review an Azure DevOps pull request.

Inputs

* Either provide prUrl (all inputs will be extracted from it), or provide all of prId, organization, project, and repo individually.

If any required input cannot be determined, ask the user before proceeding.

Priority Ordering

PRIORITY 1: Only report issues with confidence ≥ 75 — never surface false positives or stylistic concerns. PRIORITY 2: Always confirm findings with the user before posting inline comments. PRIORITY 3: Use parallel multi-model analysis (gpt-5.4 + claude-sonnet-4.6) and deduplicate overlapping findings. When these conflict, prefer fewer high-confidence findings over comprehensive low-confidence coverage.

Overview

Provide a structured code review for a given Azure DevOps PR, including validation, multi-pass analysis, and inline feedback posting.

Workflow

0. Parse Inputs

If the user provides a full PR URL (e.g. https://dev.azure.com/myorg/My%20Project/_git/my-repo/pullrequest/12345), extract:

• organization — the first path segment after dev.azure.com (e.g. myorg)
• project — the second path segment, URL-decoded (e.g. My Project from My%20Project)
• repo — the path segment after _git/ (e.g. my-repo)
• prId — the path segment after pullrequest/ (e.g. 12345)

Then set up authentication and configure the CLI defaults so every subsequent command uses these values without relying on auto-detection.

PAT setup (required before any az repos call):

If the user supplies a PAT file path, read it and set AZURE_DEVOPS_EXT_PAT. On Windows, prefer the Bash tool (available when Git for Windows is installed) — the auto-mode classifier is more permissive with it and export works reliably. Fall back to PowerShell if Bash is unavailable.

Windows path note: Git Bash uses POSIX paths. Translate C:\temp\PAT.txt → /c/temp/PAT.txt.

Bash (Git Bash on Windows or Linux/macOS):

export AZURE_DEVOPS_EXT_PAT="$(cat /c/temp/PAT.txt | tr -d '[:space:]')"

PowerShell fallback:

$env:AZURE_DEVOPS_EXT_PAT = (Get-Content "C:\temp\PAT.txt" -Raw).Trim()

Run az devops configure and all az/curl commands in the same shell where the env var is set.

CLI defaults — include repository or az repos pr show will return TF401180: not found:

az devops configure --defaults \
  organization="https://dev.azure.com/{organization}" \
  project="{project}" \
  repository="{repo}"

Use --detect false on every az command from this point on. Never rely on --detect true — it fails when the working directory is not a clone of that exact repository.

Important: az repos pr show does not accept --project, --repository, or -p/-r as inline flags. All three must come from az devops configure --defaults set above.

1. Check Eligibility

Run:

az repos pr show --id {prId} --detect false --output json

If az repos pr show fails (non-zero exit, or TF401180 error), fall back to the REST API directly — it is always reliable:

B64=$(printf '%s' ":${AZURE_DEVOPS_EXT_PAT}" | base64 -w 0)
curl -s \
  -H "Authorization: Basic $B64" \
  -H "Content-Type: application/json" \
  "https://dev.azure.com/{organization}/{project}/_apis/git/repositories/{repo}/pullRequests/{prId}?api-version=7.1"

Store the JSON response for use in subsequent steps.

Ensure the PR:

• Is open
• Is not a draft
• Has not already been reviewed by you — check with:

az repos pr reviewer list --id {prId} --detect false

If isRequired: false and the reviewer entry has vote != 0, a review has already been cast. Skip if so.

Skip if any condition fails.

2. Get Context

Instruction Files

Locate relevant guidance files:

• .github/copilot-instructions.md (root-level, if present)
• AGENTS.md
• CLAUDE.md

Search in:

• Modified directories
• Their ancestor directories

Retrieve PR Diff

Extract sourceRefName and targetRefName from the PR JSON fetched in Step 1. Strip the refs/heads/ prefix to get the branch names (e.g. refs/heads/feature/123 → feature/123).

Then fetch both branches and diff them — do not use az repos pr checkout, which modifies the working tree and can fail on dirty repos:

git fetch origin {sourceBranch}
git diff origin/{targetBranch}...origin/{sourceBranch}

Optional: Check Policy Status

az repos pr policy list --id {prId} --detect false --output table

Note any failing required policies in your review summary. Do not block or abort the review due to policy failures — just surface them to the user.

3. Review the Changes

Review Strategy

Spawn specialist subagents for the languages/domains in the diff (e.g. React, security, SQL, infrastructure). For each specialist and general reviewer, provide:

• The full diff output from Step 2
• The content of any instruction files found
• The task: "Review this diff for issues in your domain. Return a list of findings with file path, line number, description, and confidence score 0–100."

Always include two general-purpose passes:

• One pass with gpt-5.4
• One pass with claude-sonnet-4.6

Spawn all agents in parallel, then collect and deduplicate overlapping findings before proceeding.

Review Focus Areas

• Compliance with instruction files
• Functional correctness
• Git history / blame insights
• Violations of code comments or guidance

4. Validate Issues

Assign confidence scores:

Only include issues with confidence ≥ 75

5. Post Review

a. Confirm with User

• Present summary of findings
• Ask for confirmation before posting
• Skip confirmation only if explicitly requested

b. Post Inline Comments (Required)

One issue per thread. Always two separate steps — write files first, then POST.

Never use a bash heredoc assigned to a $BODY variable and passed with curl -d "$BODY". Shell expansion corrupts embedded newlines and quotes, causing 400 - commentThread null errors. Never combine the file write and curl in a single bash command — the auto-mode classifier blocks it.

Step 1 — Write each thread JSON using the Write tool (one file per finding):

{
  "comments": [{
    "parentCommentId": 0,
    "content": "<Issue title>\n\n<Why it matters in this specific code path>\n\nFix: <actionable suggestion>\n\n🤖 Generated with AI",
    "commentType": "text"
  }],
  "status": "active",
  "threadContext": {
    "filePath": "/src/path/to/file.ext",
    "rightFileStart": { "line": 42, "offset": 1 },
    "rightFileEnd": { "line": 42, "offset": 1 }
  }
}

Save files to C:\temp\thread1.json, C:\temp\thread2.json, etc.

Step 2 — POST all threads in one bash loop (separate Bash call, after all files are written):

B64=$(printf '%s' ":${AZURE_DEVOPS_EXT_PAT}" | base64 -w 0)
URL="https://dev.azure.com/{organization}/{project}/_apis/git/repositories/{repo}/pullRequests/{prId}/threads?api-version=7.1"

for i in 1 2 3 4; do
  STATUS=$(curl -s -o /tmp/resp${i}.json -w "%{http_code}" \
    -X POST \
    -H "Authorization: Basic $B64" \
    -H "Content-Type: application/json" \
    --data-binary @/c/temp/thread${i}.json \
    "$URL")
  echo "Thread $i: HTTP $STATUS"
  [ "$STATUS" != "200" ] && cat /tmp/resp${i}.json
done

Always use --data-binary @/path/to/file.json — never -d "$VARIABLE".

Line Number Rules

• Use right-side (new file) line numbers
• From diff: @@ -35,3 +39,7 @@ → use 39
• Verify with:

grep -n '<text>' <file>

Valid Thread Status

• active
• fixed
• wontFix
• closed
• byDesign
• pending

6. Tag PR as AI-Reviewed

Labels to Add

• ai-reviewed
• ai-model-gpt-5.4
• ai-model-claude-sonnet-4.6

Script

$modelIds = @("gpt-5.4", "claude-sonnet-4.6")
$labels = @("ai-reviewed") + ($modelIds | ForEach-Object { "ai-model-$_" })

Retrieve IDs:

$prInfo    = az repos pr show --id {prId} --detect false | ConvertFrom-Json
$projectId = $prInfo.repository.project.id
$repoId    = $prInfo.repository.id

Get token:

TOKEN=$(az account get-access-token \
  --resource 499b84ac-1321-427f-aa17-267ca6975798 \
  --query accessToken -o tsv)

Fetch existing labels:

curl -sS -H "Authorization: Bearer ${TOKEN}" \
"https://dev.azure.com/{organization}/${PROJECT_ID}/_apis/git/repositories/${REPOSITORY_ID}/pullRequests/{prId}/labels?api-version=7.1"

Add missing labels via POST.

7. If No Issues Found

Post:

### Code review

No issues found. Checked for bugs and instruction file compliance.

🤖 Generated with AI

Then proceed to Step 6 to tag the PR as AI-reviewed.

Avoid False Positives

Do not report:

• Pre-existing issues
• CI/linter/type errors
• Minor stylistic concerns (unless mandated)
• Intentional changes
• Issues in unmodified lines
• Missing tests/docs (unless required)

Inline Comment Format

Each thread:

<brief issue title>

<why this is a problem in this specific code path>

<clear, actionable suggestion>

🤖 Generated with AI

Azure DevOps Code Links

Format:

https://dev.azure.com/{org}/{project}/_git/{repo}?path=/{file-path}&version=GC{commit}&lineStart={start}&lineEnd={end}

Rules:

• Include full commit hash
• Include surrounding context
• File path must start with /

Notes

• Do not run builds or CI checks
• Keep comments concise and actionable
• Always deduplicate findings across models