Vcpkg

name: analyze-ci-failures metadata: version: 2025-05-07 description: > Analyze vcpkg Azure DevOps CI failures. Downloads logs, identifies regression root causes, generates a report by package and triplet.

USE FOR: CI failure analysis, regression triage, log diagnosis.

DO NOT USE FOR: general coding, creating ports, modifying portfiles.

UTILITY SKILL INVOKES: Azure DevOps REST API, PowerShell.

vcpkg CI Failures Analyzer

When to Use

• Investigating CI failures on Azure DevOps
• Identifying regressions in a PR or scheduled build
• Triaging root causes before assigning bugs

Overview

Fetches build metadata and failure logs via Azure DevOps REST API, cross-references with baselines, produces a regression report.

MCP Tools

Prerequisites

• GitHub MCP server — needed for PR URL input

Workflow

OUTPUT RULE: Write report.md as SOON as step-log analysis (Phase 1) completes. Do not wait for artifact downloads. You can update the file later. Your response MUST also contain the complete report content — not a summary.

URL RULE: Your response text (not just the report file) MUST include:

• For scheduled/manual builds: the full Azure DevOps URL https://dev.azure.com/vcpkg/public/_build/results?buildId={buildId}
• For PR builds: both the full PR URL https://github.com/microsoft/vcpkg/pull/{prNumber} AND the Azure DevOps build URL

Phase 1: Extract failures from step logs (REQUIRED — do first)

1. Parse input — Extract buildId from Azure DevOps URL. For PR URLs:

   $prNumber = 51515
   $builds = (Invoke-RestMethod "https://dev.azure.com/vcpkg/public/_apis/build/builds?reasonFilter=pullRequest&repositoryType=GitHub&repositoryId=microsoft/vcpkg&branchName=refs/pull/$prNumber/merge&api-version=7.0").value
   $buildId = ($builds | Sort-Object -Property id -Descending | Select-Object -First 1).id
   

   See references/azure-devops-api.md.

2. Fetch metadata — Build info, timeline, and artifacts list (parallelize these API calls).

3. Scan step logs — For each failed job, find "*** Test Modified Ports" task and fetch its log:

   $logText = Invoke-RestMethod "$($task.log.url)?api-version=7.0"
   $regressions = $logText -split "`n" | Where-Object { $_ -match 'REGRESSION:' }
   

   Captures all types: BUILD_FAILED, FILE_CONFLICTS, POST_BUILD_CHECKS_FAILED, CASCADED_DUE_TO_MISSING_DEPENDENCIES. Report every REGRESSION line using the EXACT failure type keyword from the log — never paraphrase. Also capture 2-3 lines around each error for root cause context. Extract the triplet name from each failed job (e.g., from job names like x64-windows Build or from log paths) — ensure ALL triplets with failures are listed in your report.

4. PR feature-test logs — PR builds may NOT have REGRESSION: lines. Instead scan for FAIL: or failed with lines showing per-feature failures. Capture and quote verbatim:

   • Compiler errors (missing headers, undefined symbols)
   • Post-build check failures (file path issues, misplaced files)
   • Version validation errors
   • Platform-specific feature guard messages

   Report each feature failure individually. Dependency ports that fail get their own entry.

5. Version validation — Check "Validate version files" task. If failed, scan for version database errors and quote them verbatim. Fix: vcpkg x-add-version.

6. Write report immediately — Generate and save report.md using all step-log data. This ensures output exists even if later steps time out.

Phase 2: Download logs and enhance (time permitting)

7. Download logs — Invoke-WebRequest or curl shell commands for artifact ZIPs (not web_fetch). Only download "failure logs for {triplet}" — skip "file lists". Extract into ci-failure-analysis/{scope}/logs/{triplet}/. If download fails, still create the directory with a placeholder noting the URL.
8. Analyze — Read stdout-{triplet}.log last lines. Classify per references/vcpkg-failure-patterns.md. Update report with additional root cause detail.
9. Baselines — Check both ci.baseline.txt and ci.feature.baseline.txt.

Report Requirements

Format per references/report-template.md:

• Full build URL: [{buildNumber}](https://dev.azure.com/vcpkg/public/_build/results?buildId={buildId})
• For PRs: [#{prNumber}](https://github.com/microsoft/vcpkg/pull/{prNumber})
• List every triplet by full name (e.g., x64-windows, arm64-linux) — never "N triplets". Only include triplets that actually had failures for this specific build.
• Use exact failure type keywords from logs: BUILD_FAILED, POST_BUILD_CHECKS_FAILED, FILE_CONFLICTS, CASCADED_DUE_TO_MISSING_DEPENDENCIES — never paraphrase
• Include error messages verbatim — quote the exact text from logs for compiler errors, path issues, and validation failures
• Dependency ports' failures as separate entries
• Include baseline/known failures — report them with their failure types, but classify them separately from new regressions

Output Structure

ci-failure-analysis/
├── ci-129315/         ← scheduled build
│   ├── report.md
│   └── logs/
│       ├── x64-windows/
│       └── arm64-linux/
└── pr-51202/          ← PR build
    ├── report.md
    └── logs/

Critical Rules

• Use Invoke-WebRequest or curl shell commands for ZIPs — web_fetch can't download binaries
• Artifact type is PipelineArtifact — Container API won't work
• Scan step logs first — FILE_CONFLICTS only appear there
• Check both baseline files
• Never suggest <= version constraints or VCPKG_BUILD_TYPE release
• If artifact download fails, still create logs/{triplet}/ directory with a placeholder noting the download URL

Source: microsoft/vcpkg — distributed by TomeVault.