Azure DevOps Testing
Shared Workflow Routing
Overview
Azure Test Plans provides manual and exploratory testing integrated with Azure Boards and Pipelines. Test plans organize test suites (static, requirement-based, query-based), which contain test cases with defined steps and expected results. Test configurations define the matrix of OS/browser/environment combinations. Test runs record pass/fail results with screenshots and attachments.
Pipeline test results published via PublishTestResults@2 feed into Analytics for trend reporting, flaky test detection, and test impact analysis.
REST API — Test Plans
| Method |
Endpoint |
Required Permissions |
Key Parameters |
| GET |
/_apis/testplan/plans?api-version=7.1 |
Test Plans (Read) |
$top, continuationToken |
| POST |
/_apis/testplan/plans?api-version=7.1 |
Test Plans (Write) |
Body: name, areaPath, iteration |
| GET |
/_apis/testplan/plans/{planId}?api-version=7.1 |
Test Plans (Read) |
— |
| PATCH |
/_apis/testplan/plans/{planId}?api-version=7.1 |
Test Plans (Write) |
Body: fields to update |
| DELETE |
/_apis/testplan/plans/{planId}?api-version=7.1 |
Test Plans (Write) |
— |
REST API — Test Suites
| Method |
Endpoint |
Key Parameters |
| GET |
/_apis/testplan/plans/{planId}/suites?api-version=7.1 |
$top, asTreeView |
| POST |
/_apis/testplan/plans/{planId}/suites?api-version=7.1 |
Body: name, suiteType, parentSuite |
| GET |
/_apis/testplan/plans/{planId}/suites/{suiteId}?api-version=7.1 |
— |
| DELETE |
/_apis/testplan/plans/{planId}/suites/{suiteId}?api-version=7.1 |
— |
Suite Types
| Type |
suiteType Value |
Description |
| Static |
staticTestSuite |
Manually grouped test cases |
| Requirement-based |
requirementTestSuite |
Auto-linked from requirement work item |
| Query-based |
dynamicTestSuite |
Populated by WIQL query |
REST API — Test Cases & Points
| Method |
Endpoint |
Key Parameters |
| POST |
/_apis/testplan/plans/{planId}/suites/{suiteId}/testcase?api-version=7.1 |
Body: [{ workItem: { id } }] |
| GET |
/_apis/testplan/plans/{planId}/suites/{suiteId}/testpoint?api-version=7.1 |
testCaseId, configurationId |
| PATCH |
/_apis/testplan/plans/{planId}/suites/{suiteId}/testpoint?api-version=7.1 |
Body: [{ id, results: { outcome } }] |
REST API — Test Runs & Results
| Method |
Endpoint |
Key Parameters |
| GET |
/_apis/test/runs?api-version=7.1 |
minLastUpdatedDate, maxLastUpdatedDate |
| POST |
/_apis/test/runs?api-version=7.1 |
Body: name, plan, pointIds |
| GET |
/_apis/test/runs/{runId}?api-version=7.1 |
— |
| PATCH |
/_apis/test/runs/{runId}?api-version=7.1 |
Body: state (Completed, Aborted) |
| GET |
/_apis/test/runs/{runId}/results?api-version=7.1 |
$top, outcomes |
| POST |
/_apis/test/runs/{runId}/results?api-version=7.1 |
Body: [{ testCaseTitle, outcome, comment }] |
Test Outcomes
| Outcome |
Value |
Description |
| Passed |
Passed |
Test passed all steps |
| Failed |
Failed |
One or more steps failed |
| Blocked |
Blocked |
Cannot execute (dependency/environment) |
| Not Applicable |
NotApplicable |
Test not relevant for this config |
| Not Executed |
NotExecuted |
Not yet run |
REST API — Test Configurations
| Method |
Endpoint |
Key Parameters |
| GET |
/_apis/testplan/configurations?api-version=7.1 |
$top |
| POST |
/_apis/testplan/configurations?api-version=7.1 |
Body: name, values |
Configuration Example
{
"name": "Windows 11 + Chrome",
"values": [
{ "name": "Operating System", "value": "Windows 11" },
{ "name": "Browser", "value": "Chrome" }
]
}
Test Analytics via OData
Endpoint: https://analytics.dev.azure.com/{org}/{project}/_odata/v4.0-preview/
| Entity Set |
Key Fields |
TestRuns |
TestRunId, Title, StartedDate, CompletedDate, State, TotalTests, PassedTests |
TestResults |
TestCaseReferenceId, Outcome, DurationSeconds, CompletedDate |
TestResultsDaily |
Aggregated daily pass/fail counts |
Flaky Test Query
TestResults?$filter=Outcome eq 'Failed' and CompletedDate ge 2024-01-01Z
&$apply=groupby((TestCaseReferenceId, TestCaseTitle),
aggregate(ResultCount with sum as TotalRuns,
FailedCount with sum as TotalFails))
&$orderby=TotalFails desc
&$top=20
Best Practices
- Use requirement-based suites to auto-link test coverage to user stories.
- Define configurations for cross-platform test matrices.
- Publish pipeline test results with
PublishTestResults@2 for Analytics integration.
- Use test impact analysis to run only tests affected by code changes.
- Track flaky tests via Analytics OData aggregation queries.
- Use shared steps for common preconditions across test cases.
Progressive Disclosure — Reference Files
1---2name: azure-devops-testing3description: Deep expertise in Azure Test Plans — test plan creation, test suite management (static, requirement-based, query-based), test case authoring, test runs and results, test configurations, and test analytics via OData.4---56# Azure DevOps Testing78## Shared Workflow Routing9- Use the shared workflow spec for deterministic multi-plugin routing: [`workflows/multi-plugin-workflows.md`](../../../workflows/multi-plugin-workflows.md#incident-triage-azure-monitor--azure-functions--azure-devops).1011## Overview1213Azure Test Plans provides manual and exploratory testing integrated with Azure Boards and Pipelines. Test plans organize test suites (static, requirement-based, query-based), which contain test cases with defined steps and expected results. Test configurations define the matrix of OS/browser/environment combinations. Test runs record pass/fail results with screenshots and attachments.1415Pipeline test results published via `PublishTestResults@2` feed into Analytics for trend reporting, flaky test detection, and test impact analysis.1617## REST API — Test Plans1819| Method | Endpoint | Required Permissions | Key Parameters |20|--------|----------|---------------------|----------------|21| GET | `/_apis/testplan/plans?api-version=7.1` | Test Plans (Read) | `$top`, `continuationToken` |22| POST | `/_apis/testplan/plans?api-version=7.1` | Test Plans (Write) | Body: `name`, `areaPath`, `iteration` |23| GET | `/_apis/testplan/plans/{planId}?api-version=7.1` | Test Plans (Read) | — |24| PATCH | `/_apis/testplan/plans/{planId}?api-version=7.1` | Test Plans (Write) | Body: fields to update |25| DELETE | `/_apis/testplan/plans/{planId}?api-version=7.1` | Test Plans (Write) | — |2627## REST API — Test Suites2829| Method | Endpoint | Key Parameters |30|--------|----------|----------------|31| GET | `/_apis/testplan/plans/{planId}/suites?api-version=7.1` | `$top`, `asTreeView` |32| POST | `/_apis/testplan/plans/{planId}/suites?api-version=7.1` | Body: `name`, `suiteType`, `parentSuite` |33| GET | `/_apis/testplan/plans/{planId}/suites/{suiteId}?api-version=7.1` | — |34| DELETE | `/_apis/testplan/plans/{planId}/suites/{suiteId}?api-version=7.1` | — |3536### Suite Types3738| Type | `suiteType` Value | Description |39|------|-------------------|-------------|40| Static | `staticTestSuite` | Manually grouped test cases |41| Requirement-based | `requirementTestSuite` | Auto-linked from requirement work item |42| Query-based | `dynamicTestSuite` | Populated by WIQL query |4344## REST API — Test Cases & Points4546| Method | Endpoint | Key Parameters |47|--------|----------|----------------|48| POST | `/_apis/testplan/plans/{planId}/suites/{suiteId}/testcase?api-version=7.1` | Body: `[{ workItem: { id } }]` |49| GET | `/_apis/testplan/plans/{planId}/suites/{suiteId}/testpoint?api-version=7.1` | `testCaseId`, `configurationId` |50| PATCH | `/_apis/testplan/plans/{planId}/suites/{suiteId}/testpoint?api-version=7.1` | Body: `[{ id, results: { outcome } }]` |5152## REST API — Test Runs & Results5354| Method | Endpoint | Key Parameters |55|--------|----------|----------------|56| GET | `/_apis/test/runs?api-version=7.1` | `minLastUpdatedDate`, `maxLastUpdatedDate` |57| POST | `/_apis/test/runs?api-version=7.1` | Body: `name`, `plan`, `pointIds` |58| GET | `/_apis/test/runs/{runId}?api-version=7.1` | — |59| PATCH | `/_apis/test/runs/{runId}?api-version=7.1` | Body: `state` (Completed, Aborted) |60| GET | `/_apis/test/runs/{runId}/results?api-version=7.1` | `$top`, `outcomes` |61| POST | `/_apis/test/runs/{runId}/results?api-version=7.1` | Body: `[{ testCaseTitle, outcome, comment }]` |6263### Test Outcomes6465| Outcome | Value | Description |66|---------|-------|-------------|67| Passed | `Passed` | Test passed all steps |68| Failed | `Failed` | One or more steps failed |69| Blocked | `Blocked` | Cannot execute (dependency/environment) |70| Not Applicable | `NotApplicable` | Test not relevant for this config |71| Not Executed | `NotExecuted` | Not yet run |7273## REST API — Test Configurations7475| Method | Endpoint | Key Parameters |76|--------|----------|----------------|77| GET | `/_apis/testplan/configurations?api-version=7.1` | `$top` |78| POST | `/_apis/testplan/configurations?api-version=7.1` | Body: `name`, `values` |7980### Configuration Example81```json82{83 "name": "Windows 11 + Chrome",84 "values": [85 { "name": "Operating System", "value": "Windows 11" },86 { "name": "Browser", "value": "Chrome" }87 ]88}89```9091## Test Analytics via OData9293Endpoint: `https://analytics.dev.azure.com/{org}/{project}/_odata/v4.0-preview/`9495| Entity Set | Key Fields |96|------------|------------|97| `TestRuns` | `TestRunId`, `Title`, `StartedDate`, `CompletedDate`, `State`, `TotalTests`, `PassedTests` |98| `TestResults` | `TestCaseReferenceId`, `Outcome`, `DurationSeconds`, `CompletedDate` |99| `TestResultsDaily` | Aggregated daily pass/fail counts |100101### Flaky Test Query102```103TestResults?$filter=Outcome eq 'Failed' and CompletedDate ge 2024-01-01Z104 &$apply=groupby((TestCaseReferenceId, TestCaseTitle),105 aggregate(ResultCount with sum as TotalRuns,106 FailedCount with sum as TotalFails))107 &$orderby=TotalFails desc108 &$top=20109```110111## Best Practices112113- Use requirement-based suites to auto-link test coverage to user stories.114- Define configurations for cross-platform test matrices.115- Publish pipeline test results with `PublishTestResults@2` for Analytics integration.116- Use test impact analysis to run only tests affected by code changes.117- Track flaky tests via Analytics OData aggregation queries.118- Use shared steps for common preconditions across test cases.119120## Progressive Disclosure — Reference Files121122| Topic | File |123|---|---|124| Test plans, suites, cases, configurations, manual testing workflow | [`references/test-plans.md`](./references/test-plans.md) |125| Test analytics, OData queries, flaky detection, trends | [`references/test-analytics.md`](./references/test-analytics.md) |
thelobbi