# Fetch Test Report > Fetch an OpenShift CI test report by name to get pass rates, test ID, and Jira component from Sippy Use when this capability is needed. - Skill: `tomevault-io/fetch-test-report` (Agent Skill, multi-file: 2 files) - Install (CLI): `npx skillmds@latest add tomevault-io/fetch-test-report` - Raw SKILL.md: https://api.skillmd.com/api/skills/tomevault-io/fetch-test-report/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Web & Frontend - Author: tomevault-io (https://skillmd.com/u/tomevault-io) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/tomevault-io/fetch-test-report --- # Fetch Test Report This skill fetches a report for an OpenShift CI test by its full name using the Sippy tests API. It returns test metadata including the BigQuery/Component Readiness test ID, Jira component, pass rates for the current and previous reporting periods, and open bug counts. The `open_bugs` field counts Jira bugs that mention this test by name, which can help surface bugs that have been filed but not yet triaged in Component Readiness. ## When to Use This Skill Use this skill when you need to: - Look up a test's Component Readiness / BigQuery test ID (`test_id`) - Check current and previous pass rates for a test - Find the Jira component associated with a test - Determine if a test is flaking, failing, or stable - Get run counts and failure/flake breakdowns for a test - Check if there are open Jira bugs mentioning this test (may not yet be triaged in Component Readiness) - See a per-variant breakdown of pass rates to identify if the test fails only in certain job types ## Prerequisites 1. **Network Access**: The Sippy API must be accessible at `https://sippy.dptools.openshift.org` - No authentication required - Check: `curl -s https://sippy.dptools.openshift.org/api/health?release=4.22` 2. **Python 3**: Python 3.6 or later - Check: `python3 --version` - Uses only standard library (no external dependencies) ## Implementation Steps ### Step 1: Determine the Release If the user did not specify a release, use the `fetch-releases` skill to determine the latest OCP release: ```bash release=$(python3 plugins/ci/skills/fetch-releases/fetch_releases.py --latest) ``` If the user specified a release, use that directly. ### Step 2: Run the Python Script The skill uses a Python script to query the Sippy tests API: ```bash # Path to the Python script script_path="plugins/ci/skills/fetch-test-report/fetch_test_report.py" # Fetch test report in JSON format (collapsed — one row for the test across all variants) python3 "$script_path" "" --release "$release" --format json # Or fetch as human-readable summary python3 "$script_path" "" --release "$release" --format summary # Fetch per-variant breakdown (one row per variant combo — useful for identifying # if the test is failing only in certain job types, e.g., aws vs metal, upgrade vs new-install) python3 "$script_path" "" --release "$release" --no-collapse --format json ``` ### Step 3: Parse the Output The script outputs a JSON array (typically with one element for an exact name match): ```bash # Store JSON output in a variable for processing test_data=$(python3 "$script_path" "[sig-api-machinery] Discovery should validate PreferredVersion for each APIGroup [Conformance]" --format json) # Extract specific fields using jq test_id=$(echo "$test_data" | jq -r '.[0].test_id') component=$(echo "$test_data" | jq -r '.[0].jira_component') pass_rate=$(echo "$test_data" | jq -r '.[0].current_pass_percentage') runs=$(echo "$test_data" | jq -r '.[0].current_runs') ``` ## API Details ### Endpoint ``` GET https://sippy.dptools.openshift.org/api/tests/v2?release={release}&filter={filter_json} GET https://sippy.dptools.openshift.org/api/tests/v2?release={release}&filter={filter_json}&collapse=false ``` When `collapse=false` is specified, the API returns one row per variant combination instead of a single collapsed row. Each row includes a `variants` array showing the specific variant combo (e.g., `["aws", "ovn", "amd64", "upgrade-micro"]`). This helps identify if the test is failing in certain types of jobs and not others. ### Filter Format The filter parameter is a JSON object using Sippy's standard filter syntax: ```json { "items": [ { "columnField": "name", "operatorValue": "equals", "value": "" } ] } ``` ### Response Schema The API returns a JSON array of test objects: ```json [ { "id": 5987, "test_id": "openshift-tests:e8f7491a505095b6356dfd8d4cf7218b", "name": "[sig-api-machinery] Discovery should validate PreferredVersion for each APIGroup [Conformance]", "suite_name": "openshift-tests", "variants": null, "jira_component": "kube-apiserver", "jira_component_id": "12367637", "current_successes": 3425, "current_failures": 22, "current_flakes": 0, "current_pass_percentage": 99.36, "current_failure_percentage": 0.64, "current_flake_percentage": 0, "current_working_percentage": 99.36, "current_runs": 3447, "previous_successes": 2570, "previous_failures": 43, "previous_flakes": 0, "previous_pass_percentage": 98.35, "previous_failure_percentage": 1.65, "previous_flake_percentage": 0, "previous_working_percentage": 98.35, "previous_runs": 2613, "net_failure_improvement": 1.01, "net_flake_improvement": 0, "net_working_improvement": 1.01, "net_improvement": 1.01, "tags": null, "open_bugs": 0 } ] ``` **Key Fields:** - `id`: Sippy PostgreSQL internal ID (less commonly used) - `test_id`: BigQuery / Component Readiness test ID (format: `suite:hash`). This is the ID used with modern skills like `fetch-regression-details` and `fetch-test-runs`. - `name`: Full test name - `suite_name`: Test suite (e.g., `openshift-tests`) - `variants`: Array of variant strings (only present when `--no-collapse` / `collapse=false` is used). Shows the specific variant combo for this row (e.g., `["aws", "ovn", "amd64", "upgrade-micro"]`). - `jira_component`: Associated OCPBUGS Jira component - `jira_component_id`: Jira component numeric ID - `current_*`: Metrics for the current 7-day reporting period - `previous_*`: Metrics for the 7 days before the current period - `net_working_improvement`: Change in working percentage (positive = improving) - `open_bugs`: Number of open Jira bugs that mention this test by name. This can surface bugs that have been filed but not yet triaged in Component Readiness. Useful for finding existing work before filing a duplicate. ## Error Handling ### Case 1: Sippy Not Reachable ```bash python3 fetch_test_report.py "[sig-api-machinery] ..." # Error: Failed to connect to Sippy API: [Errno 61] Connection refused # Check network connectivity to sippy.dptools.openshift.org. ``` ### Case 2: No Test Found If the test name doesn't match any test, the script returns an empty JSON array `[]` or "No tests found matching the given name." in summary mode. ### Case 3: Missing Arguments ```bash python3 fetch_test_report.py # usage: fetch_test_report.py [-h] [--release RELEASE] [--format {json,summary}] test_name # fetch_test_report.py: error: the following arguments are required: test_name ``` **Exit Codes:** - `0`: Success - `1`: Error (API error, network error, etc.) ## Examples ### Example 1: Fetch Test Report in JSON Format ```bash python3 plugins/ci/skills/fetch-test-report/fetch_test_report.py \ "[sig-api-machinery] Discovery should validate PreferredVersion for each APIGroup [Conformance]" \ --release 4.22 --format json ``` ### Example 2: Get a Human-Readable Summary ```bash python3 plugins/ci/skills/fetch-test-report/fetch_test_report.py \ "[sig-api-machinery] Discovery should validate PreferredVersion for each APIGroup [Conformance]" \ --release 4.22 --format summary ``` **Expected Output:** ``` Test: [sig-api-machinery] Discovery should validate PreferredVersion for each APIGroup [Conformance] Test ID: openshift-tests:abc123def456 Suite: openshift-tests Jira Component: kube-apiserver Open Bugs: 0 Current Period (last 7 days): Runs: 3447 Pass Rate: 99.36% Failures: 22 (0.64%) Flakes: 0 (0.00%) Previous Period (7 days before current): Runs: 2613 Pass Rate: 98.35% Failures: 43 (1.65%) Flakes: 0 (0.00%) Trend: improved (+1.01%) ``` ### Example 3: Fetch Per-Variant Breakdown ```bash python3 plugins/ci/skills/fetch-test-report/fetch_test_report.py \ "[sig-api-machinery] Discovery should validate PreferredVersion for each APIGroup [Conformance]" \ --release 4.22 --no-collapse --format summary ``` This returns one row per variant combination, showing which specific job types the test is passing or failing in. Useful for regression analysis to determine if the failure is platform-specific, upgrade-specific, etc. ### Example 4: Fetch Test Report for a Different Release ```bash python3 plugins/ci/skills/fetch-test-report/fetch_test_report.py \ "[sig-api-machinery] Discovery should validate PreferredVersion for each APIGroup [Conformance]" \ --release 4.21 --format json ``` ## Notes - The test name must be an **exact match** — use the full test name including sig prefix and suite tags - The API is accessed via the production Sippy URL; no authentication required - If `--release` is not specified, use the `fetch-releases` skill to determine the latest release - `current_*` fields cover the last 7 days; `previous_*` fields cover the 7 days before that - `current_working_percentage` = pass rate + flake rate (tests that ultimately passed, possibly after retries) - The `test_id` field is the one used in Component Readiness URLs and with the `fetch-regression-details` and `fetch-test-runs` skills - An empty result (`[]`) means no test matched — double-check the exact test name spelling ## See Also - Related Skill: `fetch-releases` (determines the latest OCP release) - Related Skill: `fetch-regression-details` (uses `test_id` to fetch regression details) - Related Skill: `fetch-test-runs` (uses test name to fetch individual test run results) - Related Command: `/ci:fetch-test-report` (command that invokes this skill) - Related Command: `/ci:analyze-regression` (full regression analysis workflow) --- > Converted and distributed by [TomeVault](https://tomevault.io/claim/openshift-eng) — claim your Tome and manage your conversions.