QA Test Reporter

QA Test Reporter

Purpose

Aggregate, analyze, and report test results across all testing types and frameworks. Parse outputs from Jest, Vitest, Playwright, pytest, Cypress, k6, JMeter, and other tools; produce unified dashboards, trend analysis, and ISO 29119-3 compliant reports with go/no-go recommendations.

Input Formats

See references/report-formats.md for schemas and parsing rules.

Output Formats

Report Sections

Every report includes (where data is available):

1. Summary metrics — Total pass/fail/skip, duration, pass rate %
2. Test results by category — By suite, module, priority, or test type
3. Failure analysis — Top failures by frequency, flaky test detection
4. Coverage summary — Line/branch/function coverage if provided
5. Risk areas — Low coverage, high failure density, blocked tests
6. Open defects — Linked incidents, severity distribution
7. Go/no-go recommendation — Clear recommendation with rationale

Trend Analysis

Track pass/fail rates over time using Memory MCP for history persistence:

• Store run metadata: date, build ID, pass rate, fail count, duration
• Compare current run vs. previous runs (e.g., last 7 days, last 10 builds)
• Surface regressions: pass rate drop, new flaky tests, duration increase
• Use stored trends in executive summary and status reports

Integrations

Trigger Phrases

• "Aggregate test results from [paths/artifacts]"
• "Generate test report from JUnit XML / Allure / JSON"
• "Create HTML test dashboard"
• "ISO 29119-3 test status report" / "test completion report"
• "Go/no-go recommendation for release"
• "Test trend analysis" / "pass rate over time"
• "Parse GitHub Actions test results"

Workflow

1. Collect — Parse one or more report files (JUnit XML, JSON, Allure) or fetch from GitHub Actions
2. Normalize — Map to common model: test name, suite, result, duration, failure message
3. Analyze — Compute metrics, detect flaky tests, identify top failures
4. Trend — Query Memory MCP for history; compare with current run
5. Generate — Produce requested outputs (HTML, Markdown, ISO reports, executive summary)

Scope

Can do (autonomous):

• Parse JUnit XML, JSON, Allure results from provided paths or URLs
• Aggregate results from multiple frameworks into unified view
• Generate HTML dashboard, Markdown summary, ISO 29119-3 reports
• Produce go/no-go recommendation from metrics and risk areas
• Use Memory MCP to store/retrieve trend data
• Use GitHub MCP to fetch Actions results when configured
• Call qa-diagram-generator for trend charts

Cannot do (requires confirmation):

• Fetch from private repos or artifacts without access
• Override stakeholder-defined go/no-go criteria
• Modify test execution or CI configuration

Will not do (out of scope):

• Execute tests or automation
• Modify production code or environments
• Approve release (approval is stakeholder responsibility)

Quality Checklist

• All input formats correctly parsed per references/report-formats.md
• Summary metrics accurate (pass/fail/skip counts match source)
• ISO 29119-3 reports include mandatory sections per references/iso-29119-reports.md
• Go/no-go recommendation supported by metrics and risk analysis
• Trend data stored/retrieved correctly via Memory MCP
• No hardcoded secrets; artifact URLs from env or user input
• HTML dashboard accessible (semantic structure, readable charts)

Troubleshooting

Reference Files