Overview
The role-qa-automation-engineer skill ships nothing on vibes: every release claim must be backed by hermetic test executions, deterministic retries, and a machine-verifiable certification receipt that records exit codes, coverage deltas, and pass counts.
When to Use
- When building or refactoring automated test suites across unit, integration, and E2E layers.
- When generating comprehensive test fixtures and edge-case test matrices for complex business logic.
- When validating REST/GraphQL APIs against schema specifications, error envelopes, and rate limits.
- When automating web user journeys using Playwright or Cypress with resilient locator strategies.
- When measuring true branch/condition test coverage and isolating untested execution paths.
- When diagnosing flaky tests, timing races, and non-deterministic test failures in CI.
When NOT to Use
- When fixing internal code implementation bugs or refactoring production code (use
development/role-software-engineer).
- When conducting penetration testing or hunting for security vulnerabilities (use
cybersecurity/role-security-engineer).
- When managing CI runner infrastructure or container build pipelines (use
devops/role-devops-sre-engineer).
- When designing UI design tokens or styling mockups (use
content/role-ui-ux-designer).
Deterministic Dispatch Matrix
| Verification Scope / Defect Challenge |
Delegated Child Skill |
Primary Testing Artifact |
| Auto-generating test cases from contracts, schemas, or specs |
development/automated-test-generator |
Parameterized test suite covering edge and boundary cases |
| Validating REST/GraphQL API contracts, envelopes, and status codes |
development/api-testing |
Automated API test suite with hermetic mock fixtures |
| Cross-browser UI testing, headless runs, and visual locators |
development/playwright-e2e |
Headless Playwright specs with resilient auto-waiting |
| Component testing, legacy frontend suites, and DOM interaction |
development/cypress-e2e |
Cypress integration specs and custom command fixtures |
| Live DevTools inspection, network interception, and console audits |
development/browser-testing-devtools |
CDP network traces, console error audits, DOM captures |
| Identifying untested branches, lines, and dead test paths |
development/test-coverage-analyzer |
Coverage delta report and missing-branch test targets |
| Systematic QA audit loops, pre-release certification, bug remediation |
development/qa-review-fix-loop |
Audit ledger, defect fix loop, and release sign-off |
| Flaky test stabilization, mutation testing, and chaos verification |
development/agentic-quality-engineering |
Mutation survival scores and deterministic retry harness |
Workflow
Phase 1: Test Strategy & Risk Profiling
- Identify critical business paths (e.g. checkout, auth, data export, state mutations).
- Establish the test pyramid allocation (70% Unit, 20% API/Integration, 10% E2E).
- Dispatch to
development/automated-test-generator to scaffold edge and boundary case matrices.
Phase 2: Hermetic Harness & Fixture Setup
- Isolate test execution from shared external environments (use ephemeral SQLite/PostgreSQL, mock third-party APIs).
- Wire API contract and integration test suites using
development/api-testing.
- Ensure fixtures are immutable and auto-clearing between test runs.
Phase 3: Browser & End-to-End Automation
- For modern web applications, scaffold E2E specifications via
development/playwright-e2e.
- For legacy or existing Cypress suites, implement flows via
development/cypress-e2e.
- Eliminate arbitrary
sleep() calls; enforce deterministic condition polling and accessibility role selectors (getByRole, getByTestId).
- Inspect live browser execution, network traffic, and DOM health using
development/browser-testing-devtools.
Phase 4: Coverage Analysis & Flakiness Elimination
- Measure branch and statement coverage using
development/test-coverage-analyzer.
- Target all untested error branches and exception handlers.
- Apply chaos testing, mutation testing, and race stabilization via
development/agentic-quality-engineering.
- Ensure tests pass deterministically 10/10 consecutive runs under parallel contention.
Phase 5: QA Loop & Release Certification
- Execute the full end-to-end audit battery using
development/qa-review-fix-loop.
- Compile pass/fail telemetry, execution durations, and coverage metrics.
- Issue formal QA certification receipt before staging or production rollout.
Anti-Rationalization Table
| Testing Excuse |
Flawed Rationalization |
Required QA Discipline |
| "E2E tests are too slow, unit tests are enough" |
"Unit tests prove each function works in isolation." |
Unit tests cannot catch integration drift, broken routes, or hydration mismatches. Maintain a lean, high-value E2E suite. |
| "The test only fails intermittently, it's just CI noise" |
"Rerunning the job makes it pass, so it's fine." |
Flaky tests hide real concurrency bugs and destroy team trust. Quarantine and fix root cause immediately. |
"Add sleep(5000) to fix the timing issue" |
"A 5-second pause ensures the element is loaded." |
Arbitrary sleeps slow suites and still fail on slow CI runners. Use explicit condition waits (waitForSelector, expect().toBeVisible()). |
| "100% line coverage means zero bugs" |
"Every line was touched during the test run." |
Line coverage without assertion depth is meaningless. Test boundary values, error branches, and edge conditions. |
| "Manual testing is faster for this sprint" |
"We will automate the test suite after release." |
Unautomated manual tests never get written post-release. Automate regression checks before declaring work done. |
Implementation Examples
Automated Test Runner & Flakiness Detector (Python)
import subprocess
import sys
from typing import List
def run_flakiness_check(test_target: str, iterations: int = 5) -> bool:
"""Runs a test target repeatedly to detect non-deterministic flakiness."""
print(f"=== Starting Flakiness Detection: {test_target} ({iterations} runs) ===")
for i in range(1, iterations + 1):
print(f"Run {i}/{iterations}...", end=" ", flush=True)
res = subprocess.run(
[sys.executable, "-m", "pytest", test_target, "-q"],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True
)
if res.returncode != 0:
print(f"FAILED on run {i}!")
print(f"Error output:\n{res.stdout}\n{res.stderr}")
return False
print("PASSED")
print(f"SUCCESS: {test_target} passed {iterations}/{iterations} runs deterministically.")
return True
if __name__ == "__main__":
test_target = "tests/test_api_contract.py"
# Example invocation: run_flakiness_check(test_target)
print("Flakiness detection framework initialized.")
Headless Playwright Test Script (Bash/Node)
#!/usr/bin/env bash
set -euo pipefail
echo "=== Executing Playwright E2E Suite ==="
npx playwright test --project=chromium --reporter=list
echo "=== Auditing Test Coverage ==="
npm run test:coverage -- --coverageReporters="text-summary"
Verification Checklist
npm install -g cli-tool
1---2name: role-qa-automation-engineer3description: Use when orchestrating end-to-end quality assurance, test automation4 suites, API integration testing, browser E2E flows, coverage auditing, and chaos5 resilience.6domain: development7author: oyi778version: 1.0.09category: development10tags:11- qa12- testing13- automation14- playwright15- cypress16license: MIT17subdomain: qa-automation18triggers:19- qa automation role20- e2e test suite21- playwright suite22- flaky test triage23- coverage gap audit24- api contract test25- test pyramid26- browser automation27- cypress test28- release certification29metadata:30 source: https://skillmd.com/skills/oyi77/role-qa-automation-engineer31---3233## Overview3435The `role-qa-automation-engineer` skill ships nothing on vibes: every release claim must be backed by hermetic test executions, deterministic retries, and a machine-verifiable certification receipt that records exit codes, coverage deltas, and pass counts.3637## When to Use3839- When building or refactoring automated test suites across unit, integration, and E2E layers.40- When generating comprehensive test fixtures and edge-case test matrices for complex business logic.41- When validating REST/GraphQL APIs against schema specifications, error envelopes, and rate limits.42- When automating web user journeys using Playwright or Cypress with resilient locator strategies.43- When measuring true branch/condition test coverage and isolating untested execution paths.44- When diagnosing flaky tests, timing races, and non-deterministic test failures in CI.4546## When NOT to Use4748- When fixing internal code implementation bugs or refactoring production code (use `development/role-software-engineer`).49- When conducting penetration testing or hunting for security vulnerabilities (use `cybersecurity/role-security-engineer`).50- When managing CI runner infrastructure or container build pipelines (use `devops/role-devops-sre-engineer`).51- When designing UI design tokens or styling mockups (use `content/role-ui-ux-designer`).5253## Deterministic Dispatch Matrix5455| Verification Scope / Defect Challenge | Delegated Child Skill | Primary Testing Artifact |56|---|---|---|57| Auto-generating test cases from contracts, schemas, or specs | `development/automated-test-generator` | Parameterized test suite covering edge and boundary cases |58| Validating REST/GraphQL API contracts, envelopes, and status codes | `development/api-testing` | Automated API test suite with hermetic mock fixtures |59| Cross-browser UI testing, headless runs, and visual locators | `development/playwright-e2e` | Headless Playwright specs with resilient auto-waiting |60| Component testing, legacy frontend suites, and DOM interaction | `development/cypress-e2e` | Cypress integration specs and custom command fixtures |61| Live DevTools inspection, network interception, and console audits | `development/browser-testing-devtools` | CDP network traces, console error audits, DOM captures |62| Identifying untested branches, lines, and dead test paths | `development/test-coverage-analyzer` | Coverage delta report and missing-branch test targets |63| Systematic QA audit loops, pre-release certification, bug remediation | `development/qa-review-fix-loop` | Audit ledger, defect fix loop, and release sign-off |64| Flaky test stabilization, mutation testing, and chaos verification | `development/agentic-quality-engineering` | Mutation survival scores and deterministic retry harness |6566## Workflow6768### Phase 1: Test Strategy & Risk Profiling69701. Identify critical business paths (e.g. checkout, auth, data export, state mutations).712. Establish the test pyramid allocation (70% Unit, 20% API/Integration, 10% E2E).723. Dispatch to `development/automated-test-generator` to scaffold edge and boundary case matrices.7374### Phase 2: Hermetic Harness & Fixture Setup75761. Isolate test execution from shared external environments (use ephemeral SQLite/PostgreSQL, mock third-party APIs).772. Wire API contract and integration test suites using `development/api-testing`.783. Ensure fixtures are immutable and auto-clearing between test runs.7980### Phase 3: Browser & End-to-End Automation81821. For modern web applications, scaffold E2E specifications via `development/playwright-e2e`.832. For legacy or existing Cypress suites, implement flows via `development/cypress-e2e`.843. Eliminate arbitrary `sleep()` calls; enforce deterministic condition polling and accessibility role selectors (`getByRole`, `getByTestId`).854. Inspect live browser execution, network traffic, and DOM health using `development/browser-testing-devtools`.8687### Phase 4: Coverage Analysis & Flakiness Elimination88891. Measure branch and statement coverage using `development/test-coverage-analyzer`.902. Target all untested error branches and exception handlers.913. Apply chaos testing, mutation testing, and race stabilization via `development/agentic-quality-engineering`.924. Ensure tests pass deterministically 10/10 consecutive runs under parallel contention.9394### Phase 5: QA Loop & Release Certification95961. Execute the full end-to-end audit battery using `development/qa-review-fix-loop`.972. Compile pass/fail telemetry, execution durations, and coverage metrics.983. Issue formal QA certification receipt before staging or production rollout.99100## Anti-Rationalization Table101102| Testing Excuse | Flawed Rationalization | Required QA Discipline |103|---|---|---|104| "E2E tests are too slow, unit tests are enough" | "Unit tests prove each function works in isolation." | Unit tests cannot catch integration drift, broken routes, or hydration mismatches. Maintain a lean, high-value E2E suite. |105| "The test only fails intermittently, it's just CI noise" | "Rerunning the job makes it pass, so it's fine." | Flaky tests hide real concurrency bugs and destroy team trust. Quarantine and fix root cause immediately. |106| "Add `sleep(5000)` to fix the timing issue" | "A 5-second pause ensures the element is loaded." | Arbitrary sleeps slow suites and still fail on slow CI runners. Use explicit condition waits (`waitForSelector`, `expect().toBeVisible()`). |107| "100% line coverage means zero bugs" | "Every line was touched during the test run." | Line coverage without assertion depth is meaningless. Test boundary values, error branches, and edge conditions. |108| "Manual testing is faster for this sprint" | "We will automate the test suite after release." | Unautomated manual tests never get written post-release. Automate regression checks before declaring work done. |109110## Implementation Examples111112### Automated Test Runner & Flakiness Detector (Python)113114```python115import subprocess116import sys117from typing import List118119def run_flakiness_check(test_target: str, iterations: int = 5) -> bool:120 """Runs a test target repeatedly to detect non-deterministic flakiness."""121 print(f"=== Starting Flakiness Detection: {test_target} ({iterations} runs) ===")122 123 for i in range(1, iterations + 1):124 print(f"Run {i}/{iterations}...", end=" ", flush=True)125 res = subprocess.run(126 [sys.executable, "-m", "pytest", test_target, "-q"],127 stdout=subprocess.PIPE,128 stderr=subprocess.PIPE,129 text=True130 )131 if res.returncode != 0:132 print(f"FAILED on run {i}!")133 print(f"Error output:\n{res.stdout}\n{res.stderr}")134 return False135 print("PASSED")136 137 print(f"SUCCESS: {test_target} passed {iterations}/{iterations} runs deterministically.")138 return True139140if __name__ == "__main__":141 test_target = "tests/test_api_contract.py"142 # Example invocation: run_flakiness_check(test_target)143 print("Flakiness detection framework initialized.")144```145146### Headless Playwright Test Script (Bash/Node)147148```bash149#!/usr/bin/env bash150set -euo pipefail151152echo "=== Executing Playwright E2E Suite ==="153npx playwright test --project=chromium --reporter=list154155echo "=== Auditing Test Coverage ==="156npm run test:coverage -- --coverageReporters="text-summary"157```158159## Verification Checklist160161- [ ] Critical business user flows mapped to automated test scenarios.162- [ ] API integration tests validate response payloads, status codes, and error envelopes.163- [ ] Browser E2E specs use resilient accessibility selectors without arbitrary time sleeps.164- [ ] Test coverage measured with zero uncovered critical error branches.165- [ ] Flaky test detection executed with 100% deterministic passes across consecutive runs.166- [ ] Formal QA execution report compiled with pass counts and timing receipts.167168`npm install -g cli-tool`