name: testing-strategy
description: Accessibility testing decision trees, browser/AT compatibility matrices, manual vs. automated test coverage, regression testing patterns, and acceptance criteria templates for user stories.
Testing Strategy Skill
Decision frameworks for accessibility testing — when to use automated tools vs. manual testing, which screen reader + browser combinations to test, and how to write accessibility acceptance criteria.
Automated vs. Manual Testing Coverage
| What Automated Tools Catch (~30-40%) |
What Requires Manual Testing (~60-70%) |
| Missing alt text on images |
Alt text quality and accuracy |
| Missing form labels |
Label clarity and helpfulness |
| Color contrast ratios (computed) |
Color contrast in context (gradients, images) |
| Missing lang attribute |
Correct language identification |
| Duplicate IDs |
Logical reading order |
| Missing ARIA roles on custom widgets |
Correct ARIA role for the interaction pattern |
| Heading hierarchy violations |
Heading text meaningfulness |
| Empty links and buttons |
Link/button text descriptiveness |
| Missing table headers |
Table caption and header association quality |
| Syntax errors in ARIA |
Screen reader announcement correctness |
Browser + Screen Reader Compatibility Matrix
Primary Test Combinations (Required)
| Screen Reader |
Browser |
OS |
Priority |
| NVDA |
Firefox |
Windows |
Must test |
| NVDA |
Chrome |
Windows |
Must test |
| JAWS |
Chrome |
Windows |
Must test |
| VoiceOver |
Safari |
macOS |
Must test |
| VoiceOver |
Safari |
iOS |
Must test |
| TalkBack |
Chrome |
Android |
Should test |
Secondary (Nice to Have)
| Screen Reader |
Browser |
OS |
| Narrator |
Edge |
Windows |
| JAWS |
Edge |
Windows |
Testing Decision Tree
Is it a new component or page?
├── Yes → Full test coverage (automated + manual)
│ ├── Run axe-core / Lighthouse scan
│ ├── Keyboard-only navigation test
│ ├── Screen reader announcement test (NVDA + VoiceOver minimum)
│ └── Visual check at 200% zoom
└── No → What changed?
├── Colors/styling → Contrast check + visual review
├── Interactive behavior → Keyboard + screen reader test
├── Content/text → Screen reader announcement check
├── Layout/order → Reading order + focus order test
└── Dependencies updated → Regression scan (axe-core)
Regression Testing Patterns
CI Pipeline Accessibility Gates
- axe-core scan: Run on every PR. Fail on new critical/serious violations.
- Baseline management: Store known issues in
.a11y-baseline.json. Only fail on new issues.
- Lighthouse score threshold: Set minimum accessibility score (e.g., 90). Fail on regression.
- Visual regression: Capture screenshots at 200% zoom. Compare for focus indicator and layout changes.
Preventing Regressions
- Add accessibility assertions to existing component tests (see
testing-accessibility.instructions.md)
- Include keyboard navigation in E2E test suites
- Track accessibility score trends over time (not just pass/fail)
Acceptance Criteria Template
For User Stories
Given [context],
When [action by keyboard/mouse/screen reader],
Then [accessible outcome]:
- [ ] Component has an accessible name (via label, aria-label, or aria-labelledby)
- [ ] Component has the correct ARIA role
- [ ] Component is reachable and operable by keyboard (Tab, Enter, Space, Escape, Arrows as appropriate)
- [ ] Focus is visible when the component receives focus
- [ ] State changes are announced to screen readers (aria-expanded, aria-selected, aria-checked, etc.)
- [ ] Error messages are associated with their inputs (aria-describedby or aria-errormessage)
- [ ] Content is readable at 200% zoom without horizontal scrolling
- [ ] Color is not the only means of conveying information
Common Testing Tools
| Tool |
Type |
Best For |
| axe-core / @axe-core/cli |
Automated |
CI/CD integration, broad violation scan |
| Lighthouse |
Automated |
Performance + accessibility combined score |
| WAVE |
Semi-automated |
Visual overlay of accessibility features/issues |
| Accessibility Insights |
Semi-automated |
FastPass (automated) + Assessment (guided manual) |
| pa11y |
Automated |
CI/CD, HTML CodeSniffer rules |
| jest-axe |
Unit test |
Component-level axe scans in jest |
| cypress-axe |
E2E test |
Page-level axe scans in Cypress |
| playwright + @axe-core/playwright |
E2E test |
Page-level axe scans in Playwright |
Screen Reader Testing Quick Reference
NVDA (Windows)
| Key |
Action |
| Insert + Space |
Toggle focus/browse mode |
| Tab |
Move to next focusable element |
| H |
Next heading |
| D |
Next landmark |
| F |
Next form field |
| T |
Next table |
| Insert + F7 |
Elements list (links, headings, landmarks) |
VoiceOver (macOS)
| Key |
Action |
| VO (Ctrl+Option) + Right |
Move to next element |
| VO + Space |
Activate current element |
| VO + U |
Open rotor (navigate by type) |
| VO + Cmd + H |
Next heading |
1---2name: testing-strategy-23description: Accessibility testing decision trees, browser/AT compatibility matrices, manual vs. automated test coverage, regression testing patterns, and acceptance criteria templates for user stories.4metadata:5 source: https://skillmd.com/skills/community-access/testing-strategy-26---7<!-- CANONICAL SOURCE: .github/skills/testing-strategy/SKILL.md -- Edit the canonical source; sync to Gemini via scripts/check-gemini-sync.ps1 -->89# Testing Strategy Skill1011Decision frameworks for accessibility testing — when to use automated tools vs. manual testing, which screen reader + browser combinations to test, and how to write accessibility acceptance criteria.1213---1415## Automated vs. Manual Testing Coverage1617| What Automated Tools Catch (~30-40%) | What Requires Manual Testing (~60-70%) |18|--------------------------------------|---------------------------------------|19| Missing alt text on images | Alt text quality and accuracy |20| Missing form labels | Label clarity and helpfulness |21| Color contrast ratios (computed) | Color contrast in context (gradients, images) |22| Missing lang attribute | Correct language identification |23| Duplicate IDs | Logical reading order |24| Missing ARIA roles on custom widgets | Correct ARIA role for the interaction pattern |25| Heading hierarchy violations | Heading text meaningfulness |26| Empty links and buttons | Link/button text descriptiveness |27| Missing table headers | Table caption and header association quality |28| Syntax errors in ARIA | Screen reader announcement correctness |2930## Browser + Screen Reader Compatibility Matrix3132### Primary Test Combinations (Required)3334| Screen Reader | Browser | OS | Priority |35|--------------|---------|-----|----------|36| NVDA | Firefox | Windows | Must test |37| NVDA | Chrome | Windows | Must test |38| JAWS | Chrome | Windows | Must test |39| VoiceOver | Safari | macOS | Must test |40| VoiceOver | Safari | iOS | Must test |41| TalkBack | Chrome | Android | Should test |4243### Secondary (Nice to Have)4445| Screen Reader | Browser | OS |46|--------------|---------|-----|47| Narrator | Edge | Windows |48| JAWS | Edge | Windows |4950## Testing Decision Tree5152```text53Is it a new component or page?54├── Yes → Full test coverage (automated + manual)55│ ├── Run axe-core / Lighthouse scan56│ ├── Keyboard-only navigation test57│ ├── Screen reader announcement test (NVDA + VoiceOver minimum)58│ └── Visual check at 200% zoom59└── No → What changed?60 ├── Colors/styling → Contrast check + visual review61 ├── Interactive behavior → Keyboard + screen reader test62 ├── Content/text → Screen reader announcement check63 ├── Layout/order → Reading order + focus order test64 └── Dependencies updated → Regression scan (axe-core)65```6667## Regression Testing Patterns6869### CI Pipeline Accessibility Gates70711. **axe-core scan**: Run on every PR. Fail on new critical/serious violations.722. **Baseline management**: Store known issues in `.a11y-baseline.json`. Only fail on **new** issues.733. **Lighthouse score threshold**: Set minimum accessibility score (e.g., 90). Fail on regression.744. **Visual regression**: Capture screenshots at 200% zoom. Compare for focus indicator and layout changes.7576### Preventing Regressions7778- Add accessibility assertions to existing component tests (see `testing-accessibility.instructions.md`)79- Include keyboard navigation in E2E test suites80- Track accessibility score trends over time (not just pass/fail)8182## Acceptance Criteria Template8384### For User Stories8586```text87Given [context],88When [action by keyboard/mouse/screen reader],89Then [accessible outcome]:9091- [ ] Component has an accessible name (via label, aria-label, or aria-labelledby)92- [ ] Component has the correct ARIA role93- [ ] Component is reachable and operable by keyboard (Tab, Enter, Space, Escape, Arrows as appropriate)94- [ ] Focus is visible when the component receives focus95- [ ] State changes are announced to screen readers (aria-expanded, aria-selected, aria-checked, etc.)96- [ ] Error messages are associated with their inputs (aria-describedby or aria-errormessage)97- [ ] Content is readable at 200% zoom without horizontal scrolling98- [ ] Color is not the only means of conveying information99```100101## Common Testing Tools102103| Tool | Type | Best For |104|------|------|----------|105| axe-core / @axe-core/cli | Automated | CI/CD integration, broad violation scan |106| Lighthouse | Automated | Performance + accessibility combined score |107| WAVE | Semi-automated | Visual overlay of accessibility features/issues |108| Accessibility Insights | Semi-automated | FastPass (automated) + Assessment (guided manual) |109| pa11y | Automated | CI/CD, HTML CodeSniffer rules |110| jest-axe | Unit test | Component-level axe scans in jest |111| cypress-axe | E2E test | Page-level axe scans in Cypress |112| playwright + @axe-core/playwright | E2E test | Page-level axe scans in Playwright |113114## Screen Reader Testing Quick Reference115116### NVDA (Windows)117118| Key | Action |119|-----|--------|120| Insert + Space | Toggle focus/browse mode |121| Tab | Move to next focusable element |122| H | Next heading |123| D | Next landmark |124| F | Next form field |125| T | Next table |126| Insert + F7 | Elements list (links, headings, landmarks) |127128### VoiceOver (macOS)129130| Key | Action |131|-----|--------|132| VO (Ctrl+Option) + Right | Move to next element |133| VO + Space | Activate current element |134| VO + U | Open rotor (navigate by type) |135| VO + Cmd + H | Next heading |