Accessibility Testing

Accessibility Testing

Overview

Accessibility testing ensures your application is usable by people with disabilities, including those who rely on screen readers, keyboard navigation, voice control, and other assistive technologies. Automated tools can catch 30-50% of WCAG violations; the rest requires manual testing and judgment.

WCAG Compliance Levels

WCAG 2.2 Key Updates (over 2.1)

• 2.4.11 Focus Not Obscured (Minimum) — Focus indicator not fully hidden by other content.
• 2.4.13 Focus Appearance — Visible focus indicator meets minimum size/contrast.
• 2.5.7 Dragging Movements — Drag operations have non-dragging alternatives.
• 2.5.8 Target Size (Minimum) — Touch targets at least 24x24 CSS pixels.
• 3.3.7 Redundant Entry — Don't ask users to re-enter previously provided info.
• 3.3.8 Accessible Authentication (Minimum) — No cognitive function test for login.

Common Violations (Caught by Automation)

Cross-Platform Tools

axe-core

Overview

axe-core is the industry-standard accessibility testing engine by Deque. It powers most automated a11y tools and can be integrated into any testing framework.

Playwright + axe-core Integration

// tests/a11y/homepage.spec.js
import { test, expect } from "@playwright/test";
import AxeBuilder from "@axe-core/playwright";

test.describe("Homepage accessibility", () => {
    test("should have no WCAG 2.1 AA violations", async ({ page }) => {
        await page.goto("/");

        const results = await new AxeBuilder({ page })
            .withTags(["wcag2a", "wcag2aa", "wcag21aa"])
            .analyze();

        expect(results.violations).toEqual([]);
    });

    test("should have no violations in the navigation", async ({ page }) => {
        await page.goto("/");

        const results = await new AxeBuilder({ page })
            .include("nav")
            .withTags(["wcag2a", "wcag2aa"])
            .analyze();

        expect(results.violations).toEqual([]);
    });

    test("should have no violations after modal opens", async ({ page }) => {
        await page.goto("/");
        await page.click("button#open-modal");
        await page.waitForSelector("[role='dialog']");

        const results = await new AxeBuilder({ page })
            .include("[role='dialog']")
            .analyze();

        expect(results.violations).toEqual([]);
    });
});

axe-core with Detailed Reporting

// tests/a11y/helpers.js
import AxeBuilder from "@axe-core/playwright";

export async function checkA11y(page, context, options = {}) {
    const builder = new AxeBuilder({ page })
        .withTags(options.tags || ["wcag2a", "wcag2aa", "wcag21aa"]);

    if (options.include) builder.include(options.include);
    if (options.exclude) builder.exclude(options.exclude);
    if (options.disableRules) builder.disableRules(options.disableRules);

    const results = await builder.analyze();

    if (results.violations.length > 0) {
        const report = results.violations.map((v) => ({
            id: v.id,
            impact: v.impact,
            description: v.description,
            helpUrl: v.helpUrl,
            nodes: v.nodes.map((n) => ({
                html: n.html,
                target: n.target,
                failureSummary: n.failureSummary,
            })),
        }));

        console.error("Accessibility violations:", JSON.stringify(report, null, 2));
    }

    return results;
}

@axe-core/react (Development Mode)

// src/index.jsx — enable axe-core in development
import React from "react";
import ReactDOM from "react-dom/client";
import App from "./App";

if (process.env.NODE_ENV === "development") {
    import("@axe-core/react").then((axe) => {
        axe.default(React, ReactDOM, 1000);
        // Violations will appear in the browser console
    });
}

const root = ReactDOM.createRoot(document.getElementById("root"));
root.render(<App />);

Pa11y

CLI Usage

# Install Pa11y
npm install -g pa11y

# Run against a URL
pa11y https://example.com

# WCAG 2.1 AA standard (default)
pa11y --standard WCAG2AA https://example.com

# Output as JSON for CI processing
pa11y --reporter json https://example.com > results.json

# Output as JUnit for CI
pa11y --reporter junit https://example.com > results.xml

# Wait for page load / SPA rendering
pa11y --wait 3000 https://example.com

# Run with authentication (execute actions before testing)
pa11y --actions "set field #email to test@example.com" \
      --actions "set field #password to password123" \
      --actions "click element #login-button" \
      --actions "wait for url to be https://example.com/dashboard" \
      https://example.com/login

Pa11y CI Configuration

// .pa11yci.json
{
    "defaults": {
        "standard": "WCAG2AA",
        "timeout": 30000,
        "wait": 2000,
        "chromeLaunchConfig": {
            "args": ["--no-sandbox"]
        }
    },
    "urls": [
        "https://staging.example.com/",
        "https://staging.example.com/about",
        "https://staging.example.com/contact",
        {
            "url": "https://staging.example.com/dashboard",
            "actions": [
                "set field #email to test@example.com",
                "set field #password to password123",
                "click element #login-button",
                "wait for url to be https://staging.example.com/dashboard"
            ]
        },
        {
            "url": "https://staging.example.com/form",
            "ignore": ["WCAG2AA.Principle1.Guideline1_4.1_4_3.G18.Fail"]
        }
    ]
}

Pa11y CI Runner

# Install Pa11y CI
npm install -g pa11y-ci

# Run all configured URLs
pa11y-ci

# Run with custom config path
pa11y-ci --config .pa11yci.json

# Run with JSON reporter
pa11y-ci --reporter json > results.json

GitHub Actions Integration

# .github/workflows/a11y.yml
name: Accessibility Tests
on: [push, pull_request]

jobs:
  a11y-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: npm install -g pa11y-ci
      - run: pa11y-ci --config .pa11yci.json
      - run: pa11y-ci --reporter json > a11y-results.json
        if: always()
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: a11y-results
          path: a11y-results.json

Lighthouse Accessibility Audit

CLI Usage

# Accessibility-only audit
lighthouse https://example.com \
    --only-categories=accessibility \
    --output json,html \
    --output-path ./results/a11y

# With threshold assertion
lighthouse https://example.com \
    --only-categories=accessibility \
    --output json \
    --output-path ./results/a11y.json
# Then parse JSON to check score >= 90

Lighthouse CI with Accessibility Assertions

# lighthouserc.yml
ci:
  collect:
    url:
      - https://staging.example.com/
      - https://staging.example.com/about
    numberOfRuns: 3
  assert:
    assertions:
      categories:accessibility:
        - error
        - minScore: 0.9    # Fail if accessibility score < 90
      aria-allowed-attr: "error"
      color-contrast: "warn"
      image-alt: "error"
      label: "error"

Storybook addon-a11y

Installation and Configuration

# Install the addon
npm install --save-dev @storybook/addon-a11y

// .storybook/main.js
export default {
    addons: [
        "@storybook/addon-a11y",
        // ... other addons
    ],
};

Component Story with a11y Checks

// src/components/Button/Button.stories.jsx
import { Button } from "./Button";

export default {
    title: "Components/Button",
    component: Button,
    parameters: {
        a11y: {
            // axe-core configuration
            config: {
                rules: [
                    { id: "color-contrast", enabled: true },
                    { id: "button-name", enabled: true },
                ],
            },
        },
    },
};

export const Primary = {
    args: {
        variant: "primary",
        children: "Click me",
    },
};

export const IconOnly = {
    args: {
        variant: "icon",
        "aria-label": "Close dialog",
        children: <CloseIcon />,
    },
};

// Disable a11y for a specific story (use sparingly)
export const Decorative = {
    args: {
        variant: "decorative",
        children: "Decorative element",
    },
    parameters: {
        a11y: { disable: true },
    },
};

Storybook Test Runner with a11y

// .storybook/test-runner.js
const { injectAxe, checkA11y } = require("axe-playwright");

module.exports = {
    async preVisit(page) {
        await injectAxe(page);
    },
    async postVisit(page) {
        await checkA11y(page, "#storybook-root", {
            detailedReport: true,
            detailedReportOptions: {
                html: true,
            },
        });
    },
};

# Run Storybook test runner with a11y checks
npx test-storybook

Automated vs Manual Testing

What Automation Catches (~30-50% of WCAG Issues)

What Requires Manual Testing (~50-70%)

Manual Testing Checklist

## Manual Accessibility Audit
- [ ] Tab through entire page — logical order, no traps
- [ ] All interactive elements reachable by keyboard alone
- [ ] Focus indicator visible on every focusable element
- [ ] Skip navigation link works and is first focusable element
- [ ] Screen reader reads page in logical order (test with NVDA/VoiceOver)
- [ ] Dynamic content changes are announced (aria-live regions)
- [ ] Modal focus is trapped and returns on close
- [ ] Form errors are announced and linked to fields
- [ ] Page is usable at 200% zoom (no horizontal scrolling)
- [ ] Page is usable at 400% zoom (content reflows)
- [ ] All functionality works without color as the only indicator
- [ ] Animations can be paused (prefers-reduced-motion respected)
- [ ] Touch targets are at least 24x24 CSS pixels

Best Practices

General

• Run automated a11y tests on every PR — they are fast and catch regressions.
• Treat a11y violations like bugs, not warnings — fix them before merging.
• Test with real assistive technology at least once per release (NVDA on Windows, VoiceOver on macOS).
• Include people with disabilities in user testing when possible.

axe-core

• Use withTags(["wcag2a", "wcag2aa", "wcag21aa"]) for standard WCAG 2.1 AA coverage.
• Use include() / exclude() to scope checks to specific page regions.
• Test pages in multiple states (empty, loaded, error, modal open).
• Use @axe-core/react during development to catch issues before they reach tests.

Pa11y

• Configure a .pa11yci.json with all critical URLs for consistent CI runs.
• Use actions for authenticated pages or SPAs that need user interaction before testing.
• Use ignore sparingly and document why each rule is ignored.

Storybook addon-a11y

• Enable addon-a11y for all stories by default — disable only with documented justification.
• Use the Storybook test runner with axe-playwright for CI enforcement.
• Test components in isolation and in composed layouts — a11y issues can emerge from composition.

CI Integration

• Fail the build on critical violations (missing alt text, missing labels, no keyboard access).
• Warn on moderate violations (contrast, heading order) to avoid blocking but track debt.
• Generate reports as CI artifacts for audit trails and compliance documentation.
• Combine automated testing (axe-core in Playwright) with periodic manual audits.