Visual Testing Advanced

Advanced Visual Testing

Quick Visual Testing Steps:

• Set up baseline on main branch
• Compare feature branch against baseline
• Mask dynamic elements (timestamps, avatars)
• Use AI-powered comparison to reduce false positives
• Integrate in CI/CD to block visual regressions

Critical Success Factors:

• Functional tests don't catch visual bugs
• AI-powered tools reduce false positives
• Review diffs, don't just auto-approve

Quick Reference Card

When to Use

• UI component changes
• CSS/styling modifications
• Responsive design validation
• Cross-browser consistency checks

Visual Bug Types

Comparison Algorithms

PRIMARY PATH: qe-browser visual-diff

Most visual regression work should go through the qe-browser fleet skill. It wraps Vibium (WebDriver BiDi) and provides pixel-diff against stored baselines with threshold enforcement and diff-image output. See .claude/skills/qe-browser/SKILL.md.

# Navigate
vibium go https://example.com
vibium wait load

# First run — creates baseline in .aqe/visual-baselines/homepage.png
node .claude/skills/qe-browser/scripts/visual-diff.js --name homepage

# Subsequent runs — compare, non-zero exit on mismatch
node .claude/skills/qe-browser/scripts/visual-diff.js --name homepage --threshold 0.02

# Scope to a single region
node .claude/skills/qe-browser/scripts/visual-diff.js --name hero --selector "#hero"

# Responsive — run diff at each breakpoint
for viewport in "375 667" "768 1024" "1920 1080"; do
  read w h <<< "$viewport"
  vibium viewport $w $h
  node .claude/skills/qe-browser/scripts/visual-diff.js --name "homepage-${w}x${h}"
done

# Reset baseline after an intentional design change
node .claude/skills/qe-browser/scripts/visual-diff.js --name homepage --update-baseline

Baselines live in .aqe/visual-baselines/. The script uses pixelmatch when installed, with a hash-based exact-match fallback otherwise. Non-zero exit when similarity < 1 - threshold, so CI gating is $?-based.

When to keep Playwright visual regression

Use the Playwright recipe below only when you need:

• AI semantic comparison (Percy, Applitools) to ignore insignificant pixel drift
• Cross-browser rendering checks in Firefox/WebKit (Vibium is Chrome-only today)
• Tight integration with an existing Playwright test suite

LEGACY: Visual Regression with Playwright (fallback)

import { test, expect } from '@playwright/test';

test('homepage visual regression', async ({ page }) => {
  await page.goto('https://example.com');

  // Capture and compare screenshot
  await expect(page).toHaveScreenshot('homepage.png');
  // First run: saves baseline
  // Subsequent runs: compares to baseline
});

test('responsive design', async ({ page }) => {
  // Mobile viewport
  await page.setViewportSize({ width: 375, height: 667 });
  await page.goto('https://example.com');
  await expect(page).toHaveScreenshot('homepage-mobile.png');

  // Tablet viewport
  await page.setViewportSize({ width: 768, height: 1024 });
  await expect(page).toHaveScreenshot('homepage-tablet.png');
});

Handling Dynamic Content

test('mask dynamic elements', async ({ page }) => {
  await page.goto('https://example.com');

  await expect(page).toHaveScreenshot({
    mask: [
      page.locator('.timestamp'),     // Dynamic time
      page.locator('.user-count'),    // Live counter
      page.locator('.advertisement'), // Ads
      page.locator('.avatar')         // User avatars
    ]
  });
});

AI-Powered Visual Testing (Percy)

import percySnapshot from '@percy/playwright';

test('AI-powered visual test', async ({ page }) => {
  await page.goto('https://example.com');

  // Percy uses AI to ignore anti-aliasing, minor font differences
  await percySnapshot(page, 'Homepage');
});

test('component visual test', async ({ page }) => {
  await page.goto('https://example.com/components');

  // Snapshot specific component
  const button = page.locator('.primary-button');
  await percySnapshot(page, 'Primary Button', {
    scope: button
  });
});

Playwright Configuration

// playwright.config.js
export default {
  expect: {
    toHaveScreenshot: {
      maxDiffPixels: 100,      // Allow 100 pixel difference
      maxDiffPixelRatio: 0.01, // Or 1% of image
      threshold: 0.2,          // Color similarity threshold
      animations: 'disabled',  // Disable animations
      caret: 'hide'            // Hide cursor
    }
  }
};

Agent-Driven Visual Testing

// Comprehensive visual regression
await Task("Visual Regression Suite", {
  baseline: 'main-branch',
  current: 'feature-branch',
  pages: ['homepage', 'product', 'checkout'],
  devices: ['desktop', 'tablet', 'mobile'],
  browsers: ['chrome', 'firefox', 'safari'],
  threshold: 0.01
}, "qe-visual-tester");

// Returns:
// {
//   comparisons: 27,  // 3 pages × 3 devices × 3 browsers
//   differences: 2,
//   report: 'visual-regression-report.html'
// }

Agent Coordination Hints

Memory Namespace

aqe/visual-testing/
├── baselines/*          - Baseline screenshots
├── comparisons/*        - Diff results
├── components/*         - Component snapshots
└── reports/*            - Visual regression reports

Fleet Coordination

const visualFleet = await FleetManager.coordinate({
  strategy: 'visual-testing',
  agents: [
    'qe-visual-tester',   // Screenshot comparison
    'qe-test-executor',   // Cross-browser execution
    'qe-quality-gate'     // Block on visual regressions
  ],
  topology: 'parallel'
});

Related Skills

• accessibility-testing - Visual a11y checks
• compatibility-testing - Cross-browser visuals
• regression-testing - Regression suite

Remember

Functional tests don't catch visual bugs. Layout shifts, color changes, font rendering, alignment issues - all invisible to functional tests but visible to users.

AI-powered tools reduce false positives. Percy, Applitools use AI to ignore insignificant differences (anti-aliasing, minor font rendering).

With Agents: qe-visual-tester automates visual regression across browsers and devices, uses AI to filter noise, and generates visual diff reports. Catches UI regressions before users see them.