QA Visual Testing

Visual Testing with E2E Tests

"Visual validation catches bugs that functional tests miss. Write E2E tests with screenshot comparison and Vision MCP helpers."

When to Use This Skill

Use for every game feature validation to create E2E tests that:

• Compare screenshots against baseline images using Playwright API
• Detect game states (menu, playing, game over, win) using Vision MCP helpers
• Validate UI elements (HUD, health bars, buttons) programmatically
• Verify visual appearance matches design specifications (GDD) using Vision MCP helpers

MANDATORY: Port Detection Before Browser Testing

⚠️ CRITICAL: Vite dev server may run on different ports (3000, 3001, 5174, 8080, etc.)

Before ANY browser interaction, ALWAYS detect the correct port:

# Method 1: Check listening ports
netstat -an | grep LISTEN | grep -E ":(3000|3001|5174|8080)"

# Method 2: Try curl to detect Vite
curl -s http://localhost:3000 | grep -q "vite" && echo "PORT=3000" || \
curl -s http://localhost:3001 | grep -q "vite" && echo "PORT=3001" || \
curl -s http://localhost:5174 | grep -q "vite" && echo "PORT=5174"

NOTE: E2E tests configured in playwright.config.ts use baseURL: 'http://localhost:3000' and the webServer configuration automatically starts the dev server on the correct port.

For manual testing or MCP validation, detect the port first and use http://localhost:{detectedPort}.

Core Principle: Write E2E Tests, Use Vision MCP Helpers

✅ CORRECT APPROACH:

// Write E2E test with screenshot comparison - YES!
test('visual regression check', async ({ page }) => {
  await page.goto('http://localhost:3000');
  await page.waitForTimeout(2000);

  // Screenshot comparison for regression
  await expect(page).toHaveScreenshot('baseline.png', {
    maxDiffPixelRatio: 0.01,
  });
});

✅ FOR QUALITATIVE ANALYSIS:

// Use Vision MCP helper for GDD compliance - YES!
test('shader quality meets GDD standards', async ({ page }) => {
  await page.goto('http://localhost:3000');
  await page.waitForTimeout(2000);

  // Screenshot for analysis
  const screenshot = await page.screenshot();

  // Use helper function for Vision MCP analysis
  const analysis = await analyzeVisualQuality(
    screenshot,
    'Shader material quality, GDD compliance'
  );
  expect(analysis.passes).toBe(true);
});

❌ DO NOT USE:

// Interactive MCP - NO!
mcp__playwright__browser_navigate('http://localhost:3000');
mcp__4_5v_mcp__analyze_image({ imageSource: 'screenshot.png' });

Quick Start

import { test, expect } from '@playwright/test';

// 1. Screenshot comparison test (quantitative)
test('ui matches baseline', async ({ page }) => {
  await page.goto('http://localhost:3000');
  await expect(page).toHaveScreenshot('ui-baseline.png');
});

// 2. Vision MCP helper test (qualitative)
test('shader meets GDD standards', async ({ page }) => {
  await page.goto('http://localhost:3000');
  const screenshot = await page.screenshot();

  const result = await checkGDDCompliance(screenshot, 'Material should be metallic blue');
  expect(result.compliant).toBe(true);
});

Vision MCP Helper Functions

Create helper functions in tests/helpers/visual-analysis.ts:

// tests/helpers/visual-analysis.ts

/**
 * Analyze visual quality using Vision MCP
 * @param screenshot - Buffer or path to screenshot
 * @param criteria - Description of what to check
 * @returns Analysis result with passes/notes
 */
export async function analyzeVisualQuality(
  screenshot: Buffer | string,
  criteria: string
): Promise<{ passes: boolean; notes: string[] }> {
  // This helper would use Vision MCP for qualitative analysis
  // The actual MCP call happens outside the test
  // Test files import and use this helper

  // For now, return a placeholder
  // In production, this would save the screenshot and trigger Vision MCP analysis
  return {
    passes: true,
    notes: ['Analysis pending - run Vision MCP separately'],
  };
}

/**
 * Check GDD compliance using Vision MCP
 */
export async function checkGDDCompliance(
  screenshot: Buffer | string,
  gddDescription: string
): Promise<{ compliant: boolean; deviations: string[] }> {
  // Vision MCP analysis for GDD compliance
  return {
    compliant: true,
    deviations: [],
  };
}

/**
 * Detect game state using Vision MCP
 */
export async function detectGameState(
  screenshot: Buffer | string
): Promise<{ state: string; uiElements: string[]; playerVisible: boolean }> {
  // Vision MCP analysis for game state detection
  return {
    state: 'playing',
    uiElements: ['hud', 'healthBar'],
    playerVisible: true,
  };
}

Screenshot Comparison Tests (Quantitative)

Basic Screenshot Test

test('visual appearance matches baseline', async ({ page }) => {
  await page.goto('http://localhost:3000');
  await page.waitForSelector('canvas');
  await page.waitForTimeout(2000); // Wait for scene to stabilize

  // Compare with baseline
  await expect(page).toHaveScreenshot('baseline.png', {
    maxDiffPixelRatio: 0.01,
  });
});

Tolerance Guidelines

Multi-State Screenshot Tests

test.describe('Game State Visual Regression', () => {
  test('menu state matches baseline', async ({ page }) => {
    await page.goto('http://localhost:3000');
    await page.waitForTimeout(1000);

    await expect(page).toHaveScreenshot('menu-baseline.png');
  });

  test('playing state matches baseline', async ({ page }) => {
    await page.goto('http://localhost:3000');
    await page.click('canvas');
    await page.waitForTimeout(1000);

    await expect(page).toHaveScreenshot('playing-baseline.png', {
      maxDiffPixels: 5000, // Allow for animation variation
    });
  });
});

Game State Detection Tests (Qualitative)

State Detection with Vision Helper

test('detect game playing state', async ({ page }) => {
  await page.goto('http://localhost:3000');
  await page.click('canvas');
  await page.waitForTimeout(1000);

  // Capture screenshot
  const screenshot = await page.screenshot();

  // Use helper for Vision MCP analysis
  const state = await detectGameState(screenshot);

  expect(state.state).toBe('playing');
  expect(state.playerVisible).toBe(true);
  expect(state.uiElements).toContain('hud');
});

State-Specific Validation

test('menu state has required elements', async ({ page }) => {
  await page.goto('http://localhost:3000');
  await page.waitForTimeout(1000);

  const screenshot = await page.screenshot();

  // Vision MCP would check for:
  // - Game title visible
  // - Menu buttons present
  // - No gameplay elements

  // For E2E, check programmatically:
  await expect(page.getByRole('button', { name: /play/i })).toBeVisible();
  await expect(page.getByRole('button', { name: /settings/i })).toBeVisible();
});

UI Element Validation Tests

HUD Detection (Programmatic + Vision MCP)

test('HUD elements are visible', async ({ page }) => {
  await page.goto('http://localhost:3000');
  await page.click('canvas');
  await page.waitForTimeout(1000);

  // Programmatic checks
  await expect(page.getByTestId('health-bar')).toBeVisible();
  await expect(page.getByTestId('score')).toBeVisible();
  await expect(page.getByTestId('minimap')).toBeVisible();

  // Vision MCP for qualitative assessment
  const screenshot = await page.screenshot();
  const analysis = await analyzeVisualQuality(
    screenshot,
    'HUD elements properly styled and positioned'
  );
  expect(analysis.passes).toBe(true);
});

Button Detection Tests

test('menu buttons are present and enabled', async ({ page }) => {
  await page.goto('http://localhost:3000');

  // Programmatic checks
  const playButton = page.getByRole('button', { name: /play/i });
  await expect(playButton).toBeVisible();
  await expect(playButton).toBeEnabled();

  const settingsButton = page.getByRole('button', { name: /settings/i });
  await expect(settingsButton).toBeVisible();
});

3D Asset Visual Regression Tests

Multi-Angle Shader Validation

test('terrain shader from multiple angles', async ({ page }) => {
  await page.goto('http://localhost:3000');
  await page.click('canvas');
  await page.waitForTimeout(2000);

  const cameraPositions = [
    { name: 'front', pos: [0, 10, 20], target: [0, 0, 0] },
    { name: 'side', pos: [20, 10, 0], target: [0, 0, 0] },
    { name: 'top-down', pos: [0, 30, 5], target: [0, 0, 0] },
    { name: 'iso', pos: [15, 15, 15], target: [0, 0, 0] },
  ];

  for (const angle of cameraPositions) {
    // Position camera
    await page.evaluate(
      (pos, target) => {
        (window as any).gameCamera?.position.set(...pos);
        (window as any).gameCamera?.lookAt(...target);
      },
      angle.pos,
      angle.target
    );

    await page.waitForTimeout(300); // Let render settle

    // Screenshot comparison
    await expect(page).toHaveScreenshot(`terrain-${angle.name}.png`, {
      maxDiffPixels: 500,
      threshold: 0.02,
    });
  }
});

Paint Projectile Visual Test

test('paint projectile visual validation', async ({ page }) => {
  await page.goto('http://localhost:3000');
  await page.click('canvas');
  await page.waitForTimeout(1000);

  // Shoot to create projectile
  await page.mouse.click(400, 300);
  await page.waitForTimeout(100); // During flight

  // Screenshot comparison
  await expect(page).toHaveScreenshot('projectile-flight.png', {
    maxDiffPixels: 2000, // Allow for projectile animation
  });

  // Vision MCP for qualitative check
  const screenshot = await page.screenshot();
  const analysis = await analyzeVisualQuality(
    screenshot,
    'Paint projectile has team color, glow, and trail effect'
  );
  expect(analysis.passes).toBe(true);
});

Shader Visual Regression Tests

Terrain Shader Test

test('terrain shader visual regression', async ({ page }) => {
  await page.goto('http://localhost:3000');
  await page.click('canvas');
  await page.waitForTimeout(2000);

  // Position camera for consistent view
  await page.evaluate(() => {
    (window as any).gameCamera?.position.set(0, 10, 20);
    (window as any).gameCamera?.lookAt(0, 0, 0);
  });

  await page.waitForTimeout(500);

  // Screenshot comparison
  await expect(page).toHaveScreenshot('terrain-shader-baseline.png', {
    maxDiffPixels: 500,
    threshold: 0.02,
  });
});

Paint Overlay Test

test('terrain paint overlay visibility', async ({ page }) => {
  await page.goto('http://localhost:3000');
  await page.click('canvas');
  await page.waitForTimeout(1000);

  // Position camera
  await page.evaluate(() => {
    (window as any).gameCamera?.position.set(0, 10, 15);
    (window as any).gameCamera?.lookAt(0, 0, 0);
  });

  // Screenshot before paint
  await expect(page).toHaveScreenshot('terrain-before-paint.png');

  // Shoot to create paint
  await page.mouse.click(400, 300);
  await page.waitForTimeout(500);

  // Screenshot after paint - Vision MCP for qualitative
  await expect(page).toHaveScreenshot('terrain-after-paint.png', {
    maxDiffPixels: 2000, // Allow for new paint
  });

  // Verify paint visible with Vision MCP helper
  const screenshot = await page.screenshot();
  const analysis = await analyzeVisualQuality(
    screenshot,
    'Paint splat visible on terrain with correct team color'
  );
  expect(analysis.passes).toBe(true);
});

GDD Compliance Validation Tests

Shader Quality vs GDD

test('shader meets GDD specifications', async ({ page }) => {
  await page.goto('http://localhost:3000');
  await page.click('canvas');
  await page.waitForTimeout(2000);

  // Take screenshot for analysis
  const screenshot = await page.screenshot();

  // Vision MCP helper for GDD compliance
  const gddSpec = `
    - Terrain should use raymarching SDF shader
    - Paint should appear as wet/glossy surface
    - Team colors: orange (team 1) and blue (team 2)
    - No visible shader artifacts (NaN pixels, seams)
  `;

  const result = await checkGDDCompliance(screenshot, gddSpec);

  expect(result.compliant).toBe(true);
  expect(result.deviations).toHaveLength(0);
});

Character Model GDD Validation

test('character model matches GDD', async ({ page }) => {
  await page.goto('http://localhost:3000');
  await completeCharacterSelection(page, 'knight');
  await page.waitForTimeout(2000);

  const screenshot = await page.screenshot();

  const gddSpec = `
    - Character: Knight in silver armor with blue cape
    - Should hold a broadsword
    - Properly scaled relative to environment
    - Textured (not default gray material)
  `;

  const result = await checkGDDCompliance(screenshot, gddSpec);

  expect(result.compliant).toBe(true);
});

Color Mode / Accessibility Tests

Color Blind Mode Screenshot Tests

const colorModes = ['default', 'protanopia', 'deuteranopia', 'tritanopia', 'high_contrast'];

test.describe('Color Mode Visual Regression', () => {
  colorModes.forEach((mode) => {
    test(`renders ${mode} color mode correctly`, async ({ page }) => {
      // Set color mode
      await page.goto('http://localhost:3000');
      await page.evaluate((m) => {
        localStorage.setItem(
          'project-chroma-accessibility',
          JSON.stringify({
            hasCompletedFirstLaunch: true,
            colorMode: m,
          })
        );
      }, mode);
      await page.reload();

      // Navigate to lobby
      await page.fill('#characterName', 'TestPlayer');
      await page.locator('button:has-text("Select Character")').first().click();
      await page.waitForURL('**/lobby', { timeout: 10000 });

      await page.waitForLoadState('networkidle');

      // Screenshot comparison
      await expect(page).toHaveScreenshot(`lobby-${mode}.png`, {
        maxDiffPixels: 100,
      });
    });
  });
});

Complete Visual Test Example

import { test, expect } from '@playwright/test';
import { analyzeVisualQuality, checkGDDCompliance } from '@/helpers/visual-analysis';

test.describe('Visual Validation - feat-001', () => {
  test('ui matches baseline', async ({ page }) => {
    await page.goto('http://localhost:3000');
    await expect(page).toHaveScreenshot('menu-baseline.png');
  });

  test('gameplay state detected correctly', async ({ page }) => {
    await page.goto('http://localhost:3000');
    await page.click('canvas');
    await page.waitForTimeout(1000);

    const screenshot = await page.screenshot();
    const state = await detectGameState(screenshot);

    expect(state.state).toBe('playing');
    expect(state.playerVisible).toBe(true);
  });

  test('shader meets GDD standards', async ({ page }) => {
    await page.goto('http://localhost:3000');
    await page.click('canvas');
    await page.waitForTimeout(2000);

    const screenshot = await page.screenshot();
    const gddSpec = 'Raymarching terrain with wet paint appearance';

    const result = await checkGDDCompliance(screenshot, gddSpec);
    expect(result.compliant).toBe(true);
  });

  test('visual quality passes standards', async ({ page }) => {
    await page.goto('http://localhost:3000');
    await page.click('canvas');
    await page.waitForTimeout(2000);

    const screenshot = await page.screenshot();
    const analysis = await analyzeVisualQuality(
      screenshot,
      'Material quality, shader effects, GDD compliance'
    );

    expect(analysis.passes).toBe(true);
    expect(analysis.notes.filter((n) => n.includes('FAIL'))).toHaveLength(0);
  });
});

Running Visual Tests

# Run all visual tests
npm run test:e2e -- tests/e2e/visual-suite.spec.ts

# Update baselines
npx playwright test --update-snapshots

# Run in headed mode
npm run test:e2e -- --headed

# Run specific test
npm run test:e2e -- -g "shader meets GDD"

Testing Checklist

For each visual validation:

• E2E test file created in tests/e2e/
• Screenshot comparison tests written (quantitative)
• Vision MCP helper tests written for GDD compliance (qualitative)
• Baselines committed to repository
• Tests run locally: npm run test:e2e
• Vision MCP analysis passes for qualitative criteria
• No visual glitches detected
• Deviations documented with severity

Server Management

⚠️ CRITICAL: Use shared-lifecycle skill for server management.

Server Detection (Before Visual Tests)

⚠️ IMPORTANT: Playwright's webServer config manages servers for E2E tests automatically.

When running npm run test:e2e, Playwright automatically starts:

• npm run dev (port 3000) with reuseExistingServer: !process.env.CI

DO NOT manually start servers for E2E tests.

Server Check Pattern

# Check if dev server is running (port 3000)
netstat -an | grep :3000 || lsof -i :3000

# Alternative: Try curl to detect Vite
curl -s http://localhost:3000 | grep -q "vite" && echo "RUNNING" || echo "NOT_RUNNING"

E2E Test Path (Standard Visual Validation)

# Playwright handles server lifecycle via webServer config
npm run test:e2e -- tests/e2e/visual-suite.spec.ts

# NO manual server start needed
# NO manual cleanup needed - Playwright handles it

Manual MCP Validation Path (Only when explicitly needed)

# Only for manual MCP validation (NOT E2E tests)
# Check port 3000 first
if ! netstat -an | grep :3000; then
  # Start server in background
  Bash(command="npm run dev", run_in_background=true)
  # Capture shell_id for cleanup: { shell_id: "abc123" }
fi

# After validation completes:
TaskStop(task_id="abc123")  # MANDATORY cleanup

See also: shared-lifecycle skill for complete process management patterns.

Anti-Patterns

❌ DON'T:

• Use Playwright MCP directly during test execution
• Skip baseline creation
• Ignore Vision MCP qualitative analysis for GDD compliance
• Use hardcoded waits when assertions work
• Commit without visual tests

✅ DO:

• Write E2E tests with screenshot comparison
• Use Vision MCP helper functions for qualitative analysis
• Create baselines for all visual states
• Use appropriate tolerance for GPU variation
• Commit visual tests with implementation

References

• qa-e2e-test-creation/SKILL.md - Full E2E test patterns
• Playwright Screenshot API
• Playwright Visual Testing
• tests/helpers/visual-analysis.ts - Vision MCP helpers

UI Visual Regression Testing (Added: ui-001 Playtest)

Learned from ui-001: Visual quality requires dedicated testing beyond functional validation.

UI Design System Validation Tests

test.describe('UI Design System Compliance', () => {
  test('16:9 aspect ratio is enforced', async ({ page }) => {
    await page.goto('http://localhost:3000');
    await page.waitForTimeout(1000);

    // Check for aspect ratio container
    const aspectContainer = page.locator('[style*="aspect-ratio"]');
    await expect(aspectContainer).toBeVisible();

    // Verify centered positioning
    const container = await aspectContainer.boundingBox();
    const viewport = page.viewportSize();

    if (container && viewport) {
      // Container should be centered
      const centerX = viewport.width / 2;
      const containerCenterX = container.x + container.width / 2;
      expect(Math.abs(centerX - containerCenterX)).toBeLessThan(5);
    }
  });

  test('design system tokens are applied', async ({ page }) => {
    await page.goto('http://localhost:3000');

    // Check for custom fonts
    const fonts = await page.evaluate(() => {
      const styles = window.getComputedStyle(document.body);
      return {
        displayFont: styles.getPropertyValue('--font-display'),
        uiFont: styles.getPropertyValue('--font-ui'),
      };
    });

    expect(fonts.displayFont).toBeTruthy();
    expect(fonts.uiFont).toBeTruthy();
  });

  test('buttons have metallic styling', async ({ page }) => {
    await page.goto('http://localhost:3000');
    const button = page.getByRole('button').first();

    // Check for gradient background
    const background = await button.evaluate((el) => {
      return window.getComputedStyle(el).background;
    });

    expect(background).toContain('gradient');
  });

  test('hover states provide visual feedback', async ({ page }) => {
    await page.goto('http://localhost:3000');
    const button = page.getByRole('button').first();

    // Screenshot before hover
    await expect(page).toHaveScreenshot('button-before-hover.png');

    // Hover and capture
    await button.hover();
    await page.waitForTimeout(150);

    // Screenshot after hover - check for glow/transform
    await expect(page).toHaveScreenshot('button-after-hover.png', {
      maxDiffPixels: 500, // Allow for animation timing
    });
  });

  test('screen transitions use custom easing', async ({ page }) => {
    await page.goto('http://localhost:3000');

    // Click to trigger transition
    await page.getByRole('button', { name: /main menu/i }).click();

    // Capture mid-transition (challenging but useful)
    await page.waitForTimeout(100);
    await page.screenshot({ path: 'mid-transition.png' });

    // Wait for completion
    await page.waitForTimeout(400);
    await expect(page).toHaveScreenshot('after-transition.png');
  });
});

Multi-Resolution Testing

const resolutions = [
  { name: '1080p', width: 1920, height: 1080 },
  { name: '720p', width: 1280, height: 720 },
  { name: '1440p', width: 2560, height: 1440 },
  { name: 'ultrawide', width: 3440, height: 1440 },
];

test.describe('UI Scaling Across Resolutions', () => {
  resolutions.forEach((res) => {
    test(`UI scales correctly at ${res.name}`, async ({ page }) => {
      await page.setViewportSize({ width: res.width, height: res.height });
      await page.goto('http://localhost:3000');
      await page.waitForTimeout(1000);

      // UI should remain centered and proportional
      await expect(page).toHaveScreenshot(`ui-${res.name}.png`);
    });
  });
});

Visual Quality Checklist for QA

Before passing visual validation:

• Aspect Ratio - 16:9 enforced with letterbox
• Scaling - UI scales proportionally
• Typography - Gaming fonts applied
• Buttons - Metallic styling, hover/active states
• Transitions - Custom easing curves
• Colors - Theme palette used throughout
• Spacing - Consistent grid/spacing system
• Accessibility - WCAG AA contrast minimum
• Performance - 60+ FPS with animations
• E2E Tests - All UI paths covered

Converted and distributed by TomeVault — claim your Tome and manage your conversions.