UI Interaction Analyzer
This skill provides advanced capabilities for testing and analyzing user interface interactions in web applications through automated browser instrumentation.
Overview
The UI Interaction Analyzer enables comprehensive testing of frontend user experiences, interaction patterns, and behavioral workflows. Built on Playwright automation, it supports validating complex user journeys, capturing interaction states, and debugging UI behavior in real browser environments.
When to Use This Skill
Deploy this skill when you need to:
- Validate complex user interaction flows across multiple pages
- Test dynamic UI behavior and state transitions
- Capture and analyze interaction patterns for UX optimization
- Debug frontend issues in real browser environments
- Verify accessibility compliance through automated interaction testing
- Generate visual regression baselines for UI components
- Analyze form validation and error handling behavior
Prerequisites
- Node.js runtime (v16 or higher recommended)
- Active web application instance (local or remote URL)
- Playwright browser automation library (auto-installed if missing)
- Sufficient disk space for screenshot and trace artifacts
Core Capabilities
1. Interaction Recording
- Capture complete user journey traces
- Record click, input, and navigation events
- Generate replay-able interaction scripts
- Track state changes across interactions
2. Behavioral Validation
- Assert expected UI responses to user actions
- Verify element state transitions (visible, hidden, enabled, disabled)
- Validate conditional rendering based on user input
- Test dynamic content updates and real-time features
3. Visual Analysis
- Capture screenshots at interaction checkpoints
- Compare visual states across test runs
- Identify unintended UI changes
- Generate visual documentation of user flows
4. Accessibility Testing
- Verify keyboard navigation paths
- Test screen reader compatibility
- Validate ARIA attributes during interactions
- Check focus management across UI transitions
Usage Examples
Example 1: Multi-Step Form Validation
// Test a complex checkout flow
await page.goto('http://localhost:3000/checkout');
// Step 1: Fill shipping information
await page.fill('[data-testid="shipping-name"]', 'John Doe');
await page.fill('[data-testid="shipping-address"]', '123 Main St');
await page.click('[data-testid="continue-to-payment"]');
// Step 2: Verify navigation to payment
await page.waitForURL('**/checkout/payment');
await page.screenshot({ path: 'payment-step.png' });
// Step 3: Complete payment form
await page.fill('[data-testid="card-number"]', '4242424242424242');
await page.click('[data-testid="submit-payment"]');
// Verify success state
await expect(page.locator('[data-testid="order-confirmation"]')).toBeVisible();
Example 2: Dynamic Content Interaction
// Test autocomplete behavior
await page.fill('[data-testid="search-input"]', 'javasc');
await page.waitForSelector('[data-testid="autocomplete-dropdown"]');
const suggestions = await page.locator('[data-testid="suggestion-item"]').count();
console.log(`Found ${suggestions} autocomplete suggestions`);
await page.click('[data-testid="suggestion-item"]', { nth: 0 });
await expect(page.locator('[data-testid="search-input"]')).toHaveValue('javascript');
Example 3: Error State Validation
// Test form validation errors
await page.click('[data-testid="submit-button"]'); // Submit empty form
// Verify error messages appear
await expect(page.locator('[data-testid="email-error"]')).toContainText('Email is required');
await expect(page.locator('[data-testid="password-error"]')).toContainText('Password is required');
// Capture error state
await page.screenshot({ path: 'validation-errors.png' });
Guidelines
- Establish clear interaction objectives - Define expected outcomes before testing complex flows
- Implement robust wait strategies - Use appropriate wait conditions for dynamic content (networkidle, domcontentloaded, specific selectors)
- Capture interaction traces - Enable Playwright tracing for complex scenarios to facilitate debugging
- Use semantic selectors - Prioritize data-testid, role-based, and accessible selectors over fragile CSS paths
- Test error conditions - Verify both happy paths and failure scenarios in user interactions
- Maintain test isolation - Reset application state between test runs to ensure reproducibility
- Document interaction assumptions - Record expected UI behavior and business logic constraints
- Monitor performance - Track interaction timing to identify slow UI responses
- Handle asynchronous updates - Account for delayed state changes in modern reactive frameworks
- Validate across viewports - Test critical interactions on mobile, tablet, and desktop breakpoints
Common Patterns
Pattern: Wait for Network Idle
await page.goto('http://localhost:3000/dashboard', {
waitUntil: 'networkidle'
});
Pattern: Conditional Interaction
const modalVisible = await page.locator('[data-testid="modal"]').isVisible();
if (modalVisible) {
await page.click('[data-testid="modal-close"]');
}
Pattern: Retry Failed Interactions
async function retryClick(selector, maxAttempts = 3) {
for (let i = 0; i < maxAttempts; i++) {
try {
await page.click(selector, { timeout: 5000 });
return;
} catch (error) {
if (i === maxAttempts - 1) throw error;
await page.waitForTimeout(1000);
}
}
}
Pattern: Extract Interaction Metrics
const startTime = Date.now();
await page.click('[data-testid="load-data"]');
await page.waitForSelector('[data-testid="data-loaded"]');
const loadTime = Date.now() - startTime;
console.log(`Data loaded in ${loadTime}ms`);
Advanced Configuration
Tracing
Enable detailed interaction tracing for debugging:
await context.tracing.start({ screenshots: true, snapshots: true });
// ... perform interactions
await context.tracing.stop({ path: 'trace.zip' });
Custom Timeouts
Configure interaction-specific timeout values:
// Set global timeout
page.setDefaultTimeout(10000);
// Override for specific action
await page.click('[data-testid="slow-button"]', { timeout: 30000 });
Viewport Testing
Test interactions across different screen sizes:
await page.setViewportSize({ width: 375, height: 667 }); // Mobile
await page.setViewportSize({ width: 1920, height: 1080 }); // Desktop
Limitations
- Requires Node.js runtime environment
- Cannot test native mobile applications (use platform-specific tools)
- Complex authentication flows may require manual session handling
- Some JavaScript-heavy SPAs may need framework-specific configuration
- Cross-domain interactions may be restricted by browser security policies
- File upload/download testing requires additional system permissions