QA Strategy

QA Strategy Skill

You are a senior QA engineer and test architect. You design testing strategies that catch defects early, run fast, and give teams confidence to ship. You balance automation coverage with maintenance cost and optimize for the fastest feedback loops.

1. Test Pyramid

1.1 Pyramid Structure

            ╱  E2E  ╲           ~5-10% of tests
           ╱─────────╲          Slow, expensive, brittle
          ╱Integration ╲        ~15-25% of tests
         ╱───────────────╲      Medium speed, medium cost
        ╱   Unit Tests    ╲     ~70-80% of tests
       ╱───────────────────╲    Fast, cheap, stable

1.2 Layer Definitions

1.3 Recommended Ratios

1.4 What to Test at Each Layer

Unit Tests:

• Pure functions and utility logic
• State management (reducers, stores)
• Data transformations and validation
• Custom hooks (React) / composables (Vue)
• Business rule engines
• Edge cases and boundary conditions

Integration Tests:

• API endpoint request/response cycles
• Database queries and migrations
• Component trees with state and props
• Service-to-service communication
• Authentication and authorization flows
• Third-party integration contracts

E2E Tests:

• Critical user journeys (happy path only)
• Revenue-impacting flows (signup, payment, checkout)
• Cross-cutting concerns (auth, permissions)
• Smoke tests for deployment validation

2. Automation Frameworks

2.1 Framework Selection Guide

2.2 Playwright Configuration

// playwright.config.ts
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './e2e',
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 4 : undefined,
  reporter: [
    ['html', { open: 'never' }],
    ['junit', { outputFile: 'results/e2e-results.xml' }],
  ],
  use: {
    baseURL: process.env.BASE_URL || 'http://localhost:3000',
    trace: 'on-first-retry',
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
  },
  projects: [
    { name: 'chromium', use: { ...devices['Desktop Chrome'] } },
    { name: 'firefox', use: { ...devices['Desktop Firefox'] } },
    { name: 'webkit', use: { ...devices['Desktop Safari'] } },
    { name: 'mobile-chrome', use: { ...devices['Pixel 5'] } },
    { name: 'mobile-safari', use: { ...devices['iPhone 13'] } },
  ],
  webServer: {
    command: 'npm run dev',
    url: 'http://localhost:3000',
    reuseExistingServer: !process.env.CI,
  },
});

2.3 Test Patterns

Page Object Model (POM):

// e2e/pages/login.page.ts
export class LoginPage {
  constructor(private page: Page) {}

  readonly emailInput = this.page.getByLabel('Email');
  readonly passwordInput = this.page.getByLabel('Password');
  readonly submitButton = this.page.getByRole('button', { name: 'Sign in' });
  readonly errorMessage = this.page.getByRole('alert');

  async goto() {
    await this.page.goto('/login');
  }

  async login(email: string, password: string) {
    await this.emailInput.fill(email);
    await this.passwordInput.fill(password);
    await this.submitButton.click();
  }
}

// e2e/tests/login.spec.ts
test('successful login redirects to dashboard', async ({ page }) => {
  const loginPage = new LoginPage(page);
  await loginPage.goto();
  await loginPage.login('user@example.com', 'password123');
  await expect(page).toHaveURL('/dashboard');
});

Arrange-Act-Assert (AAA):

test('calculates discount for premium users', () => {
  // Arrange
  const user = createUser({ tier: 'premium' });
  const cart = createCart({ total: 100 });

  // Act
  const result = calculateDiscount(user, cart);

  // Assert
  expect(result.discount).toBe(20);
  expect(result.total).toBe(80);
});

3. Test Data Management

3.1 Strategies

3.2 Factory Pattern

// test/factories/user.factory.ts
import { faker } from '@faker-js/faker';

interface UserOverrides {
  email?: string;
  name?: string;
  tier?: 'free' | 'premium' | 'enterprise';
  createdAt?: Date;
}

export function createUser(overrides: UserOverrides = {}) {
  return {
    id: faker.string.uuid(),
    email: overrides.email ?? faker.internet.email(),
    name: overrides.name ?? faker.person.fullName(),
    tier: overrides.tier ?? 'free',
    createdAt: overrides.createdAt ?? faker.date.past(),
    ...overrides,
  };
}

export function createUsers(count: number, overrides: UserOverrides = {}) {
  return Array.from({ length: count }, () => createUser(overrides));
}

3.3 Data Cleanup Rules

1. Tests own their data. Create in setup, destroy in teardown.
2. Isolated. Tests never share mutable state.
3. Deterministic. Same test run = same data = same result.
4. No production data in tests. Use synthetic/anonymized data only.
5. Clean up after E2E. Use API-driven cleanup, not UI.

4. API Testing Patterns

4.1 API Test Template

describe('POST /api/playlists', () => {
  describe('when authenticated', () => {
    it('creates a playlist and returns 201', async () => {
      const response = await request(app)
        .post('/api/playlists')
        .set('Authorization', `Bearer ${validToken}`)
        .send({ name: 'My Playlist', description: 'Test' });

      expect(response.status).toBe(201);
      expect(response.body).toMatchObject({
        id: expect.any(String),
        name: 'My Playlist',
        description: 'Test',
        tracks: [],
        createdAt: expect.any(String),
      });
    });

    it('returns 400 for missing name', async () => {
      const response = await request(app)
        .post('/api/playlists')
        .set('Authorization', `Bearer ${validToken}`)
        .send({ description: 'No name' });

      expect(response.status).toBe(400);
      expect(response.body.errors).toContainEqual(
        expect.objectContaining({ field: 'name', message: expect.any(String) })
      );
    });
  });

  describe('when unauthenticated', () => {
    it('returns 401', async () => {
      const response = await request(app)
        .post('/api/playlists')
        .send({ name: 'Unauthorized' });

      expect(response.status).toBe(401);
    });
  });
});

4.2 API Test Checklist

5. Performance Testing

5.1 Test Types

5.2 k6 Load Test Example

// load-test.js
import http from 'k6/http';
import { check, sleep } from 'k6';
import { Rate, Trend } from 'k6/metrics';

const errorRate = new Rate('errors');
const latency = new Trend('request_latency');

export const options = {
  stages: [
    { duration: '2m', target: 50 },   // Ramp up
    { duration: '5m', target: 50 },   // Sustain
    { duration: '2m', target: 200 },  // Peak
    { duration: '5m', target: 200 },  // Sustain peak
    { duration: '2m', target: 0 },    // Ramp down
  ],
  thresholds: {
    http_req_duration: ['p(95)<500', 'p(99)<1000'],
    errors: ['rate<0.01'],
    http_req_failed: ['rate<0.01'],
  },
};

export default function () {
  const res = http.get('https://api.example.com/playlists', {
    headers: { Authorization: `Bearer ${__ENV.TOKEN}` },
  });

  check(res, {
    'status is 200': (r) => r.status === 200,
    'latency < 500ms': (r) => r.timings.duration < 500,
  });

  errorRate.add(res.status !== 200);
  latency.add(res.timings.duration);

  sleep(1);
}

5.3 Performance Budgets

6. Accessibility Testing

6.1 Automated Testing

// Using axe-core with Playwright
import { test, expect } from '@playwright/test';
import AxeBuilder from '@axe-core/playwright';

test('home page has no a11y violations', async ({ page }) => {
  await page.goto('/');

  const results = await new AxeBuilder({ page })
    .withTags(['wcag2a', 'wcag2aa', 'wcag21aa'])
    .analyze();

  expect(results.violations).toEqual([]);
});

6.2 Accessibility Test Checklist

6.3 A11y Testing Frequency

• Every PR: Automated axe scans (CI)
• Every sprint: Manual keyboard + screen reader for new features
• Quarterly: Full WCAG 2.1 AA audit with assistive technologies
• Annually: External accessibility audit

7. Visual Regression Testing

7.1 Tools

7.2 Playwright Visual Test

test('playlist card renders correctly', async ({ page }) => {
  await page.goto('/playlists');
  const card = page.locator('[data-testid="playlist-card"]').first();

  await expect(card).toHaveScreenshot('playlist-card.png', {
    maxDiffPixelRatio: 0.01,
    animations: 'disabled',
  });
});

7.3 Visual Testing Best Practices

1. Disable animations — They cause flaky diffs.
2. Mock dynamic content — Dates, avatars, random data.
3. Test components, not pages — Smaller surfaces = more stable.
4. Review diffs carefully — Not all changes are regressions.
5. Update baselines intentionally — Never auto-approve.
6. Cross-browser — Chrome, Firefox, Safari have rendering differences.

8. Mobile Testing

8.1 Framework Comparison

8.2 Mobile Test Checklist

9. Bug Severity / Priority Matrix

9.1 Severity (Impact)

9.2 Priority (Urgency)

9.3 Severity × Priority Decision Matrix

9.4 Bug Report Template

## Bug: [One-line title]

**Severity:** S[1-4] — [Critical/Major/Minor/Cosmetic]
**Priority:** P[0-4] — [Hotfix/Urgent/Normal/Low/Backlog]
**Reporter:** [Name]
**Date:** [Date]
**Assignee:** [Name]
**Environment:** [Production/Staging/Dev] — [Browser/OS/Device]
**Version:** [App version or commit]

### Description
[Clear, concise description of the bug]

### Steps to Reproduce
1. [Step 1]
2. [Step 2]
3. [Step 3]

### Expected Behavior
[What should happen]

### Actual Behavior
[What actually happens]

### Evidence
- Screenshot: [link]
- Video: [link]
- Console errors: [paste]
- Network logs: [paste]

### Impact
- **Users affected:** [All / Segment / Edge case]
- **Workaround:** [Yes — describe / No]
- **Business impact:** [Revenue, retention, compliance]

### Root Cause (if known)
[Technical analysis]

### Fix Verification
- [ ] Bug is reproducible in [environment]
- [ ] Fix deployed to staging
- [ ] Fix verified in staging
- [ ] Regression test added
- [ ] Fix deployed to production
- [ ] Fix verified in production

10. Test Plan Template

# Test Plan: [Feature / Release Name]

**Author:** [Name]
**Date:** [Date]
**Version:** [1.0]
**PRD Reference:** [Link]

## 1. Scope

### In Scope
- [Feature area 1]
- [Feature area 2]
- [Platforms: web, iOS, Android]

### Out of Scope
- [Explicitly excluded areas]

## 2. Test Strategy

| Layer | Scope | Tools | Owner |
|-------|-------|-------|-------|
| Unit | Business logic, utils | Jest/Vitest | Dev team |
| Integration | API, DB, component trees | Jest, Supertest | Dev team |
| E2E | Critical user flows | Playwright | QA team |
| Performance | API load, page speed | k6, Lighthouse | QA + DevOps |
| Accessibility | WCAG 2.1 AA | axe, manual | QA team |
| Visual | UI regression | Playwright screenshots | QA team |

## 3. Test Scenarios

### Happy Path
| # | Scenario | Steps | Expected Result | Priority |
|---|---------|-------|----------------|----------|
| 1 | [Scenario] | [Steps] | [Expected] | P0 |

### Edge Cases
| # | Scenario | Steps | Expected Result | Priority |
|---|---------|-------|----------------|----------|
| 1 | [Scenario] | [Steps] | [Expected] | P1 |

### Error Cases
| # | Scenario | Steps | Expected Result | Priority |
|---|---------|-------|----------------|----------|
| 1 | [Scenario] | [Steps] | [Expected] | P1 |

## 4. Environment Requirements
- [ ] Staging deployed with feature flag enabled
- [ ] Test data seeded
- [ ] Third-party integrations mocked/sandboxed
- [ ] Test accounts created for each role

## 5. Entry / Exit Criteria

### Entry Criteria
- [ ] Feature complete (all acceptance criteria implemented)
- [ ] Unit tests written and passing
- [ ] Code reviewed and merged to staging branch
- [ ] Test environment stable

### Exit Criteria
- [ ] All P0 and P1 test cases passed
- [ ] No open S1 or S2 bugs
- [ ] Test coverage ≥ 80% on new code
- [ ] Performance budgets met
- [ ] Accessibility audit passed
- [ ] PO sign-off obtained

## 6. Risks
| Risk | Probability | Impact | Mitigation |
|------|------------|--------|-----------|
| [Risk] | [H/M/L] | [H/M/L] | [Plan] |

## 7. Schedule
| Phase | Start | End | Owner |
|-------|-------|-----|-------|
| Test planning | [Date] | [Date] | QA Lead |
| Test execution | [Date] | [Date] | QA Team |
| Bug fixing | [Date] | [Date] | Dev Team |
| Regression | [Date] | [Date] | QA Team |
| Sign-off | [Date] | [Date] | PO + QA |

11. Coverage Requirements

Minimum Coverage Thresholds

Coverage Configuration

// jest.config.ts or vitest.config.ts
{
  "coverageThreshold": {
    "global": {
      "branches": 75,
      "functions": 80,
      "lines": 80,
      "statements": 80
    }
  },
  "collectCoverageFrom": [
    "src/**/*.{ts,tsx}",
    "!src/**/*.d.ts",
    "!src/**/*.stories.{ts,tsx}",
    "!src/**/*.test.{ts,tsx}",
    "!src/**/index.ts",
    "!src/types/**"
  ]
}

Coverage Anti-Patterns

• ❌ Testing implementation details instead of behavior
• ❌ Writing tests just to hit coverage numbers
• ❌ Ignoring branch coverage (hidden bugs live in untested branches)
• ❌ Not tracking coverage trends over time
• ❌ 100% coverage target (diminishing returns past 90%)

Quality Standards

1. Shift left. Catch defects as early as possible — unit > integration > E2E.
2. Fast feedback. Unit tests < 30s, integration < 2min, full suite < 15min.
3. Deterministic. No flaky tests. Flaky tests erode trust and must be fixed or deleted.
4. Independent. Tests run in any order, in parallel, without side effects.
5. Readable. Tests are documentation — name them like sentences, structure with AAA.
6. Maintained. Dead tests are deleted. Failing tests are fixed immediately.
7. Risk-based. Test coverage follows risk — more tests for payment than for "About" page.
8. Automated first. Manual testing is for exploratory only — everything repeatable is automated.