# Regression Testing

> Use when planning or writing a behaviour-first regression safety net BEFORE a major change — a Java/JDK version upgrade, a framework migration (e.g. Spring Boot 2→3, AngularJS→React), a runtime change, or a large refactor. Prioritises tests that validate observable external behaviour (API contracts, E2E flows, pure business logic) so the suite survives the change boundary, and provides a decision-gated, per-unit test-generation workflow. Triggers on regression test, pre-upgrade test, test strategy, safety net, behaviour parity, API contract test, test prioritisation, what to test before upgrade. For the mechanics of writing web/UI E2E tests, also use the web-test-automation skill.

- Skill: `aws-samples/regression-testing` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add aws-samples/regression-testing`
- Raw SKILL.md: https://api.skillmd.com/api/skills/aws-samples/regression-testing/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Web & Frontend
- License: MIT-0
- Author: aws-samples (https://skillmd.com/u/aws-samples)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/aws-samples/regression-testing

---


# Regression Testing (Pre-Change Safety Net)

Build a regression safety net **before** a major change so the tests catch regressions across the change boundary. The strategy below decides *what* to test; the referenced workflow describes *how* to drive test generation in a decision-gated, resumable way.

## When to Load Reference Files

| If the task involves… | Load |
|---|---|
| Driving end-to-end test generation on an existing codebase before an upgrade — the staged, decision-gated, per-unit workflow (analysis → planning → generation → verify) | `references/pre-upgrade-testing-workflow.md` |

For writing the actual web/UI E2E tests (Playwright/Cypress/Selenium, locators, flakiness, CI sharding), use the `web-test-automation` skill.

## Core Principle

> Tests written before an upgrade should validate **observable external behavior**, not internal implementation details. The goal is a safety net that catches regressions without being coupled to the thing being replaced.

## Test Priority Tiers

### Tier 1: Upgrade-Critical (Must Have)

These tests have the highest ROI because they validate the contracts that external consumers depend on. If these break after an upgrade, real users are affected.

**Backend → API Contract Tests**
- Validate REST endpoint paths, HTTP methods, request/response JSON shapes, status codes, and error responses
- Use framework-provided test utilities (e.g., MockMvc for Spring, supertest for Express) to test the HTTP layer without a running server
- Pin down serialization formats — field names, date formats, null handling, collection ordering
- Cover both success and error paths (404, 400, 500 responses)
- These survive framework upgrades because they test the HTTP contract, not the framework internals

**Frontend → E2E Browser Tests**
- Use framework-agnostic browser automation (Selenium, Playwright, Cypress etc.)
- Test real user flows: navigation, form submission, data display, error states
- Do NOT write framework-specific component tests (e.g., Angular TestBed, React Testing Library) if a frontend framework migration is planned — these get thrown away
- E2E tests survive any frontend framework swap because they test what the user sees

### Tier 2: Durable Logic (Should Have)

These tests are cheap to write and survive any upgrade because they test pure business logic with no framework dependency.

**Business Logic Unit Tests**
- Game engines, calculators, validators, transformers — any pure function or class
- Domain object behavior (lazy initialization, state transitions, collection management)
- Utility classes (ID generation, formatting, parsing)
- These have zero framework coupling and will pass identically before and after any upgrade

### Tier 3: Integration Verification (Nice to Have)

These tests validate internal wiring and are more likely to need adjustment after an upgrade, but still provide value.

**Factory/Service Integration Tests**
- Test orchestration logic with mocked dependencies
- Validate that business rules are applied correctly across component boundaries
- May need updates if dependency injection patterns change during upgrade

**Data Access Tests**
- Validate query construction, entity mapping, CRUD operations
- More likely to need adjustment if ORM or SDK versions change
- Consider using embedded/local versions of data stores where available

## Decision Rules

When deciding what to test for an upgrade, apply these rules:

1. **Will this test break because of the upgrade itself?** If yes, it's testing implementation, not behavior. Deprioritize or redesign it.
2. **Does this test validate something an external consumer depends on?** If yes, it's Tier 1. Write it first.
3. **Does this test have zero framework dependencies?** If yes, it's Tier 2. Write it — it's cheap and durable.
4. **Does this test require mocking framework internals?** If yes, it's Tier 3. Write it only if time permits.

## Integration Testing: When It Matters

**Worth it:** No shared API spec between repos, complex auth flows, race conditions, or service-to-service calls that only surface when both sides are live.

**Skip it:** If Tier 1 tests (API contracts + E2E) already cover the integration boundary from both sides, a separate integration suite adds complexity without proportional value.

## Anti-Patterns to Avoid

- **Testing framework internals**: Don't test that Spring wires beans correctly — test that the HTTP endpoint returns the right response
- **Framework-coupled component tests before migration**: Don't write AngularJS unit tests if you're about to migrate to React
- **Over-mocking**: If a test requires 10 mocks to run, it's testing wiring, not behavior. Consider testing at a higher level instead.
- **Testing generated code**: Don't test getters/setters or boilerplate — test behavior that could break

## Applying This to Specific Upgrade Paths

### Java Version Upgrade (e.g., 8 → 17/21/25)
- Focus: API contracts (serialization may change), business logic (reflection access restrictions)
- Watch for: `javax.*` → `jakarta.*` namespace changes, removed APIs, `--add-opens` requirements

### Spring Boot Major Upgrade (e.g., 1.x → 3.x)
- Focus: API contracts (annotation changes, error handling changes), request/response shapes
- Watch for: `@RequestMapping` behavior changes, Jackson defaults, CORS configuration changes

### Frontend Framework Migration (e.g., AngularJS → React)
- Focus: E2E browser tests only — everything else gets rewritten
- Watch for: URL routing changes, form behavior differences, async loading patterns

### AWS SDK Upgrade (e.g., v1 → v2)
- Focus: API contracts (external behavior unchanged), data access patterns
- Watch for: Client builder API changes, async vs sync patterns, credential provider changes