# E2e Setup > Set up Playwright E2E testing framework for any project — auto-detects stack, auth, routes, and selectors - Skill: `code-shock/e2e-setup` (Agent Skill, multi-file: 2 files) - Install (CLI): `npx skillmds@latest add code-shock/e2e-setup` - Raw SKILL.md: https://api.skillmd.com/api/skills/code-shock/e2e-setup/raw - Safety review: pending (external: skill-scanner PASS, skillspector PASS) - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: Code-Shock (https://skillmd.com/u/code-shock) - Updated: 2026-09-22 - Page: https://skillmd.com/skills/code-shock/e2e-setup --- # Playwright E2E Testing Setup (Universal) You are setting up a production-grade Playwright E2E testing framework for the current project. You must auto-detect as much as possible and ask the user only what you can't figure out. --- ## Phase 1: Auto-Detection (Do All of This BEFORE Writing Any Code) ### 1.1 Detect Package Manager & Framework Read `package.json` (and `package-lock.json`, `yarn.lock`, `pnpm-lock.yaml` if they exist) to determine: - **Package manager**: npm, yarn, or pnpm - **Framework**: Vite, Next.js, CRA, Remix, Astro, SvelteKit, Nuxt, Angular, or plain Node/Express - **Language**: JavaScript or TypeScript (check for `tsconfig.json`, `.ts`/`.tsx` files) - **Module system**: ESM (`"type": "module"`) or CJS ### 1.2 Detect Dev Server Port Check these sources in order to find the dev server port: - `vite.config.*` → `server.port` - `next.config.*` → check for custom port - `angular.json` → `serve.options.port` - `package.json` scripts → look for `--port` flags in `dev`/`start` scripts - Framework defaults: Vite=5173, Next=3000, CRA=3000, Angular=4200, Remix=3000, Astro=4321, SvelteKit=5173, Nuxt=3000 Record this as `DEV_PORT`. The **test port** will be `DEV_PORT + 26` (e.g., 5173→5199, 3000→3026) to avoid conflicts with a running dev server. This is configurable via `TEST_PORT` env var. ### 1.3 Detect Dev Server Start Command Read the `dev` or `start` script from `package.json`. This is what Playwright's `webServer` block will run, with the port overridden to the test port. ### 1.4 Detect Auth Provider Search for imports and usage of: - **Firebase Auth**: `firebase/auth`, `connectAuthEmulator` - **Supabase Auth**: `@supabase/supabase-js`, `supabase.auth` - **Auth0**: `@auth0/auth0-react`, `@auth0/nextjs-auth0` - **Clerk**: `@clerk/clerk-react`, `@clerk/nextjs` - **NextAuth/Auth.js**: `next-auth`, `@auth/core` - **Passport**: `passport`, `passport-local` - **Custom/JWT**: manual JWT handling, `jsonwebtoken` - **None**: no auth detected Record which provider and which auth services are initialized (email/password, Google SSO, magic link, etc.). ### 1.5 Detect Database / Backend Search for: - **Firestore**: `firebase/firestore`, `connectFirestoreEmulator` - **Supabase DB**: `supabase.from()` - **Prisma**: `@prisma/client` - **MongoDB/Mongoose**: `mongoose`, `mongodb` - **PostgreSQL**: `pg`, `postgres` - **SQLite**: `better-sqlite3`, `sql.js` - **Drizzle**: `drizzle-orm` - **API-only**: REST/GraphQL calls to external services ### 1.6 Detect Emulator / Local Dev Setup Check for: - `firebase.json` → emulator config (which emulators, which ports) - `.env.local`, `.env.development` → local service URLs - `docker-compose.yml` → local database/service containers - `supabase/config.toml` → Supabase local config - Any seed scripts that already exist Record which emulators/local services are available and their ports. ### 1.7 Detect Router & Routes Find the router configuration: - **React Router**: search for `createBrowserRouter`, `.json' }]]`. Timestamp uses `new Date().toISOString().replace(/[:.]/g, '-')`. - **`webServer`**: start the project's dev server on the test port. Set any emulator env vars when target is local. Use `reuseExistingServer: false` always. - **Projects** (only include confirmed targets): - `local` — baseURL from env config, runs all tests - `staging` — only if user confirmed staging exists - `prod` — baseURL from prod URL, runs all tests - `smoke-only` — prod baseURL, testDir limited to `tests/e2e/smoke/` **Framework-specific webServer adjustments:** - **Vite**: `npx vite --port {TEST_PORT}` - **Next.js**: `npx next dev -p {TEST_PORT}` - **CRA**: `PORT={TEST_PORT} npx react-scripts start` - **Angular**: `npx ng serve --port {TEST_PORT}` - **Remix**: `PORT={TEST_PORT} npx remix dev` - **Astro**: `npx astro dev --port {TEST_PORT}` - **SvelteKit**: `npx vite dev --port {TEST_PORT}` - **Nuxt**: `npx nuxt dev --port {TEST_PORT}` **Framework-specific emulator env vars:** - **Vite projects**: prefix with `VITE_` (e.g., `VITE_USE_EMULATORS=true`) - **Next.js**: prefix with `NEXT_PUBLIC_` (e.g., `NEXT_PUBLIC_USE_EMULATORS=true`) - **CRA**: prefix with `REACT_APP_` (e.g., `REACT_APP_USE_EMULATORS=true`) - **Angular**: use `environment.ts` or env vars - **Others**: use plain env vars ### 3.5 Emulator / Local Service Connection Based on what was detected: **Firebase**: Add conditional emulator connection in the Firebase init file if not already present. Only connect emulators that are actually configured. Use ports from `firebase.json`. Prefer port 8085 for Firestore if configuring fresh (8080 conflicts with other services). **Supabase**: Point to local Supabase instance (default `http://localhost:54321`). **Docker services**: Document that `docker-compose up` is needed before tests. **No local services**: Skip this step, tests will run against live services. **Important**: Minimize changes to source code. Only add the emulator/local connection conditional if it doesn't already exist. ### 3.6 Flow Definitions (two layers) Flows are the core abstraction of this framework. There are two layers: #### Layer 1: Flow Helpers (`tests/e2e/flows/helpers/auth.{ext}`) Low-level building blocks that other flows compose. These perform a single action and return — they are **not** full journeys. Using the **actual selectors detected in Phase 1**, create: - **`loginUser(page, email, password)`**: Navigate to login page, fill credentials using real selectors, submit, wait for navigation away from login - **`signupUser(page, { name, email, password, ...fields })`**: Navigate to register page, fill all fields using real selectors, submit (note any post-signup redirects like payment) - **`logoutUser(page)`**: Find and click logout using real selectors, wait for redirect to login - **`waitForAuth(page)`**: Wait for auth state to resolve — either dashboard content appears or auth page appears **Use the exact selectors from the actual components.** Do not guess or use generic selectors. If a selector can't be determined, leave a `// TODO: update selector` comment. #### Layer 2: Flow Definitions (`tests/e2e/flows/*.flow.{ext}`) Full user journeys that take a `page` (and optional params) and walk through a complete path **with assertions at every step**. Not just "click and move on" — each step waits, then verifies the UI is correct before proceeding. The specific flows to generate are **auto-detected from Phase 1 scanning** of routes and components. Do not prescribe fixed flows — detect what the project has and generate flows that match. Common patterns: **Browse flow** (`browse.flow.{ext}`) — for projects with public pages: - Land on home page → assert key elements render (hero, nav, CTA) → navigate to another public page → assert content renders → navigate to a third → assert no console errors throughout - Every navigation step has an assertion before proceeding **Registration flow** (`registration.flow.{ext}`) — for projects with user signup: - Land on home → click registration link → assert register form renders → fill form → submit → assert redirect to expected post-signup destination → assert welcome/onboarding content visible - Uses `signupUser` helper internally **Dashboard/primary app flow** (`dashboard.flow.{ext}`) — for projects with authenticated areas: - Uses `loginUser` helper to authenticate → assert main app view renders → navigate to primary feature → assert feature content loads → navigate to secondary feature → assert it loads → logout → assert return to public/login page - Uses `loginUser` and `logoutUser` helpers internally **Generate flows based on what the project actually has.** A CRUD app gets a CRUD flow. A read-only dashboard gets a dashboard flow. A content site gets a browse flow. A project with settings gets a settings flow. The key contract for every flow: ```javascript // Every flow follows this contract: // - Takes a page and optional params // - Walks through a complete linear journey // - Asserts at EVERY step before proceeding // - Throws on any assertion failure export async function browseFlow(page) { await page.goto('/'); await expect(page.locator('...')).toBeVisible(); // assert before moving on await page.click('...'); await expect(page.locator('...')).toBeVisible(); // assert before moving on // ... every step follows this pattern } ``` #### Concurrency Utility (`tests/e2e/flows/helpers/concurrency.{ext}`) Export a utility for asserting on `Promise.allSettled` results — this prevents concurrent tests from silently swallowing failures: ```javascript import { expect } from '@playwright/test'; export function assertAllFlowsSucceeded(results) { const failures = results .map((r, i) => ({ ...r, index: i })) .filter(r => r.status === 'rejected'); if (failures.length > 0) { const details = failures .map(f => ` User ${f.index}: ${f.reason?.message || f.reason}`) .join('\n'); expect(failures, `${failures.length} flow(s) failed:\n${details}`).toHaveLength(0); } } ``` ### 3.7 Test User Fixtures (`tests/fixtures/test-users.json`) Create 5 test users with email/password. If the registration form has additional required fields (like `displayName`), include those. If admin roles exist, note that an admin user will be seeded separately in the setup script. ### 3.8 Test User Seeding (`tests/e2e/helpers/setup.{ext}`) **Only create this if local emulators or a local database are available.** Create a script that: - Reads from `tests/fixtures/test-users.json` - Creates users in the local auth system - Creates corresponding database records with appropriate roles and any required fields (e.g., subscription status, onboarding completion) so tests can skip setup flows - If admin role exists: seeds an admin user separately - Skips users that already exist - Logs what it creates - Is runnable standalone: `node tests/e2e/helpers/setup.{ext}` **Provider-specific seeding:** - **Firebase**: Use `firebase-admin` SDK with emulator env vars - **Supabase**: Use `@supabase/supabase-js` with local service URL - **Prisma**: Use Prisma client to seed the DB directly - **Custom**: Use the project's existing user creation logic or API endpoints ### 3.9 Auth Tests **User login tests** (`tests/e2e/auth/user-login.spec.{ext}`): - Read creds from `E2E_USER_EMAIL` / `E2E_USER_PASSWORD` env vars with defaults - Test: user can log in and reach the main authenticated page - Test: user can access key protected routes (detected in Phase 1) - Test: user cannot access admin routes (if admin role exists) - Test: user can log out - Use `domcontentloaded` for page waits (avoid `networkidle` — live content like timers/websockets prevent it) **Admin login tests** (`tests/e2e/auth/admin-login.spec.{ext}`) — only if admin role detected: - Read creds from `E2E_ADMIN_EMAIL` / `E2E_ADMIN_PASSWORD` env vars with defaults - Test: admin can log in - Test: admin can access admin routes - Test: admin can also access regular protected routes **Invalid credentials tests** (`tests/e2e/auth/invalid-creds.spec.{ext}`): - Test: shows error for wrong password - Test: shows error for nonexistent email - Test: blocks invalid email format (check HTML5 validation if applicable) - Test: login form fields and submit button are present ### 3.10 Smoke Test (`tests/e2e/smoke/health.spec.{ext}`) A minimal test **safe to run against production** — NO writes, NO account creation: - Navigate to `/` (home page) - Assert page loads with expected title (detected from `index.html` or layout) - Assert no console errors (collect via `page.on('console')`) - Check key elements render (detected from the homepage component): - Logo / brand name - Navigation links - Primary CTA - Does NOT authenticate, write data, or modify state ### 3.11 Single-User Flow Tests (`tests/e2e/single/*.spec.{ext}`) Create one spec file per flow definition. Each test imports the flow and runs it once in a standard Playwright page: ```javascript // tests/e2e/single/browse.spec.{ext} import { test } from '@playwright/test'; import { browseFlow } from '../flows/browse.flow.{ext}'; test('browse flow completes successfully', async ({ page }) => { await browseFlow(page); }); ``` Create a spec for every flow generated in section 3.6. These are your "does this walkthrough work" tests — run them to verify each journey is correct before using them in concurrent tests. ### 3.12 Multi-User Concurrent Tests (`tests/e2e/concurrent/multi-user.spec.{ext}`) Create tests that import flow definitions and run them in parallel using separate browser contexts. Read `CONCURRENT_USERS` from the env config (default 5). **Homogeneous concurrency** — N users running the same flow: ```javascript import { test } from '@playwright/test'; import { dashboardFlow } from '../flows/dashboard.flow.{ext}'; import { assertAllFlowsSucceeded } from '../flows/helpers/concurrency.{ext}'; import { getConfig } from '../helpers/env.{ext}'; import testUsers from '../../fixtures/test-users.json'; const { concurrentUsers } = getConfig(); test('multiple users access dashboard simultaneously', async ({ browser }) => { const users = testUsers.slice(0, concurrentUsers); const results = await Promise.allSettled( users.map(async (user) => { const context = await browser.newContext(); const page = await context.newPage(); try { await dashboardFlow(page, user); } finally { await context.close(); } }) ); assertAllFlowsSucceeded(results); }); ``` **Mixed traffic simulation** — different flows running concurrently (the realistic scenario): ```javascript test('mixed traffic simulation', async ({ browser }) => { const newPage = async () => { const ctx = await browser.newContext(); return { page: await ctx.newPage(), ctx }; }; const tasks = [ ...Array(Math.ceil(concurrentUsers * 0.5)).fill(null).map(async () => { const { page, ctx } = await newPage(); try { await browseFlow(page); } finally { await ctx.close(); } }), ...testUsers.slice(0, Math.ceil(concurrentUsers * 0.3)).map(async (user) => { const { page, ctx } = await newPage(); try { await dashboardFlow(page, user); } finally { await ctx.close(); } }), ...testUsers.slice(0, Math.ceil(concurrentUsers * 0.2)).map(async (user) => { const { page, ctx } = await newPage(); try { await registrationFlow(page, user); } finally { await ctx.close(); } }), ]; const results = await Promise.allSettled(tasks); assertAllFlowsSucceeded(results); }); ``` **Key details:** - Each flow gets its own browser context — full isolation - `assertAllFlowsSucceeded` ensures no silent failures from `allSettled` - Default 5 concurrent users locally (safe for laptops running emulators + dev server + N Chromium contexts). Scale via `CONCURRENT_USERS` env var in CI. - Screenshots on failure per-user via Playwright's built-in failure handling - Mixed traffic ratios (50/30/20) are a starting point — adjust based on the project's actual usage patterns ### 3.13 Package.json Scripts Add scripts using the detected package manager and framework. Template: ```json { "pretest:e2e": "lsof -ti:${TEST_PORT:-{test_port}} | xargs kill -9 2>/dev/null || true", "test:e2e": "{emulator_wrapper_start}node tests/e2e/helpers/setup.{ext} && npx playwright test --project=local{emulator_wrapper_end}", "test:e2e:quick": "npx playwright test tests/e2e/smoke/ tests/e2e/auth/invalid-creds.spec.{ext}", "test:e2e:prod": "TEST_TARGET=prod npx playwright test --project=smoke-only", "test:e2e:smoke": "npx playwright test tests/e2e/smoke/", "test:e2e:auth": "{emulator_wrapper_start}node tests/e2e/helpers/setup.{ext} && npx playwright test tests/e2e/auth/{emulator_wrapper_end}", "test:e2e:flows": "{emulator_wrapper_start}node tests/e2e/helpers/setup.{ext} && npx playwright test tests/e2e/single/{emulator_wrapper_end}", "test:e2e:concurrent": "{emulator_wrapper_start}node tests/e2e/helpers/setup.{ext} && npx playwright test tests/e2e/concurrent/{emulator_wrapper_end}", "test:e2e:headed": "{emulator_wrapper_start}node tests/e2e/helpers/setup.{ext} && npx playwright test --headed{emulator_wrapper_end}", "test:e2e:debug": "{emulator_wrapper_start}node tests/e2e/helpers/setup.{ext} && npx playwright test --headed --workers=1 --timeout=0{emulator_wrapper_end}", "test:e2e:codegen": "npx playwright codegen http://localhost:${TEST_PORT:-{test_port}}", "test:e2e:seed": "node tests/e2e/helpers/setup.{ext}", "test:e2e:logs": "{emulator_wrapper_start}node tests/e2e/helpers/setup.{ext} && E2E_LOG=1 npx playwright test{emulator_wrapper_end}" } ``` Where `{emulator_wrapper_start/end}` is: - **Firebase**: `firebase emulators:exec --only {services} '...'` - **Supabase**: `supabase start && ... ; supabase stop` (or assume running) - **Docker**: `docker-compose up -d && ... ; docker-compose down` - **None**: just run the commands directly **Only add staging scripts if user confirmed staging exists.** **Design principles:** - `test:e2e` is the default — starts everything needed, seeds data, runs all tests. No footguns. - `test:e2e:flows` runs all single-user flow walkthroughs — verify each journey works before testing concurrency. - `test:e2e:quick` runs only tests that don't need emulators/local services. - `pretest:e2e` kills any process on the test port before starting. - `reuseExistingServer: false` ensures Playwright starts its own dev server. ### 3.14 Gitignore Add these entries to `.gitignore` (if not already present): ``` # E2E test artifacts e2e-logs/ test-results/ playwright-report/ ``` ### 3.15 GitHub Actions Workflow (`.github/workflows/e2e.yml`) Create a CI workflow that: - Triggers on push and pull_request - Installs dependencies and Playwright chromium - Installs any required local services (Firebase CLI, Supabase CLI, etc.) - Starts emulators/services, seeds data, runs tests - Sets appropriate emulator env vars - Uploads `test-results/` and `playwright-report/` as artifacts on failure --- ## Phase 4: Validate 1. Run `npx playwright test --list` to verify all tests are discovered 2. Show the user a summary of everything created 3. Suggest running `npm run test:e2e:quick` (or equivalent) as a first sanity check --- ## How to Approach This 1. **Do ALL of Phase 1 first** — read files, detect everything, build your mental model 2. **Ask Phase 2 questions** — present findings, proposed flows, get user input 3. **Scaffold everything in Phase 3** — use exact detected values, not placeholders. Generate flow helpers, flow definitions, single-user specs, and concurrent specs. 4. **Validate in Phase 4** — make sure it actually works 5. **Commit message**: `feat: add Playwright E2E testing framework with flow-based architecture`