Loom E2e Testing

E2E Testing

Overview

Browser E2E validates full user journeys. This file owns selectors, auto-waiting, network interception, POM, browser flakiness, and trace/video debugging (Playwright-first, Cypress noted where it differs). E2E is the tip of the pyramid — keep it thin and reserve it for critical paths; push logic down to unit/integration (loom-test-strategy). For test-double taxonomy and AAA see loom-testing.

Framework choice

Default to Playwright for new suites: true multi-browser (incl. WebKit/Safari), multi-tab/origin, out-of-process network interception, built-in parallelism and trace viewer, and non-JS bindings (Python/.NET/Java). Choose Cypress only when the team is already invested or wants its live time-travel debugger; its in-browser model limits multi-tab, cross-origin, and true WebKit. Selenium/Puppeteer: legacy or Chrome-only automation, not greenfield E2E.

npm init playwright@latest         # scaffolds config + CI + browsers

// playwright.config.ts — the load-bearing knobs
export default defineConfig({
  testDir: "./e2e",
  fullyParallel: true,                       // parallel within files too
  forbidOnly: !!process.env.CI,              // fail CI if a .only slips in
  retries: process.env.CI ? 2 : 0,           // retry ONLY to surface flakes
  reporter: [["html"], ["junit", { outputFile: "results.xml" }]],
  use: {
    baseURL: "http://localhost:3000",
    trace: "on-first-retry",                 // trace the retry, not every run
    screenshot: "only-on-failure",
    video: "retain-on-failure",
  },
  webServer: {                                // boots the app, waits for it
    command: "npm run dev",
    url: "http://localhost:3000",
    reuseExistingServer: !process.env.CI,
  },
  projects: [
    { name: "chromium", use: { ...devices["Desktop Chrome"] } },
    { name: "firefox",  use: { ...devices["Desktop Firefox"] } },
    { name: "webkit",   use: { ...devices["Desktop Safari"] } },
  ],
});

⚠ retries exists to diagnose flakes (a test that passes only on retry is flaky and must be fixed), never to hide them — a retry-masked bug ships. webServer is why page.goto doesn't race the server boot; without it you get connection-refused flakes.

Selectors — user-facing first

Priority, best → worst. Higher options survive refactors and assert accessibility for free; CSS/XPath couple tests to DOM structure and shatter on markup changes.

1. Role + accessible name — getByRole("button", { name: "Sign in" })
2. Label / placeholder / text — getByLabel("Email"), getByText("Welcome")
3. data-testid — for elements with no stable role/text
4. CSS — only for structural selection (.modal-content)
5. XPath — avoid (brittle, unreadable)

page.getByRole("textbox", { name: "Email" });
page.getByLabel("Password");
page.getByTestId("product-card-123");           // add data-testid in the component
// chain/filter instead of nth-index (index shifts → flake)
page.getByTestId("product-list")
    .getByRole("listitem").filter({ hasText: "Widget" })
    .getByRole("button", { name: "Add to cart" });

⚠ Never select by index (.nth(2), items[2]) or auto-generated CSS-module class hashes — both change silently. Add data-testid at the component and strip it in prod builds (babel-plugin-react-remove-properties). Cypress: use @testing-library/cypress (cy.findByRole) for the same priority.

Auto-waiting vs explicit waits

Playwright locator actions auto-wait for the element to be attached, visible, stable, and enabled before acting, and web-first assertions (expect(locator).toBeVisible()) retry until they pass or time out. This eliminates almost all manual waiting.

// RIGHT — assert the condition; it retries internally
await page.getByRole("button", { name: "Submit" }).click();
await expect(page.getByText("Success")).toBeVisible();

// WRONG — arbitrary sleep: flaky if slow, wasteful if fast
await page.click("#submit");
await page.waitForTimeout(2000);                 // ⛔ never

Use explicit waits only for things auto-wait can't see — navigation and network:

await page.waitForURL("/dashboard");             // after a click that navigates
await page.getByTestId("spinner").waitFor({ state: "hidden" });

⚠ waitForTimeout is banned in real suites — it's the #1 source of both flakiness and slowness. waitForLoadState("networkidle") is discouraged by Playwright (racy on apps that poll); wait for a concrete UI condition instead. Cypress auto-retries assertions/queries but not raw values — re-query, don't cache elements.

Network interception

Stub the network to make E2E deterministic and to reach error states the UI can't otherwise hit. Register the route before the navigation that triggers it.

// Deterministic data — no dependence on backend state
await page.route("**/api/products", (route) =>
  route.fulfill({ status: 200, contentType: "application/json",
                  body: JSON.stringify([{ id: 1, name: "Test Widget" }]) }));

// Force an error path
await page.route("**/api/checkout", (route) => route.fulfill({ status: 500 }));

// Assert a request happened (wait for it, don't sleep)
const resp = page.waitForResponse("**/api/order");
await page.getByRole("button", { name: "Place order" }).click();
expect((await resp).status()).toBe(201);

⚠ Use glob/regex (**/api/x) — exact strings miss query params and absolute vs relative URLs. Decide deliberately: stub for speed/determinism/error-injection; hit the real backend for a handful of true full-stack smoke tests. Cypress uses cy.intercept() with an alias + cy.wait("@alias").

Page Object Model

Encapsulate page structure so a UI change touches one file, not fifty tests. Expose user intentions (login(email, pw)), not raw locators.

// e2e/pages/LoginPage.ts
export class LoginPage {
  constructor(private page: Page) {}
  readonly email = () => this.page.getByLabel("Email");
  readonly submit = () => this.page.getByRole("button", { name: "Sign in" });
  async goto()  { await this.page.goto("/login"); }
  async login(email: string, pw: string) {
    await this.email().fill(email);
    await this.page.getByLabel("Password").fill(pw);
    await this.submit().click();
  }
}

⚠ Keep assertions out of page objects (except small expect helpers) — a POM describes the page; the test owns the verification. Prefer Playwright fixtures over a monolithic App object for composing pages/state; they scope setup/teardown automatically.

Fixtures & auth (speed)

Logging in through the UI on every test is the biggest E2E time sink. Authenticate once, save storageState, and reuse it — cutting minutes off a suite.

// global-setup: log in once, persist cookies/localStorage
await page.goto("/login"); /* ...fill+submit... */
await page.context().storageState({ path: "e2e/.auth/user.json" });

// then in config or a fixture:
use: { storageState: "e2e/.auth/user.json" }

Custom fixtures give per-test isolated data with automatic cleanup:

export const test = base.extend<{ user: User }>({
  user: async ({}, use) => {
    const u = await prisma.user.create({ data: UserFactory.create() });
    await use(u);
    await prisma.user.delete({ where: { id: u.id } });   // teardown always runs
  },
});

⚠ Even faster: seed state via the API request context rather than clicking through setup — see hybrid pattern below.

Preventing browser flakiness

General flaky-test theory (clock, RNG, shared state) and the full cause table are in loom-test-strategy. Browser-specific rules:

• Wait for conditions, never timeouts — covered above; this is 80% of E2E flake.
• Select by role/text/testid, never index — DOM order shifts.
• Isolate per test — Playwright gives each test a fresh browser context (clean cookies/storage) by default; don't defeat it with shared global state. Mint unique data (test-${Date.now()}@x.com).
• Freeze time & animations for anything time- or motion-sensitive:

await page.clock.setFixedTime(new Date("2024-01-15T10:00:00Z"));
await page.emulateMedia({ reducedMotion: "reduce" });

• Broken-window rule: fix or quarantine a flaky test the day it appears; a tolerated flake trains the team to ignore red.

Debugging: trace, video, screenshot

npx playwright test --trace on      # force a trace for a run
npx playwright show-trace trace.zip # timeline, DOM snapshots, network, console
npx playwright test --debug         # step through with Inspector

Configure artifacts on failure only (trace: "on-first-retry", video/screenshot: retain/only-on-failure — see config) so green runs stay fast. The trace viewer (time-travel DOM + network + console per action) is the fastest way to root-cause a CI-only failure — wire trace upload into CI artifacts. await page.pause() opens the Inspector mid-test for local poking.

Visual regression

await expect(page).toHaveScreenshot("dashboard.png", {
  mask: [page.getByTestId("timestamp")],   // hide dynamic regions
  maxDiffPixelRatio: 0.01,
});

⚠ Screenshots are OS/font/GPU-dependent — generate and compare baselines in the same container you run CI in (--update-snapshots locally on a Mac then failing in Linux CI is the classic trap). Disable animations, mask dynamic content (dates, avatars, ads), and pin the viewport. Treat baselines as reviewed artifacts, not auto-accepted.

Hybrid API + UI

Do setup and teardown over HTTP (fast, reliable); reserve the browser for the user-facing assertion. Cuts runtime and removes setup-related flake.

test("cart shows seeded item", async ({ page, request }) => {
  await request.post("/api/cart/add", { data: { productId: "123", quantity: 2 } });
  await page.goto("/cart");
  await expect(page.getByTestId("cart-item")).toHaveCount(1);
  await expect(page.getByTestId("quantity")).toHaveText("2");
});

Cross-browser scope

Run the full matrix (Chromium/Firefox/WebKit + mobile viewports) on critical paths only; run the rest on Chromium to keep CI fast. Skip project-specifically rather than duplicating tests:

test.describe("Admin panel", () => {
  test.skip(({ browserName }) => browserName !== "chromium");  // Chrome-only
});

Smoke suite

A tiny must-pass set gating every release — homepage renders, auth works, key APIs return < 500:

test("user can sign in", async ({ page }) => {
  await page.goto("/login");
  await page.getByLabel("Email").fill("user@example.com");
  await page.getByLabel("Password").fill("password123");
  await page.getByRole("button", { name: "Sign in" }).click();
  await expect(page).toHaveURL("/dashboard");
});

Verify before done

• Selectors are role/label/text/testid — no index, XPath, or generated-class-hash selectors
• Zero waitForTimeout/sleep; waits target a condition, URL, or response
• Auth via storageState; per-test data is unique; cleanup in fixture teardown
• Network stubbed where determinism/error-injection is needed; real backend only for true smoke tests
• retries used to surface flakes, not mask them; any retry-only pass is triaged
• trace/video/screenshot retained on failure and uploaded as CI artifacts
• E2E limited to critical journeys; logic-level cases pushed down (loom-test-strategy)
• Visual baselines generated in the CI container, with dynamic regions masked
• No .only left in the suite (forbidOnly on in CI)