# Om Integration Tests

> Run and create QA integration tests (Playwright TypeScript), including executing the full suite, converting optional markdown scenarios, and generating new tests from specs or feature descriptions. Defers all environment boot/reuse to the `om-prepare-test-env` skill and attaches to the shared descriptor it writes. Use when the user says "run integration tests", "test this feature", "create test for", "convert test case", "run QA tests", or "integration test".

- Skill: `open-mercato/om-integration-tests-2` (Agent Skill)
- Install (CLI): `npx skillmds@latest add open-mercato/om-integration-tests-2`
- Raw SKILL.md: https://api.skillmd.com/api/skills/open-mercato/om-integration-tests-2/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Docs & Writing
- Author: open-mercato (https://skillmd.com/u/open-mercato)
- Updated: 2026-09-21
- Page: https://skillmd.com/skills/open-mercato/om-integration-tests-2

---


# Integration Tests Skill

This skill generates executable Playwright tests in module-local `__integration__` directories (for example `packages/core/src/modules/sales/__integration__/TC-SALES-*.spec.ts`) by exploring the running application. It also covers running existing integration tests after feature/bug implementation and reporting failures with artifact-based diagnosis. It optionally produces a markdown scenario (`.ai/qa/scenarios/TC-*.md`) for documentation — the scenario is **not required**.

**Environment boot/reuse is not this skill's job.** Discovering, provisioning, reusing (PID/readiness/freshness), and locking the test environment all live in the shared `om-prepare-test-env` skill (`.agents/skills/om-prepare-test-env/SKILL.md`), which wraps this repo's ephemeral tooling (see "Open Mercato environment" below) and writes a shared descriptor at `.ai/qa/test-env.json`. Integration tests **attach** to that already-booted instance (read `baseUrl`, `credentials`, and the Playwright runner `config` from the descriptor), invoking `om-prepare-test-env` to discover/provision when no valid descriptor exists.

## Quick Reference

| Action | Command |
|--------|---------|
| Run all tests (**preferred**) | `yarn test:integration:ephemeral` — provisions or safely reuses the ephemeral env itself |
| Run tests matching a path substring with the managed env | `yarn mercato test:integration <substring>` (no `--retries` support) |
| Iterate against a running ephemeral env (small test loops) | `yarn test:integration:ephemeral:start` once, then filtered `yarn mercato test:integration <filter>` runs |
| Run interactive ephemeral mode | `yarn test:integration:ephemeral:interactive` |
| Start ephemeral app only (for MCP exploration, tests development, and debugging) | `yarn test:integration:ephemeral:start` |
| Run all tests against an already-running app (low-level — needs the full runner env block) | `yarn test:integration` |
| Run single test (against an attached env, with the runner env block) | `npx playwright test --config .ai/qa/tests/playwright.config.ts <path>` |
| Run standalone create-app integration parity from monorepo | `yarn test:create-app:integration` |
| View report | `yarn test:integration:report` |
| Test files location | `<module>/__integration__/TC-XXX.spec.ts` |
| Scenario sources (optional) | `.ai/qa/scenarios/TC-XXX-*.md` |
| Shared env descriptor (attach here) | `.ai/qa/test-env.json` (written by `om-prepare-test-env`) |
| Underlying repo env state file | `.ai/qa/ephemeral-env.json` (managed by `om-prepare-test-env`) |

## Run Mode — prefer ephemeral, ask the user

Always prefer `yarn test:integration:ephemeral` over plain `yarn test:integration`: the ephemeral
runner prepares (or safely reuses) its own isolated app + database and injects the full runner env
block, so it is more autonomous and cannot touch the developer's dev data. Plain
`yarn test:integration` is a low-level command that only works with the full env block exported —
never reach for it first.

Before the first run in a session, when a user is present and has not already chosen, ask which
mode they want (recommend the first — more autonomous, safer regarding data):

1. **Fully managed ephemeral per run** — `yarn test:integration:ephemeral [filter]`.
2. **Boot once, iterate against the running ephemeral env** — `yarn test:integration:ephemeral:start`,
   then small filtered `yarn mercato test:integration <filter>` batches; best for short
   author/debug loops.

When running unattended, default to the fully managed ephemeral mode. The reuse/TTL/lock
semantics live in the repo-local `om-prepare-test-env` skill (`.ai/skills/om-prepare-test-env/SKILL.md`).

## Runtime Policy

Default QA runtime policy:
- Keep global settings in `.ai/qa/tests/playwright.config.ts`:
  - `timeout: 10_000`
  - `expect.timeout: 10_000`
  - `retries: 1`
- Do not add per-test timeout or retry overrides in `.spec.ts` files (`test.setTimeout`, `test.describe.configure({ retries })`, `test.retry`).

Debug/development policy (fail fast while authoring/fixing tests):
- Override retries at command level with `--retries=0`.
- Do not edit global config just to debug a single test.


## Rendering and Performance Gates

When a feature touches Next.js routes, generated frontend, Client Islands, shared providers, loading/error boundaries, or heavy widgets, plan tests beyond CRUD correctness:

- verify the server-rendered shell loads before client-only interaction is required,
- exercise each changed Client Island interaction (table/form/dialog/editor/calendar/graph),
- cover loading and error boundaries for changed routes,
- include accessibility assertions for labels, roles, focus, keyboard submit/cancel, and icon-only buttons,
- add a regression E2E for critical flows,
- record a smoke performance signal when feasible (cold load timing, Web Vitals/Lighthouse, or process/RSS note from the agreed profiling script).

If performance evidence is not feasible in the environment, state the blocker and the exact command/check that should be run before merge.

## Open Mercato environment (what om-prepare-test-env wraps)

Do not re-implement environment discovery, boot, reuse, PID/freshness checks, or locking here — that is `om-prepare-test-env`'s job (`.agents/skills/om-prepare-test-env/SKILL.md`). This section only names the repo tooling that skill honors under its "Honor repo tooling" rule:

- Ephemeral bring-up/reuse commands: `yarn test:integration:ephemeral` (full suite in ephemeral containers), `yarn test:integration:ephemeral:start` (app only — for MCP exploration, test development, debugging), `yarn test:integration:ephemeral:interactive` (reused ephemeral app + DB for short local loops).
- Plain suite against an already-running app: `yarn test:integration`. **Caution:** this only works when the caller exports the full runner env block (at minimum `BASE_URL`, `DATABASE_URL` of the ephemeral DB, and `QUEUE_BASE_DIR`); with `BASE_URL` alone, DB-fixture helpers fall back to `apps/mercato/.env`'s dev `DATABASE_URL` and fail with cross-database FK violations. Prefer `yarn mercato test:integration [filter]`, which injects the env block itself. See the repo-local `om-prepare-test-env` skill (`.ai/skills/om-prepare-test-env/SKILL.md`) for the full env contract, reuse-TTL semantics, and owner-lock teardown.
- Default ephemeral app port is `5001` when available; the actual bound port is recorded in the repo state file.
- Repo state file: `.ai/qa/ephemeral-env.json` (managed by `om-prepare-test-env`, not written by hand).

`om-prepare-test-env` uses these commands to bring up or reuse the environment, then writes the shared descriptor `.ai/qa/test-env.json` that tests attach to. To get a running instance, invoke that skill; to run tests, read `baseUrl`/`credentials`/runner `config` from the descriptor.

## Workflow

### Phase 1 — Identify What to Test

Determine the feature scope from one of these sources (in priority order):

1. **Spec file**: If a spec is referenced or was just implemented, read it from `.ai/specs/*.md` or `.ai/specs/enterprise/*.md`. Prefer the new `{YYYY-MM-DD}-{slug}.md` filenames, but tolerate legacy numbered names while the repo is being normalized. Extract testable scenarios from the API Contracts, UI/UX, and Data Models sections.
2. **User description**: If the user describes a feature ("test the company creation flow"), map it to the relevant module and pages.
3. **Recent changes**: If triggered after implementation, use `git diff` or recent commits to identify changed endpoints, pages, and components.

For each feature, identify:
- Which **category** it belongs to (AUTH, CAT, CRM, SALES, ADMIN, INT, API-*)
- Whether it's a **UI test** or **API test**
- The **priority** (High for CRUD operations, Medium for settings/config, Low for edge cases)
- The **prerequisite role** (superadmin, admin, or employee)

### Phase 2 — Find the Next TC Number

List existing test cases in the target category to determine the next sequential number:

```bash
ls .ai/qa/scenarios/TC-{CATEGORY}-*.md 2>/dev/null | sort | tail -1
find apps packages -type f -path "*/__integration__/*" -name "TC-{CATEGORY}-*.spec.ts" 2>/dev/null | sort | tail -1
```

Use the highest number found across both directories, then increment. For example, if the last scenario is TC-CRM-011 but the last test is TC-CRM-013, use TC-CRM-014.

### Phase 3 — Attach to the Shared Test Environment

Do not boot or reuse an environment yourself. Defer to `om-prepare-test-env` (`.agents/skills/om-prepare-test-env/SKILL.md`), which owns the reuse/PID/freshness/lock protocol and wraps this repo's ephemeral tooling (see "Open Mercato environment" above).

- Read the shared descriptor `.ai/qa/test-env.json`. When it reports `status: running` and passes that skill's validation, **attach** to it: take `baseUrl`, `credentials`, and the Playwright runner `config` from the descriptor.
- When no valid descriptor exists, invoke `om-prepare-test-env` to discover/provision one (it brings up or reuses the ephemeral env via the OM commands above, then writes the descriptor), then attach.
- Never hardcode a guessed `localhost:<port>` — always take the base URL from the descriptor.

### Phase 4 — Explore the Feature via Playwright MCP

Use the active `baseUrl` from the shared descriptor `.ai/qa/test-env.json` for MCP navigation, then discover the actual UI:

1. Login with the appropriate role
2. Navigate to the relevant page
3. Take snapshots to identify exact element labels, button text, form fields
4. Walk through the happy path to discover the actual flow
5. Note any validation messages, success states, redirects

For API tests, use cURL to discover:
1. The exact endpoint path and method
2. Required request headers and body shape
3. The actual response structure
4. Error responses for invalid inputs

### Phase 5 — Write the Playwright Test

Create the test in the module where the behavior lives:

- Core/shared module: `packages/<package>/src/modules/<module>/__integration__/TC-{CATEGORY}-{XXX}.spec.ts`
- App-specific module: `apps/mercato/src/modules/<module>/__integration__/TC-{CATEGORY}-{XXX}.spec.ts`
- Create-app template module: `packages/create-app/template/src/modules/<module>/__integration__/TC-{CATEGORY}-{XXX}.spec.ts`
- Enterprise overlay test: `packages/enterprise/modules/<module>/__integration__/TC-{CATEGORY}-{XXX}.spec.ts`
  - Only create enterprise overlay tests as additions to modules that already have base module tests.
  - Do not add dependencies from base code to the enterprise package.
  - Subfolders inside `__integration__` are supported.

Use the locators discovered in Phase 3 (not guessed). If a scenario was written, reference it in a comment.
Do not hardcode entity IDs in routes, payloads, or assertions. Resolve entities dynamically at runtime by creating fixtures through API/UI steps or by selecting existing rows via stable UI text/role locators.

Metadata for conditional test enablement:

- Helpers:
  - Put shared helpers in `packages/core/src/helpers/integration/` (importable as `@open-mercato/core/helpers/integration/*`).
  - Module-local `__integration__/helpers/` files should re-export central helpers where possible.
  - Standalone app developers: import helpers from `@open-mercato/core/helpers/integration/*` (included in the npm package).

- Folder-level metadata:
  - Add `meta.ts` or `index.ts` anywhere under `__integration__/`.
  - Supported module keys: `dependsOnModules`, `requiredModules`, `requiresModules`.
  - Supported env keys: `requiredEnvVars`, `requiresEnvVars`, `requiredAnyEnvVars`, `requiresAnyEnvVars`.
  - Example:

```ts
export const integrationMeta = {
  description: 'Billing integration coverage',
  dependsOnModules: ['sales', 'currencies'],
}
```

- Per-test metadata:
  - Add metadata directly inside the `.spec.ts` file using the same keys, or create sibling file `TC-XXX.meta.ts`.
  - Example sibling file:

```ts
export const integrationMeta = {
  dependsOnModules: ['catalog'],
}
```

- Evaluation model:
  - Dependencies inherit from `__integration__/` root through nested subfolders and then per-test metadata is applied.
  - If any required module is not enabled in the app, matching tests are skipped automatically (excluded from discovery/run).
  - If any `requiredEnvVars` entry is missing or blank, matching tests are skipped automatically (excluded from discovery/run).
  - If `requiredAnyEnvVars` is set and none of the listed env vars is configured, matching tests are skipped automatically.
  - Only env-gate tests that truly require external services. If an AI/LLM flow can be stubbed or can skip only the live model-backed subcase, keep the test runnable without secrets.

### Phase 6 — Optionally Write the Markdown Scenario

If documentation is desired, create `.ai/qa/scenarios/TC-{CATEGORY}-{XXX}-{slug}.md` using the template:

```markdown
# Test Scenario [NUMBER]: [TITLE]

## Test ID
TC-{CATEGORY}-{XXX}

## Category
{Category Name}

## Priority
{High/Medium/Low}

## Type
{UI Test / API Test}

## Description
{What this test validates — derived from spec or feature description}

## Prerequisites
- User is logged in as {role}
- {Other prerequisites from spec}

## Test Steps
| Step | Action | Expected Result |
|------|--------|-----------------|
| 1 | {Discovered action} | {Observed result} |
| 2 | {Discovered action} | {Observed result} |

## Expected Results
- {Derived from spec's API Contracts or UI/UX section}

## Edge Cases / Error Scenarios
- {Derived from spec's Risks section or discovered during exploration}
```

Fill steps with **actual** actions and results observed during Phase 3, not hypothetical ones.

This step is **optional** — skip it if the user only wants the executable test.

### Phase 7 — Verify

Run the new test to confirm it passes:

```bash
npx playwright test --config .ai/qa/tests/playwright.config.ts <path-to-test-file>
```

When developing/debugging the test, run fail-fast with no retries:

```bash
npx playwright test --config .ai/qa/tests/playwright.config.ts <path-to-test-file> --retries=0
```

If it fails, fix it. Do not leave broken tests.

### Create-App / Standalone Parity

When the change affects `packages/create-app`, standalone scaffolding, or CLI behavior consumed by scaffolded apps, prefer the monorepo parity command:

```bash
yarn test:create-app:integration
```

What it does:
- builds the local monorepo package artifacts
- scaffolds a fresh temporary standalone app with the local `create-mercato-app`
- installs local packed `@open-mercato/*` tarballs into that app
- runs the standalone app's own ephemeral integration command via the local CLI

Use this instead of plain `yarn test:integration` when the risk is specifically "works in monorepo, breaks in scaffolded standalone app".

### Shared — Failure Analysis and User Reporting (Mandatory on Failures)

After any failed test run (single test or suite), analyze failure artifacts before responding. This shared section applies both when:
- writing/updating tests
- only running existing tests after implementing features or bug fixes

1. Parse terminal output to capture the failing test names and first error stack/assertion.
2. Inspect Playwright artifacts for each failed test from `test-results/` and the HTML report:
   - `error-context.md`
   - screenshots (expected/actual/diff where available)
   - trace/video attachments if present
3. Classify each failure into one primary reason:
   - Product regression / real app bug
   - Test issue (stale locator, brittle assertion, bad fixture/cleanup)
   - Environment / data issue (service unavailable, auth/session drift, shared-state collision)
4. Decide ownership per failing test:
   - `User/Product team` when behavior looks like a real regression or requirement mismatch
   - `Agent/QA` when failure is test-code quality, selector drift, or fixture instability
   - `Shared` when both product behavior and test assumptions need adjustment
5. Respond with a table (required format) before any optional narrative:

| Failing test | Evidence used | Reasoning (why it failed) | Suggested owner | Next action |
|--------------|---------------|---------------------------|-----------------|-------------|
| `<path>::<test name>` | `stdout + screenshot + error-context` | `Concise technical diagnosis` | `User/Product team` / `Agent/QA` / `Shared` | `Concrete fix recommendation` |

Do not provide a generic "tests failed" summary without per-test reasoning.

### Running-Only Mode (No New Test Authoring)

If the user asks only to run integration tests (full suite/category/single file), skip authoring phases and execute the requested run directly.  
If the run fails, apply the shared failure-analysis section above.

## Rules

- MUST explore the running app before writing — never guess selectors or flows
- MUST prefer `yarn test:integration:ephemeral` over plain `yarn test:integration` — the ephemeral runner provisions/reuses its own isolated env and injects the runner env block; plain `yarn test:integration` is low-level and unsafe without that block
- MUST ask the user which run mode they want (fully managed ephemeral per run vs boot-once via `yarn test:integration:ephemeral:start` + filtered iteration) before the first run when interactive, recommending fully managed ephemeral; default to fully managed ephemeral when unattended
- MUST defer environment boot/reuse to `om-prepare-test-env`; never re-implement PID/reuse/freshness/lock logic here
- MUST read the shared descriptor `.ai/qa/test-env.json` first and attach to that running instance; invoke `om-prepare-test-env` to discover/provision when no valid descriptor exists (it manages the underlying `.ai/qa/ephemeral-env.json` state file)
- MUST use the active `baseUrl` from `.ai/qa/test-env.json` (never assume `localhost:3000`)
- MUST NOT hardcode record IDs (UUIDs/PKs) in generated tests
- MUST discover or create test entities at runtime, then navigate using discovered links/URLs
- MUST NOT rely on seeded/demo data for prerequisites
- MUST create required fixtures per test (prefer API fixture setup for stability)
- MUST clean up any data created by the test in `finally`/teardown
- MUST keep tests deterministic and isolated from run order or retries
- MUST NOT add per-test timeout/retry overrides in `.spec.ts`; rely on global Playwright config (`timeout: 10s`, `expect.timeout: 10s`, `retries: 1`)
- MUST create the `.spec.ts` — the markdown scenario is optional
- MUST use actual locators from Playwright MCP snapshots (`getByRole`, `getByLabel`, `getByText`)
- MUST verify the test passes before finishing
- MUST analyze failed test artifacts (`stdout`, `error-context.md`, screenshots/report) before reporting failures
- MUST report failures in a per-test table that includes reason, evidence, and suggested owner
- MUST apply the same failure-analysis and table-reporting rules when only running existing tests after implementation work
- MUST place executable tests in module-local `__integration__` directories; never add `.spec.ts` files under `.ai/qa/tests/`
- MUST keep module-specific helper utilities next to tests under `<module>/__integration__/helpers/`; for shared/cross-module helpers, import from `@open-mercato/core/helpers/integration/*`
- MUST treat `packages/enterprise/modules/<module>/__integration__/` as an optional overlay and keep base code independent from enterprise
- MUST use `meta.ts` or `index.ts` dependency metadata for module-gated folders and per-test `.meta.ts` (or in-file metadata) for individual gating
- When deriving from a spec, focus on the happy path first, then add edge cases as separate test cases if they warrant it
- Each test file covers one scenario — create multiple files for multiple scenarios

## Deriving Scenarios from a Spec

When reading a spec, extract test scenarios from these sections:

| Spec Section | Generates |
|-------------|-----------|
| API Contracts — each endpoint | One API test per endpoint (CRUD) |
| UI/UX — each user flow | One UI test per flow |
| Edge Cases / Error Scenarios | One test per significant error path |
| Risks & Impact Review | Regression tests for documented failure modes |

Typical spec produces 3-8 test cases. Prioritize:
1. **High**: CRUD happy paths, authentication, authorization
2. **Medium**: Validation errors, edge cases with business impact
3. **Low**: Cosmetic, minor UX edge cases

## Example

Given SPEC-017 (Version History Panel), the skill would produce:

- `packages/core/src/modules/admin/__integration__/TC-ADMIN-011.spec.ts` — UI: open history panel on an entity
- `packages/core/src/modules/admin/__integration__/TC-API-AUD-007.spec.ts` — API: fetch audit logs for entity
- `packages/core/src/modules/admin/__integration__/TC-ADMIN-012.spec.ts` — UI: restore a previous version
- Optionally: matching `.ai/qa/scenarios/TC-ADMIN-011-*.md` files for documentation

## Running Existing Tests

Pick the run mode first (see "Run Mode — prefer ephemeral, ask the user"); the environment
itself is `om-prepare-test-env`'s job (see "Open Mercato environment"):

```bash
# Run all integration tests headlessly (preferred — provisions/reuses the ephemeral env itself)
yarn test:integration:ephemeral

# Run tests matching a module/category path fragment with the managed env
yarn mercato test:integration sales

# Iterate: boot the ephemeral env once, then run small filtered batches against it
yarn test:integration:ephemeral:start
yarn mercato test:integration auth

# Run a single test, fail-fast while debugging (requires an attached env + runner env block)
npx playwright test --config .ai/qa/tests/playwright.config.ts packages/core/src/modules/auth/__integration__/TC-AUTH-001.spec.ts --retries=0
```

## Batch Conversion

When converting multiple scenarios at once:

1. List unconverted scenarios by comparing `.ai/qa/scenarios/` vs discovered `**/__integration__/**/*.spec.ts`
2. Convert one category at a time
3. Run the full suite after each category to catch cross-test issues
4. Report summary: total converted, passed, failed