App Validation

App Validation

Use this skill together with the playwright-cli skill when validating the running app in a browser. The playwright-cli skill is upstream-managed and only covers the tool itself; this skill captures everything specific to this template. This app can only be tested and validated with the Fabric portal embed flow. See Testing inside the Fabric portal embed. NEVER test directly against localhost - the app will not work correctly.

Performance Rules

Use minimal initial_wait

Always use initial_wait: 1 for all playwright-cli tool calls to avoid unnecessary delays. Do not increase it unless explicitly required.

Carve-out: the first open against the Fabric portal embed (*.fabric.microsoft.com/...?devUri=...) takes 20–40s to render the appbackend chrome. Use initial_wait: 30 once for that call, then 1 afterward. See Testing inside the Fabric portal embed.

Skip auth — always

NEVER interact with or validate the token/auth prompt page. Inside your run-code call, inject the auth token via sessionStorage.setItem (or localStorage.setItem) and mock API responses with page.route() before calling page.reload(). The app must skip the auth prompt and render actual content immediately.

Carve-out: this rule applies to apps that gate themselves behind a token check the agent controls. For real AAD redirects (e.g. the Fabric portal sign-in flow at login.microsoftonline.com), do not click sign-in buttons or fill credentials, and do not mock sessionStorage — use a --persistent profile instead so the user signs in once and cookies replay on subsequent runs. See Testing inside the Fabric portal embed.

Skip screenshots unless asked

Only take a screenshot if the user explicitly requests one. Use snapshot (YAML accessibility tree) for validation.

Required Checks

• UI elements render correctly and are visible.
• Text meets accessibility standards.
• No console errors from the app. Ignore Fabric portal noise - see console-error-filter.

Visual Consistency Checks

• Verify key layout containers have non-zero computed padding and gap values. Zero spacing usually indicates an invalid token class mapping.
• Verify each visual's card background matches the app's other cards — it comes from containerClassName, not from a wrapper div.
• Verify card chrome is not painted twice — by both a wrapper and the visual's own container.
• Verify bar and arc stroke colors match card background (not primary text color).
• Verify axis label and data-label colors are consistent across charts using shared foreground-secondary semantics.
• Verify grouped/multi-series bar charts do not show auto-injected data labels unless explicitly requested by design.
• Verify every visual displays its expected content. Charts must render visible, correctly positioned data marks; grids must render their expected headers and cells.
• Verify chart render height is healthy for each chart canvas/SVG. Treat charts rendering below ~100px as suspicious and below ~50px as likely squished.
• Verify standalone (non-grid) chart sections use definite height instead of only minHeight when chart wrappers use h-full.
• Compare computed fontSize, fontFamily, and color across related input, select, and button controls in the same toolbar/filter row.
• Include a page.evaluate style check in validation runs that reports spacing/token and form typography mismatches as structured failures.
• Include a page.evaluate chart-height check that inspects chart canvas/SVG client heights and reports squished-chart mismatches.

Testing inside the Fabric portal embed

The URL under test should be *.fabric.microsoft.com and contain devUri=, so the app is being rendered inside the Fabric portal as a deeply-nested iframe (portal → *pbiabd.powerbi.com/appbackend → http://localhost:5173). Use the template's wired-up flow instead of the generic open recipe.

Provisioning preflight

Before launching the browser, check that the combined values in .env.local and .env.fabric include all three values required to identify an existing Fabric AppBackend:

• VITE_FABRIC_PORTAL_URL
• VITE_FABRIC_WORKSPACE_ID
• VITE_FABRIC_ITEM_ID

If any value is missing, provision the app using the exact target Fabric workspace URI supplied by the user or task:

npx rayfin up --workspace-uri "<target-workspace-uri>"

If no target workspace URI was supplied, ask the user for one. rayfin up provisions the Fabric AppBackend and writes the required values to the environment files; it does not need to be rerun before every validation when valid deployment configuration already exists.

Start the development server

Run the Vite development server in a separate, long-running terminal and confirm that the URL reported by Vite responds before opening the Fabric portal:

npm run dev

If Vite uses a non-default URL (a URL that is not http://localhost:5173), set DEV_URL to that URL in the same shell command that runs npm run test:fabric.

Launch the Fabric browser session

npm run test:fabric

This runs scripts/open-fabric-portal.mjs, which composes the embed URL from the VITE_FABRIC_* environment files and launches a named persistent session with the right Chromium flags:

playwright-cli -s=fabric open --persistent --config=.playwright-config.json "<embed-url>"

npm run test:fabric only opens the browser session. It does not provision the AppBackend, start the development server, or perform the required checks. Use subsequent playwright-cli -s=fabric commands to inspect and validate the embedded app frame.

Why three pieces are required

Frame discovery snippet

The app loads three frames deep. Use this single run-code to locate it (replace localhost:5173 with the actual Vite URL if different):

async page => {
  await page.waitForFunction(
    () => Array.from(document.querySelectorAll('iframe')).some(i => i.src.includes('localhost:5173')),
    { timeout: 30000 }
  );
  await page.waitForTimeout(3000);
  const f = page.frames().find(x => x.url().startsWith('http://localhost:5173'));
  const errFrame = page.frames().find(x => x.url().startsWith('chrome-error'));
  return {
    loaded: !!f,
    blockedByLNA: !!errFrame,
    title: f ? await f.title() : null,
  };
}

If blockedByLNA: true, the Chromium flag isn't taking effect — confirm --config=.playwright-config.json was passed.

Console error filter

The Fabric portal emits its own errors that are not app bugs. Treat them as portal noise and ignore them. Only errors whose source origin matches the embedded app server’s origin count as app errors.

See references/fabric-embed.md for the full frame walker, classifyConsoleMessages helper, and troubleshooting matrix.

Spec Files

Add spec files alongside source files as needed — for components, hooks, utilities, and query factory functions. Co-locate each spec file with the file it tests.

When to add spec files:

• Always for pure utility functions in src/lib/ — these are easiest to unit-test and most likely to have edge cases.
• Always for query factory functions in src/queries/ — verify that parameter combinations produce the correct query string, column metadata, and spec modifications.
• As needed — for hooks, test state transitions, returned values, and side effects using a React hooks testing library.
• As needed — for components, add spec files when the component contains non-trivial logic (e.g., conditional rendering, derived state, error states). Simple presentational components with no logic do not need a spec file.

Key rules:

• Never create a spec file just to satisfy coverage targets. Write tests only when they document expected behavior or guard against regressions.
• Tests must not use mock or hardcoded data to stand in for real query results — use representative fixture data that matches the real column shape.
• Keep each spec focused on one unit; do not write integration tests that span multiple layers.