# Eval Generator Gen Pages

> Generate a three-layer eval suite (presence checks + unit tests + security analysis) for Power Apps Generative Pages in Model-Driven Apps. Generative pages are single-file React/TypeScript components using Fluent UI V9 and the dataApi prop for Dataverse operations — NOT Code Apps, NOT PCF components. Code may be generated by the App Agent in make.powerapps.com or by AI code generation tools (Claude Code, GitHub Copilot CLI) using the /genpage skill. Always produces: evals/manifest.json, evals/presence/<id>.check.ts, evals/unit/<feature>.test.tsx, evals/runner/run-evals.ts, evals/runner/presence-runner.ts, evals/runner/security-runner.ts, and evals/dashboard/index.html. Triggers: 'eval my generative page', 'generate evals for gen page', 'review my genpage code', 'test my generative page', 'check my model driven app page', 'eval generator gen pages'.

- Skill: `microsoft/eval-generator-gen-pages` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add microsoft/eval-generator-gen-pages`
- Raw SKILL.md: https://api.skillmd.com/api/skills/microsoft/eval-generator-gen-pages/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Web & Frontend
- Author: Microsoft (https://skillmd.com/u/microsoft)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/microsoft/eval-generator-gen-pages

---


# Eval Generator — Power Apps Generative Pages

## Purpose

Answer: **"Is this Generative Page correctly structured, does the logic work, and are there security risks?"**

Three eval layers are always generated:
1. **Presence** — static checks that assert code artifacts exist and follow Generative Page conventions
2. **Unit** — Vitest tests for business logic using a mocked `dataApi`
3. **Security** — static analysis across 14 security categories specific to gen page patterns

All three are mandatory.

---

## Generative Page Stack — Critical Assumptions

Generative Pages are **single-file React/TypeScript components** embedded in Model-Driven Apps. They are NOT Code Apps, NOT PCF components. The architecture is fundamentally different from both.

| Concern | Technology |
|---|---|
| Runtime | React 17 + TypeScript, compiled and hosted by Power Apps |
| UI components | **Fluent UI V9** — `@fluentui/react-components` |
| Data access | **`dataApi` prop** — passed into the component by the Power Apps host |
| Data operations | `dataApi.queryTable()`, `dataApi.retrieveRow()`, `dataApi.createRow()`, `dataApi.updateRow()`, `dataApi.deleteRow()`, `dataApi.getChoices()` |
| Data source | **Dataverse only** (up to 6 tables per page) |
| Input params | `props.recordId`, `props.entityName`, `props.data` (optional, configured in page definition) |
| File structure | **Single `.tsx` file** — `export default GeneratedComponent` |
| Type generation | `pac model genpage generate-types` → local `*.d.ts` files |
| Deployment | `pac model genpage upload` via PAC CLI |
| Generated by | App Agent (make.powerapps.com) **or** AI code generation tools (Claude Code / GitHub Copilot CLI with `/genpage` skill) |
| Package manager | npm (for local development and eval harness) |
| Test runner | Vitest + @testing-library/react |

> ⚠️ **NEVER look for**: `power.config.json`, `src/generated/services/`, `@microsoft/power-apps`, `@microsoft/powerapps-component-framework`, `context.webAPI.*`, `context.parameters`, `initialize()`, `PowerProvider`. These are Code App or PCF patterns — they do not exist in Generative Pages.

> ⚠️ **ALWAYS look for**: `props.dataApi` or `dataApi` usage, `@fluentui/react-components` imports, `export default` component with React TSX, Dataverse operation patterns via `dataApi.*`.

---

## Step 0 — Snapshot Previous Results

> **Run this first, before collecting inputs or writing any files.**

1. Check if `evals/results/latest.json` exists relative to the project directory.
2. If it **does** exist:
   - Read it.
   - Count files matching `evals/results/snapshots/iter-*.json` to determine N (next iteration = count + 1).
   - Write a copy to `evals/results/snapshots/iter-<N>-<ISO-timestamp>.json`.
   - Update `evals/results/snapshots/index.json` (create if absent, append if present).
3. If it does **not** exist — skip silently. This is the first invocation.

---

## Step 0b — Collect Inputs

Ask the user (via `m_ask_user` for structured choices) for:

1. **Generative Page file or directory** — one of:
   - Path to a single `.tsx` file (the gen page)
   - Path to a directory containing one or more `.tsx` gen page files
   - Current working directory (default if already in the right place)

2. **Requirements document** — OPTIONAL. One of:
   - Local file path (`.md`, `.txt`, `.docx`)
   - OneDrive / SharePoint URL
   - Paste / describe in chat
   - **None / Skip** — features will be derived from code review (Step 1b)

3. **Eval output mode** (optional):
   - `scaffold+write` — write all eval files into an `evals/` directory (default ✅)
   - `describe-only` — print what would be generated, no file writes

> ⚠️ Ask ALL questions in a SINGLE `m_ask_user` call before doing any file work.

**Determine project directory:** If user provided a single file, `PROJECT_DIR` = the file's parent directory. If a directory was given, `PROJECT_DIR` = that directory. All `evals/` output goes into `PROJECT_DIR/evals/`.

---

## Step 1 — Read Requirements Document (if provided)

> **If no requirements document was provided, skip to Step 1b.**

### Local `.md` or `.txt`
Use the `view` tool.

### Local `.docx`
Use the `docx` skill. Call `m_get_skill('docx')` first.

### OneDrive / SharePoint URL
Use `m365_download_file` (resolve file ID with `m365_search_files` first).

---

## Step 1b — Code-Review Fallback (No BRD)

> **Run this step when no requirements document was provided, AND on every re-run.**

Read all `.tsx` files in the project directory (non-recursively first, then `src/` if present).

For each gen page file, use `view` and `grep` to identify:

| Signal | Derived Feature |
|---|---|
| `dataApi.queryTable('tableName', ...)` | One feature per distinct `tableName` — data list/grid |
| `dataApi.retrieveRow('tableName', ...)` | Detail/record view feature |
| `dataApi.createRow('tableName', ...)` | Create/form feature |
| `dataApi.updateRow('tableName', ...)` | Edit/update feature |
| `dataApi.deleteRow('tableName', ...)` | Delete/remove feature |
| `dataApi.getChoices(...)` | Choice field rendering feature |
| `props.recordId` or `props.entityName` | Input parameter handling feature |
| Named React sections / major UI blocks | UI composition features (dashboards, cards, grids) |
| `useState` / `useEffect` clusters | State management features |
| Localization / translation dictionary | Localization feature |
| `loadMoreRows` / pagination | Pagination feature |
| Filter/search inputs | Search/filter feature |
| Chart / visualization components | Data visualization feature |

Synthesize a feature entry per logical unit with `id`, `title`, `description`, `acceptance_criteria`, `priority`, and `tables` (Dataverse table names involved).

Set `manifest.generatedFrom = "code-review"`.

**Re-runs:** Repeat Step 1b to pick up new components or new dataApi calls. Match features by `id` — preserve `presence_status` for any existing `id`.

---

## Step 2 — Parse Requirements → Feature Manifest

Extract a structured feature list from the requirements document (Step 1) **or** use the code-derived list from Step 1b.

For each feature, capture:
- **id** — slugified identifier (e.g. `account-list`, `contact-detail`, `opportunity-create`)
- **title** — short label
- **description** — what the feature does
- **acceptance_criteria** — array of verifiable statements
- **priority** — `high` / `medium` / `low` (default `medium`)
- **tables** — Dataverse table logical names involved (e.g. `["account", "contact"]`)
- **operations** — dataApi operations expected: `["queryTable", "retrieveRow", "createRow"]`

Track in SQL:
```sql
CREATE TABLE IF NOT EXISTS features (
  id TEXT PRIMARY KEY,
  title TEXT,
  description TEXT,
  acceptance_criteria TEXT,
  priority TEXT DEFAULT 'medium',
  tables TEXT,
  operations TEXT,
  presence_status TEXT DEFAULT 'pending',
  mapped_file TEXT
);
```

Write `evals/manifest.json`:

```json
{
  "generated": "<ISO timestamp>",
  "project": "<page file name or directory basename>",
  "generatedFrom": "brd | code-review",
  "features": [
    {
      "id": "account-list",
      "title": "Account List",
      "description": "Display a paginated list of Account records using dataApi.queryTable",
      "acceptance_criteria": [
        "Calls dataApi.queryTable('account', { select: [...], pageSize: N })",
        "Shows loading state while data fetches",
        "Shows empty state when no records returned",
        "Handles dataApi errors with a user-friendly message"
      ],
      "priority": "high",
      "tables": ["account"],
      "operations": ["queryTable"],
      "layers": { "presence": true, "unit": true }
    }
  ]
}
```

---

## Step 3 — Audit the Generative Page File(s)

### 3a. Verify Generative Page structure

Use `glob` and `view` to confirm:

- At least one `.tsx` file exists with `export default` — the gen page component
- The file imports from `@fluentui/react-components` (Fluent UI V9)
- The component uses `props.dataApi` or destructures `dataApi` from props
- The component does NOT import from `@microsoft/power-apps` or `@microsoft/powerapps-component-framework`

If `@microsoft/power-apps` or `power.config.json` is found → stop and warn:
> "This appears to be a Code App, not a Generative Page. Use the `eval-generator-code-app` skill instead."

If `ComponentFramework` or `context.webAPI` is found → stop and warn:
> "This appears to be a PCF component. This skill only supports Power Apps Generative Pages."

### 3b. Map the gen page structure

Use `grep` on the `.tsx` file(s) to identify:

| Pattern | Role |
|---|---|
| `export default function \|export default const ` | Root component (required) |
| `import { .* } from '@fluentui/react-components'` | Fluent UI V9 components in use |
| `props\.dataApi\|const { dataApi }` | dataApi access pattern |
| `dataApi\.queryTable\s*\(` | Table query operations |
| `dataApi\.retrieveRow\s*\(` | Single record retrieval |
| `dataApi\.createRow\s*\(` | Record creation |
| `dataApi\.updateRow\s*\(` | Record update |
| `dataApi\.deleteRow\s*\(` | Record deletion |
| `dataApi\.getChoices\s*\(` | Choice column lookup |
| `props\.recordId\|props\.entityName\|props\.data` | Input parameter handling |
| `hasMoreRows\|loadMoreRows` | Pagination |
| `useState.*[Ll]oading\|isLoading` | Loading state |
| `useState.*[Ee]rror\|setError` | Error state |
| `\.length === 0\|rows\.length === 0` | Empty state |
| `try\s*{` | Error handling |
| `useEffect\s*\(` | Side effects / data fetching triggers |

Record matched line numbers in the SQL `features` table (`mapped_file`, `operations` columns).

### 3c. Flag structural issues

A feature is **Not Implemented** if its expected `dataApi.*` call is not found. Mark `presence_status = 'not_found'`.

Flag these as warnings (not blockers) if found:
- `dataApi.queryTable` called **without** a `select` array → performance risk (fetches all columns)
- `dataApi.*` calls **outside** a `try/catch` block → unhandled async errors
- No loading state detected → poor UX
- No empty state detected → poor UX

---

## Step 4 — Generate Presence Checks

Create `evals/presence/<feature-id>.check.ts` for each feature.

### Presence check template

```typescript
// evals/presence/<feature-id>.check.ts
// AUTO-GENERATED by eval-generator-gen-pages
// Feature: <feature title>
// Description: <feature description>

import { readFileSync, existsSync } from 'fs';
import { join, dirname } from 'path';
import { fileURLToPath } from 'url';
import { globSync } from 'glob';

// Path-anchored to this file's location — works correctly from any working directory
const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
const PROJECT_DIR = join(__dirname, '..', '..');  // evals/presence → evals → project dir

interface PresenceResult {
  featureId: string;
  checks: Array<{ name: string; passed: boolean; detail?: string }>;
}

export function check(): PresenceResult {
  const checks: PresenceResult['checks'] = [];

  // Locate gen page .tsx files (non-recursive, then src/ if present)
  const tsxFiles = [
    ...globSync(`${PROJECT_DIR}/*.tsx`),
    ...globSync(`${PROJECT_DIR}/src/**/*.tsx`),
  ];

  if (tsxFiles.length === 0) {
    checks.push({ name: 'gen page .tsx file exists', passed: false,
      detail: 'No .tsx file found in project directory. Expected a generative page component file.' });
    return { featureId: '<feature-id>', checks };
  }

  const allSource = tsxFiles.map(f => readFileSync(f, 'utf-8')).join('\n');

  // ── Check: dataApi is accessed ──
  const dataApiUsed = /props\.dataApi|const\s*\{[^}]*dataApi/.test(allSource);
  checks.push({
    name: 'dataApi prop accessed',
    passed: dataApiUsed,
    detail: dataApiUsed ? undefined : 'dataApi not found in props. Generative pages receive dataApi as a prop from the Power Apps host.',
  });

  // ── Check: dataApi.queryTable called for this feature's table ──
  const queryFound = /dataApi\.queryTable\s*\(\s*['"]<table-name>['"]/.test(allSource);
  checks.push({
    name: 'dataApi.queryTable called for <table-name>',
    passed: queryFound,
    detail: queryFound ? undefined : 'dataApi.queryTable("' + '<table-name>' + '", ...) not found. Feature may not be fetching data.',
  });

  // ── Check: select columns specified (performance best practice) ──
  const selectFound = /dataApi\.queryTable\s*\([^)]*select\s*:/.test(allSource);
  checks.push({
    name: 'select columns specified in queryTable (performance)',
    passed: selectFound,
    detail: selectFound ? undefined : 'queryTable called without select property — fetches ALL columns. Always specify select: ['col1', 'col2'] for performance.',
  });

  // ── Check: try/catch around dataApi call ──
  // Strategy: find dataApi.queryTable and check for surrounding try block
  const tryCatchFound = /try\s*\{[^}]*dataApi\.|dataApi\.[^;]*;\s*}\s*catch/.test(allSource.replace(/\n/g, ' '));
  checks.push({
    name: 'dataApi calls wrapped in try/catch',
    passed: tryCatchFound,
    detail: tryCatchFound ? undefined : 'No try/catch detected around dataApi calls. Unhandled promise rejections will crash the page.',
  });

  // ── Check: loading state ──
  const loadingState = /useState.*[Ll]oading|isLoading|isPending/.test(allSource);
  checks.push({
    name: 'loading state handled',
    passed: loadingState,
    detail: loadingState ? undefined : 'No loading state found. Users see blank content while data fetches.',
  });

  // ── Check: error state ──
  const errorState = /useState.*[Ee]rror|setError|errorMessage/.test(allSource);
  checks.push({
    name: 'error state handled',
    passed: errorState,
    detail: errorState ? undefined : 'No error state found. Connector failures will be silently swallowed.',
  });

  // ── Check: empty state ──
  const emptyState = /\.length\s*===\s*0|rows\.length\s*===\s*0|!rows\.length/.test(allSource);
  checks.push({
    name: 'empty state handled',
    passed: emptyState,
    detail: emptyState ? undefined : 'No empty-state check found. An empty result set may render a blank or broken UI.',
  });

  // ── Check: Fluent UI V9 imported ──
  const fluentV9 = /@fluentui\/react-components/.test(allSource);
  checks.push({
    name: 'Fluent UI V9 (@fluentui/react-components) imported',
    passed: fluentV9,
    detail: fluentV9 ? undefined : 'Fluent UI V9 not imported. Generative pages should use @fluentui/react-components for consistent MDA styling.',
  });

  return { featureId: '<feature-id>', checks };
}
```

### Generative Page-specific presence patterns

| Feature type | Checks to generate |
|---|---|
| **Data list / grid** | `dataApi.queryTable('tableName', {...})` called AND `select:` specified AND loading state AND empty state AND try/catch |
| **Detail / single record** | `dataApi.retrieveRow('tableName', { id: ..., select: [...] })` AND input param (`props.recordId`) handled |
| **Create form** | `dataApi.createRow('tableName', {...})` AND form validation AND submit disabled during save |
| **Edit form** | `dataApi.updateRow('tableName', rowId, {...})` AND rowId sourced from `props.recordId` or state |
| **Delete action** | `dataApi.deleteRow('tableName', rowId)` AND confirmation UI (dialog/button) AND error handling |
| **Choice fields** | `dataApi.getChoices('tableName-columnName')` AND choices rendered as dropdown or display label |
| **Input params** | `props.recordId`, `props.entityName`, or `props.data` destructured AND used in `useEffect` or initialization |
| **Pagination** | `hasMoreRows` checked AND `loadMoreRows()` callable AND loading state for next-page fetch |
| **Localization** | Translation dictionary or `useEffect` for language detection AND date/number locale formatting |
| **Search / filter** | Filter state connected to `dataApi.queryTable` `filter` option AND no raw user input in filter string |
| **Data visualization** | Chart component found AND data mapped from `dataApi.queryTable` result |
| **Error disclosure** | No `e.message` or `String(e)` set directly into user-visible state |

> ⚠️ Never generate presence checks that grep for `context.webAPI`, `initialize()`, `PowerProvider`, `src/generated/`, or `result.value` — these are Code App / PCF patterns and will never appear in a Generative Page.

---

## Step 5 — Generate Vitest Unit Tests

> ⛔ **This step is MANDATORY on every invocation.** Write real unit tests grounded in actual source code. Use `it.todo()` stubs only when a feature has no testable logic. Never skip this step — even on re-runs.

### Re-run behaviour
If `evals/unit/` already exists, **merge** — add tests for new features, preserve passing tests for existing ones. Never delete existing passing test files.

### File layout

```
evals/unit/
  helpers/
    setup.ts          → @testing-library/jest-dom matchers, global mocks
    mocks.ts          → mockDataApi factory (vi.fn() for all 6 methods)
    factories.ts      → createMock<TableName>Row() helpers for Dataverse rows
  <Feature>.test.tsx  → one file per logical feature group
```

### evals/unit/helpers/setup.ts

```typescript
// evals/unit/helpers/setup.ts
import '@testing-library/jest-dom';
import { vi } from 'vitest';

// Suppress FluentUI console warnings in tests
vi.spyOn(console, 'warn').mockImplementation(() => {});
```

### evals/unit/helpers/mocks.ts

```typescript
// evals/unit/helpers/mocks.ts
import { vi } from 'vitest';

// mockDataApi — the dataApi prop passed by Power Apps host, fully mocked
export function createMockDataApi() {
  return {
    queryTable: vi.fn().mockResolvedValue({ rows: [], hasMoreRows: false }),
    retrieveRow: vi.fn().mockResolvedValue({}),
    createRow: vi.fn().mockResolvedValue('new-row-id'),
    updateRow: vi.fn().mockResolvedValue(undefined),
    deleteRow: vi.fn().mockResolvedValue(undefined),
    getChoices: vi.fn().mockResolvedValue([]),
  };
}
```

### evals/unit/helpers/factories.ts

> ⚠️ **CRITICAL:** Before writing this file, read the actual gen page source to find what fields each `dataApi.queryTable` result is used for. Factory fields must match the actual column names the component reads from rows. Never invent field names.

```typescript
// evals/unit/helpers/factories.ts
// Grounded in actual source: fields match what the component reads from dataApi results

// Example — replace with actual fields from source
export function createMockAccountRow(overrides: Partial<Record<string, unknown>> = {}) {
  return {
    accountid: 'acc-001',
    name: 'Contoso Ltd',
    emailaddress1: 'info@contoso.com',
    telephone1: '555-0100',
    statecode: 0,
    ...overrides,
  };
}
```

### Unit test template — per feature

```typescript
// evals/unit/<feature>.test.tsx
// Feature: <feature title>
// Tests grounded in actual source: <gen page file name>

import { describe, it, expect, vi, beforeEach } from 'vitest';
import { render, screen, waitFor, fireEvent } from '@testing-library/react';
import { createMockDataApi } from './helpers/mocks';
import { createMockAccountRow } from './helpers/factories';

// Import the default export from the gen page file
// Adjust path relative to evals/unit/ — typically ../../PageName.tsx
import GeneratedComponent from '../../<PageName>.tsx';

describe('<feature-id>', () => {
  let dataApi: ReturnType<typeof createMockDataApi>;

  beforeEach(() => {
    dataApi = createMockDataApi();
  });

  it('<feature-id>: renders loading state while data fetches', async () => {
    // Delay resolution so we can catch the loading state
    dataApi.queryTable.mockReturnValue(new Promise(() => {}));
    render(<GeneratedComponent dataApi={dataApi} />);
    // queryBy* returns null instead of throwing — safe to use with || fallback
    // Adjust selectors to match the actual loading indicator in the component
    const loadingIndicator =
      screen.queryByRole('progressbar') ??
      screen.queryByText(/loading/i) ??
      screen.queryByLabelText(/loading/i);
    expect(loadingIndicator).toBeInTheDocument();
  });

  it('<feature-id>: renders account rows returned by dataApi.queryTable', async () => {
    const rows = [createMockAccountRow(), createMockAccountRow({ accountid: 'acc-002', name: 'Fabrikam Inc' })];
    dataApi.queryTable.mockResolvedValue({ rows, hasMoreRows: false });
    render(<GeneratedComponent dataApi={dataApi} />);
    await waitFor(() => {
      expect(screen.getByText('Contoso Ltd')).toBeInTheDocument();
      expect(screen.getByText('Fabrikam Inc')).toBeInTheDocument();
    });
  });

  it('<feature-id>: renders empty state when no rows returned', async () => {
    dataApi.queryTable.mockResolvedValue({ rows: [], hasMoreRows: false });
    render(<GeneratedComponent dataApi={dataApi} />);
    await waitFor(() => {
      // Adjust text to match actual empty-state message in the component
      expect(screen.getByText(/no records|no data|no results/i)).toBeInTheDocument();
    });
  });

  it('<feature-id>: renders error state when dataApi.queryTable rejects', async () => {
    dataApi.queryTable.mockRejectedValue(new Error('Dataverse unavailable'));
    render(<GeneratedComponent dataApi={dataApi} />);
    await waitFor(() => {
      // Adjust to match actual error message shown in component (should be generic, not e.message)
      expect(screen.getByText(/error|failed|unable/i)).toBeInTheDocument();
    });
  });

  it('<feature-id>: calls dataApi.queryTable with correct table name and select columns', async () => {
    dataApi.queryTable.mockResolvedValue({ rows: [], hasMoreRows: false });
    render(<GeneratedComponent dataApi={dataApi} />);
    await waitFor(() => {
      expect(dataApi.queryTable).toHaveBeenCalledWith(
        'account',   // table logical name — verify from source
        expect.objectContaining({ select: expect.arrayContaining(['name']) })
      );
    });
  });
});
```

### it() naming convention — CRITICAL for dashboard traceability

Every `it()` description **MUST** begin with the exact feature `id` from `manifest.json`, followed by a colon:

```typescript
it('account-list: renders loading state while data fetches', ...)    // ✅ correct
it('account list: renders loading state', ...)                        // ❌ wrong — space not hyphen
it('renders loading state', ...)                                      // ❌ wrong — no feature ID prefix
```

The runner extracts the feature ID using `/([\w-]+):\s*(.+)$/` and uses it to map test results to dashboard rows. A mismatch causes tests to pass but show "⏭ No tests" per feature in the dashboard.

### Post-generation verification (MANDATORY)

After writing all test files, grep each test file for each manifest feature ID to confirm traceability:

```
For each feature.id in manifest.json:
  grep -c "it\('" + feature.id + ":" in evals/unit/**/*.test.tsx
  → If count === 0: traceability broken — fix the it() names before proceeding
```

---

## Step 5b — Eval Package Setup

The gen page project may not have a `package.json` with test dependencies. Check:

1. Does `PROJECT_DIR/package.json` exist?
2. If yes, does it have `vitest`, `@testing-library/react`, `jsdom` in dev dependencies?
3. If no to either: create or patch `package.json` with the minimal eval harness.

### Minimal package.json for evals (create only if needed)

```json
{
  "name": "<project-basename>-evals",
  "private": true,
  "type": "module",
  "scripts": {
    "eval": "tsx evals/runner/run-evals.ts",
    "eval:presence": "tsx evals/runner/presence-runner.ts",
    "eval:security": "tsx evals/runner/security-runner.ts",
    "eval:unit": "vitest run --config evals/vitest.config.ts"
  },
  "devDependencies": {
    "@testing-library/jest-dom": "^6.0.0",
    "@testing-library/react": "^16.0.0",
    "@fluentui/react-components": "^9.0.0",
    "jsdom": "^25.0.0",
    "react": "^17.0.0",
    "react-dom": "^17.0.0",
    "@types/react": "^17.0.0",
    "@types/react-dom": "^17.0.0",
    "tsx": "^4.0.0",
    "vitest": "^2.0.0",
    "glob": "^11.0.0"
  }
}
```

> If `package.json` already exists, **append** only the missing dev dependencies and scripts — never overwrite existing ones.

After writing `package.json`, run `npm install` to install dependencies.

### evals/vitest.config.ts

```typescript
// evals/vitest.config.ts
import { defineConfig } from 'vitest/config';
import { resolve, dirname } from 'path';
import { fileURLToPath } from 'url';

const __dirname = dirname(fileURLToPath(import.meta.url));
const PROJECT_ROOT = resolve(__dirname, '..');   // evals/ → project root

export default defineConfig({
  root: PROJECT_ROOT,   // CRITICAL: anchors all paths to project root, not evals/
  test: {
    environment: 'jsdom',
    globals: true,
    setupFiles: [resolve(__dirname, 'unit/helpers/setup.ts')],
    include: ['evals/unit/**/*.test.{ts,tsx}'],
    reporters: ['json'],
    outputFile: resolve(__dirname, 'results/unit-results.json'),
  },
  resolve: {
    alias: {
      // Allow tests to import from the project root without relative ../../
      '@': PROJECT_ROOT,
    },
  },
});
```

> ⚠️ `root: PROJECT_ROOT` is mandatory. Without it, Vitest defaults `root` to `evals/`, causing all `setupFiles` and `include` paths to resolve from `evals/evals/...` — a path that doesn't exist.

---

## Step 6 — Generate Security Runner

Write `evals/runner/security-runner.ts`:

```typescript
#!/usr/bin/env tsx
// evals/runner/security-runner.ts
// Static security analysis for Power Apps Generative Pages

import { readFileSync } from 'fs';
import { join, dirname } from 'path';
import { fileURLToPath } from 'url';
import { globSync } from 'glob';

const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
const ROOT = join(__dirname, '..', '..');   // evals/runner → evals → project root

type Severity = 'critical' | 'high' | 'medium' | 'low';

interface SecurityFinding {
  id: string; category: string; severity: Severity;
  file: string; line: number; snippet: string; detail: string;
}

interface SecurityResult {
  layer: 'security';
  passed: number; failed: number; skipped: number;
  findings: SecurityFinding[];
}

const findings: SecurityFinding[] = [];
// Scan all .tsx/.ts files in the project (excluding evals/ itself)
const srcFiles = globSync(`${ROOT}/**/*.ts?(x)`, { ignore: `${ROOT}/evals/**` });

function scan(id: string, category: string, severity: Severity, pattern: RegExp, detail: string, exclude?: RegExp) {
  for (const file of srcFiles) {
    let content: string;
    try { content = readFileSync(file, 'utf-8'); } catch { continue; }
    const lines = content.split('\n');
    lines.forEach((line, i) => {
      if (pattern.test(line) && !(exclude && exclude.test(line))) {
        findings.push({ id, category, severity,
          file: file.replace(ROOT + '', '').replace(ROOT + '/', ''),
          line: i + 1, snippet: line.trim().slice(0, 120), detail });
      }
    });
  }
}

// GP-SEC-001: OData Injection — filter string built from user input
scan('GP-SEC-001', 'OData Injection', 'critical',
  /filter\s*:\s*[^,}]*['"`]\s*\+|filter\s*:\s*[^,}]*\$\{/,
  'OData filter string built by concatenation or template literal. User input in filter strings is an injection risk. Use fixed filter strings or validate/sanitize inputs.');

// GP-SEC-002: XSS — dangerouslySetInnerHTML or .innerHTML
scan('GP-SEC-002', 'XSS', 'critical',
  /dangerouslySetInnerHTML|\.innerHTML\s*=/,
  'Direct HTML injection. Use React text rendering or sanitize with DOMPurify. Fluent UI components handle escaping automatically — prefer those over raw HTML.');

// GP-SEC-003: Hardcoded secrets
// Exclude: (1) comment lines, (2) values that are purely alphabetical/kebab-case — real credentials
// always contain digits or special chars. This prevents false positives on TypeScript object maps
// where property keys like `token` hold UI label strings like 'skipToken' or 'badge-token'.
scan('GP-SEC-003', 'Hardcoded Secret', 'critical',
  /(?:apikey|api_key|password|secret|token|client_secret)\s*[:=]\s*['"`][^'"`\s]{6,}/i,
  'Possible hardcoded credential in source. Move secrets to environment variables. In Generative Pages, use Dataverse environment variables or Azure Key Vault references.',
  /^\s*\/\/|(?:apikey|api_key|password|secret|token|client_secret)\s*[:=]\s*['"`][a-zA-Z$_][a-zA-Z$_\-]*['"`]/i);

// GP-SEC-004: PII in console logs
// Exclude: single-string-only console.log calls (e.g. "Email field is required") — UI messages, not variable dumps.
scan('GP-SEC-004', 'PII in Logs', 'high',
  /console\.(log|warn|error|debug)\b.*\b(email|mail|userId|phone|address|password|token|DisplayName|firstname|lastname)/i,
  'Sensitive data written to the browser console. Remove PII from log statements — model-driven app users can open DevTools.',
  /console\.(log|warn|error|debug)\s*\(\s*(?:'[^']*'|"[^"]*"|`[^`]*`)\s*\)/i);

// GP-SEC-005: Raw error surfaced to UI state
scan('GP-SEC-005', 'Error Disclosure', 'high',
  /set\w+\(\s*(e\.message|String\(e\)|e\.toString\(\)|error\.message)/,
  'Raw error message surfaced directly to UI state. Show a generic user-friendly message instead (e.g., "Something went wrong. Please try again."). Technical details may reveal system internals.');

// GP-SEC-006: fetch() to non-Microsoft external URLs
scan('GP-SEC-006', 'Unsafe fetch() Call', 'high',
  /fetch\s*\(\s*['"`]https?:\/\//,
  'fetch() to an external URL. Generative pages should use dataApi methods for all Dataverse operations. External fetch calls bypass Power Platform governance and connector policies.',
  /\.dynamics\.com|\.microsoft\.com|\.microsoftonline\.com|\.sharepoint\.com/);

// GP-SEC-007: Client-side auth bypass
scan('GP-SEC-007', 'Client-Side Auth Bypass', 'high',
  /\b(isAdmin|userRole|hasPermission|canEdit|canDelete|isOwner)\b.*&&.*dataApi\.(create|update|delete)/i,
  'Role check in UI code but no server-side guard before dataApi write. Power Apps security roles should be the authoritative gate — UI checks alone can be bypassed.');

// GP-SEC-008: PII in browser storage
// Removed bare 'user' — too generic (matches user-preferences, user-theme etc.)
scan('GP-SEC-008', 'PII in Browser Storage', 'medium',
  /(localStorage|sessionStorage)\.setItem\s*\([^,]*(?:email|userId|userEmail|userName|token|auth|authToken|mail|profile|password|DisplayName)/i,
  'PII or auth data stored in localStorage/sessionStorage. This data persists across sessions and is accessible to any script on the page. Use Dataverse or in-memory state instead.');

// GP-SEC-009: Unvalidated user input in dataApi filter
// Removed 'value' and 'text' — too generic.
scan('GP-SEC-009', 'Unvalidated Input in Filter', 'medium',
  /dataApi\.(queryTable|retrieveRow)\s*\([^)]*\b(input|userInput|searchTerm|searchQuery|filterValue|rawInput)\b/i,
  'User input variable passed directly to dataApi query. Validate and sanitize before using in queries — while Dataverse has server-side guards, defence-in-depth is best practice.');

// GP-SEC-010: Sensitive data in React state
scan('GP-SEC-010', 'Sensitive Data in State', 'low',
  /useState[^)]*\b(password|token|secret|apiKey|api_key)\b/i,
  'Sensitive data stored in React component state. State is accessible via React DevTools. Consider alternatives like Dataverse secure columns or environment variables.');

// GP-SEC-011: Dynamic code execution
scan('GP-SEC-011', 'Dynamic Code Execution', 'critical',
  /\beval\s*\(/,
  'eval() executes a string as code. If user-controlled content can reach this call, it is a code injection vulnerability. Replace with a safer pattern.');

scan('GP-SEC-011', 'Dynamic Code Execution', 'critical',
  /new\s+Function\s*\(/,
  'new Function() constructs executable code from a string — equivalent to eval(). Avoid or ensure the source string is never user-controlled.');

scan('GP-SEC-011', 'Dynamic Code Execution', 'high',
  /(setTimeout|setInterval)\s*\(\s*['"`]/,
  'setTimeout/setInterval with a string argument evaluates it like eval(). Pass a function reference instead: setTimeout(() => doSomething(), delay).');

// GP-SEC-012: Broader external HTTP clients
scan('GP-SEC-012', 'Unsafe External HTTP Client', 'high',
  /axios\s*\.\s*(get|post|put|patch|delete|request)\s*\(\s*['"`]https?:\/\//i,
  'axios call to an external URL. Use dataApi methods for Dataverse access. External HTTP calls bypass Power Platform governance.',
  /\.dynamics\.com|\.microsoft\.com|\.microsoftonline\.com/);

scan('GP-SEC-012', 'Unsafe External HTTP Client', 'high',
  /new\s+XMLHttpRequest\s*\(\s*\)/,
  'XMLHttpRequest instantiation in a generative page. Dataverse access should always go through dataApi methods, not raw HTTP calls.');

// GP-SEC-013: Sensitive data in URL parameters
scan('GP-SEC-013', 'Sensitive URL Parameters', 'high',
  /useSearchParams[^;]*\b(token|auth|apikey|secret|password|credential)\b/i,
  'Sensitive parameter read from URL query string. Tokens and credentials must never appear in URLs — they are logged by browsers, proxies, and servers.');

scan('GP-SEC-013', 'Sensitive URL Parameters', 'high',
  /[?&](token|auth|apikey|api_key|secret|password|userId|user_id|credential)=/i,
  'Sensitive parameter embedded in a URL string. Use Dataverse record context (props.recordId, props.entityName) for passing identifiers — never URL query strings.');

// GP-SEC-014: queryTable without select (excessive data retrieval)
scan('GP-SEC-014', 'Excessive Data Retrieval', 'medium',
  /dataApi\.queryTable\s*\(\s*['"`]\w+['"`]\s*,\s*\{(?![^}]*select\s*:)/,
  'dataApi.queryTable called without a select property — retrieves ALL columns from the table. This is a performance risk and may expose columns the UI does not need. Always specify select: ['col1', 'col2'].');

const ALL_CATEGORIES = [
  'GP-SEC-001','GP-SEC-002','GP-SEC-003','GP-SEC-004','GP-SEC-005','GP-SEC-006',
  'GP-SEC-007','GP-SEC-008','GP-SEC-009','GP-SEC-010','GP-SEC-011','GP-SEC-012',
  'GP-SEC-013','GP-SEC-014',
];

const hitIds = new Set(findings.map(f => f.id));
const result: SecurityResult = {
  layer: 'security',
  passed: ALL_CATEGORIES.filter(c => !hitIds.has(c)).length,
  failed: hitIds.size,
  skipped: 0,
  findings,
};
process.stdout.write(JSON.stringify(result));
```

---

## Step 7 — Generate the Eval Runners

### evals/runner/presence-runner.ts

```typescript
#!/usr/bin/env tsx
// evals/runner/presence-runner.ts

import { join, dirname } from 'path';
import { fileURLToPath } from 'url';
import { pathToFileURL } from 'url';
import { globSync } from 'glob';

const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
const PRESENCE_DIR = join(__dirname, '..', 'presence');

interface LayerResult {
  layer: 'presence';
  passed: number; failed: number; skipped: number;
  details: Array<{ featureId: string; name: string; status: 'pass' | 'fail' | 'skip'; detail?: string }>;
}

const checkFiles = globSync(`${PRESENCE_DIR}/*.check.ts`).sort();
const details: LayerResult['details'] = [];
let passed = 0, failed = 0;

for (const file of checkFiles) {
  const mod = await import(pathToFileURL(file).href);
  const result = mod.check();
  for (const c of result.checks) {
    const status = c.passed ? 'pass' : 'fail';
    if (c.passed) passed++; else failed++;
    details.push({ featureId: result.featureId, name: c.name, status, detail: c.detail });
  }
}

const output: LayerResult = { layer: 'presence', passed, failed, skipped: 0, details };
process.stdout.write(JSON.stringify(output));
```

### evals/runner/run-evals.ts

> ⚠️ CRITICAL: Copy the code below EXACTLY. Do not simplify or restructure.

```typescript
#!/usr/bin/env tsx
// evals/runner/run-evals.ts
// Orchestrates all three eval layers and bakes results into the dashboard.

import { execSync, execFileSync } from 'child_process';
import { readFileSync, writeFileSync, mkdirSync, existsSync, copyFileSync } from 'fs';
import { join, dirname, resolve } from 'path';
import { fileURLToPath, pathToFileURL } from 'url';

const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);
const ROOT = join(__dirname, '..', '..');  // evals/runner → evals → project root
const RESULTS_DIR = join(ROOT, 'evals', 'results');
mkdirSync(RESULTS_DIR, { recursive: true });

function run(cmd: string, label: string): string {
  try {
    return execSync(cmd, { cwd: ROOT, encoding: 'utf-8', stdio: 'pipe' });
  } catch (e: any) {
    // Some runners exit 1 on findings (security) or failures (vitest) — capture output anyway
    return e.stdout || e.output?.[1] || '';
  }
}

console.log('▶ Running presence checks...');
const presOut = run('npx tsx evals/runner/presence-runner.ts', 'presence');

console.log('▶ Running unit tests...');
const unitRawOut = run('npx vitest run --config evals/vitest.config.ts --reporter=json', 'unit');

console.log('▶ Running security scan...');
const secOut = run('npx tsx evals/runner/security-runner.ts', 'security');

// Parse presence
let presLayer: any = { layer: 'presence', passed: 0, failed: 0, skipped: 0, details: [] };
try { presLayer = JSON.parse(presOut); } catch { console.warn('Could not parse presence output'); }

// Parse unit (Vitest JSON format)
let unitLayer: any = { layer: 'unit', passed: 0, failed: 0, skipped: 0, details: [] };
try {
  const vitestJson = JSON.parse(unitRawOut);
  const details: any[] = [];
  let p = 0, f = 0, s = 0;
  // Vitest JSON: testResults[].assertionResults[]
  for (const suite of vitestJson.testResults || []) {
    for (const t of suite.assertionResults || []) {
      const m = t.fullName?.match(/([\w-]+):\s*(.+)$/);
      const featureId = m ? m[1] : 'unknown';
      const name = t.fullName || t.title;
      const status = t.status === 'passed' ? 'pass' : t.status === 'pending' ? 'skip' : 'fail';
      const detail = t.failureMessages?.join('\n').slice(0, 500) || undefined;
      details.push({ featureId, name, status, detail });
      if (status === 'pass') p++; else if (status === 'skip') s++; else f++;
    }
  }
  unitLayer = { layer: 'unit', passed: p, failed: f, skipped: s, details };
} catch { console.warn('Could not parse vitest output'); }

// Parse security
let secLayer: any = { layer: 'security', passed: 0, failed: 0, skipped: 0, findings: [] };
try { secLayer = JSON.parse(secOut); } catch { console.warn('Could not parse security output'); }

// Load manifest for feature count
let manifest: any = { features: [] };
const manifestPath = join(ROOT, 'evals', 'manifest.json');
try { manifest = JSON.parse(readFileSync(manifestPath, 'utf-8')); } catch {}

// Load snapshots for iteration history
const SNAP_DIR = join(RESULTS_DIR, 'snapshots');
let snapshots: any[] = [];
try {
  const idx = JSON.parse(readFileSync(join(SNAP_DIR, 'index.json'), 'utf-8'));
  snapshots = idx.map((s: any) => {
    try { return JSON.parse(readFileSync(join(SNAP_DIR, s.file), 'utf-8')); } catch { return null; }
  }).filter(Boolean);
} catch {}

const timestamp = new Date().toISOString();
const evalResults = {
  timestamp,
  project: manifest.project || 'Generative Page',
  iteration: snapshots.length + 1,
  layers: [presLayer, unitLayer, secLayer],
  snapshots,
  summary: {
    totalFeatures: manifest.features.length,
    openBRDGaps: manifest.openBRDGaps || [],
    blockers: manifest.blockers || [],
    securityFindings: secLayer.findings || [],
  },
};

// Write latest.json
writeFileSync(join(RESULTS_DIR, 'latest.json'), JSON.stringify(evalResults, null, 2));

// Bake results into dashboard
const dashPath = join(ROOT, 'evals', 'dashboard', 'index.html');
if (existsSync(dashPath)) {
  let html = readFileSync(dashPath, 'utf-8');
  const dataBlock = `/* BAKED DATA — replaced by npm run eval — do not edit manually */\nvar DATA = ${JSON.stringify(evalResults)};\n/* END BAKED DATA */`;
  html = html.replace(/\/\* BAKED DATA[\s\S]*?END BAKED DATA \*\//m, dataBlock);
  writeFileSync(dashPath, html);
  console.log(`\n✅ Dashboard updated: evals/dashboard/index.html`);
}

// Summary
const totalFailed = presLayer.failed + unitLayer.failed;
console.log(`\nPresence: ${presLayer.passed} passed, ${presLayer.failed} failed`);
console.log(`Unit:     ${unitLa

…(truncated)