# Build Tools Tests

> Build integration tests for MCP tool collections. Reads .discover.json and creates test setup, builders, helpers, and test files per collection. Use after running '/build-tools'.

- Skill: `umbraco/build-tools-tests` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add umbraco/build-tools-tests`
- Raw SKILL.md: https://api.skillmd.com/api/skills/umbraco/build-tools-tests/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: umbraco (https://skillmd.com/u/umbraco)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/umbraco/build-tools-tests

---


# Build Tests

Generate integration tests for MCP tool collections created by `/build-tools`. This skill reads `.discover.json` and the existing tool files, then builds test infrastructure and test files one collection at a time.

**IMPORTANT: This skill ONLY creates files inside `__tests__/` directories — test files, setup, builders, and helpers. Do NOT create or modify ANY other files. This means: no tool files, no collection indexes, no registrations, no API client files (`src/umbraco-api/api/`), no generated code (`src/umbraco-api/api/generated/`), no mock handlers (`src/mocks/`). If existing code doesn't support what you need, work within the constraints — do not modify it.**

## Prerequisites

Before running, ensure:
1. You have run `/build-tools` (tool collections exist in `src/umbraco-api/tools/`)
2. The project compiles: `npm run compile`
3. The Umbraco instance is running
4. An API user exists — remind the user: **"You need to create an API user via the Umbraco backoffice UI: Settings > Users, with Client ID `umbraco-back-office-mcp` and Client Secret `1234567890`"**

## Arguments

- No arguments: build tests for all collections from `.discover.json`
- Single collection name: build tests only for that collection (e.g. `/build-tools-tests form`)

## Agents

This skill orchestrates the following agents — use them for the relevant steps:

| Agent | When to use |
|-------|-------------|
| `test-builder-helper-creator` | Creating builders and helpers (Steps 4-5) |
| `integration-test-creator` | Creating test files (Step 7) |
| `integration-test-validator` | Validating test quality (Step 8) |

## Critical Rules

**ONE FILE AT A TIME.** This applies to ALL files — builders, helpers, builder tests, and tool tests. After creating any file:
1. Compile: `npm run compile`
2. If it has tests, run them: `npm test -- path/to/file.test.ts`
3. Fix any failures
4. Only then create the next file

**RUN COMMANDS SEPARATELY.** Always run compile and test as separate Bash calls. Never chain them with `&&`.

**NEVER:**
- Create multiple files at once
- Move on while a compile error or test failure exists
- Skip the compile step
- Assume anything works without running it
- Chain commands with `&&` (run each command separately)

**ONE TEST FILE PER TOOL.** Every tool gets its own test file. Never combine tests for multiple tools into one file.

**SNAPSHOT TESTING PREFERRED.** Use `createSnapshotResult` from `@umbraco-cms/mcp-server-sdk/testing` with `toMatchSnapshot()` for success responses. Only use assertion testing (`expect(x).toBe(y)`) for error cases where `isError` is checked.

**REAL API — NO MOCKING.** These are integration tests that run against a real Umbraco instance. Do NOT set `USE_MOCK_API`. Do NOT create, modify, or reference anything in `src/mocks/`. Do NOT import `server` from mocks. Do NOT add MSW handlers. Do NOT use any mocking framework. The tests call tool handlers directly and those handlers call the real API. If a test fails, the fix is in the test or the tool — never add mock infrastructure.

## Test Data Gotchas

These fail in ways that look like tool bugs but are really test-data bugs. Check here first when
a builder can't create its entity.

**Password fields.** Umbraco's default password policy rejects weak values, and the API returns a
validation `ProblemDetails` that reads like a broken tool. Any builder creating users, members, or
anything else with a password must use a value that is:

- at least 10 characters
- contains at least one digit
- contains at least one non-alphanumeric symbol

```typescript
const TEST_PASSWORD = "_TestPassword1!";
```

Put it in a `TEST_` constant at the head of the builder — never inline a short password, and never
weaken the test to work around the rejection.

**Email and login uniqueness.** Users and members are unique by email/username. Derive them from
the `TEST_` name constant (`_test-user-1@example.invalid`) so cleanup by prefix reliably removes
them, otherwise a failed run leaves data that breaks the next one.

**Flattened parent/target params.** Tools that flatten a nested reference (`parentId` → `parent: { id }`,
`path` → `parent: { path }`) need at least one test that actually supplies it. The API accepts
`parent: null` happily, so root-only coverage passes even when the transform is broken. See the
schema flattening section of `/mcp-patterns`.

**Trees and folders need real ancestry.** `ancestors`/`children`/`root` and folder tools can't be
tested against a single flat entity — the builder must be able to create a child under a known
parent. If the builder has no `withParentId()`, add it before writing those tests.

## Umbraco Instance Management

If testing hits roadblocks — builders can't create data, APIs reject requests due to missing configuration, or features aren't available — you are able to manipulate the Umbraco instance to your needs. You can add connection strings, change settings, install packages, or even write C# code in `demo-site/`. **Read `instance-management.md` in this skill directory for the full process and concrete examples.**

## Workflow

Process **one collection at a time**. Complete each collection fully before starting the next.

### Step 0: Read Discovery Manifest

Read `.discover.json` from the project root:

```json
{
  "apiName": "Umbraco Forms Management API",
  "swaggerUrl": "https://localhost:44324/umbraco/swagger/forms-management/swagger.json",
  "baseUrl": "https://localhost:44324",
  "collections": ["form", "form-template", "field-type", "folder"]
}
```

If an argument was provided, filter to only that collection. If `.discover.json` doesn't exist, tell the user to run `npx @umbraco-cms/create-umbraco-mcp-server discover` first.

### Step 1: Understand Existing Tools

For each collection, read:
- `src/umbraco-api/tools/{collection}/index.ts` — to get the list of tools
- Each tool file — to understand input schemas, handler logic, and entity names
- `src/umbraco-api/api/generated/` — to identify the API client function and Zod schemas

If `src/umbraco-api/tools/{collection}/index.ts` doesn't exist, skip — tell the user to run `/build-tools` first.

### Step 2: Check for Existing Tests

**Skip if `src/umbraco-api/tools/{collection}/__tests__/setup.ts` already exists** — tests have already been created for this collection.

### Step 3: Per Collection — Create Test Setup

Create `src/umbraco-api/tools/{collection}/__tests__/setup.ts`:

```typescript
import {
  setupTestEnvironment,
  createMockRequestHandlerExtra,
  createSnapshotResult,
} from "@umbraco-cms/mcp-server-sdk/testing";
import { configureApiClient, initializeUmbracoFetch } from "@umbraco-cms/mcp-server-sdk";
import { getYourAPI } from "../../../../api/generated/yourApi.js";
import { EntityBuilder } from "./helpers/{entity}-builder.js";
import { EntityTestHelper } from "./helpers/{entity}-test-helper.js";

// Initialize fetch with credentials — required for integration tests hitting the real API
initializeUmbracoFetch({
  baseUrl: process.env.UMBRACO_BASE_URL!,
  clientId: process.env.UMBRACO_CLIENT_ID!,
  clientSecret: process.env.UMBRACO_CLIENT_SECRET!,
});

configureApiClient(() => getYourAPI());

export {
  setupTestEnvironment,
  createMockRequestHandlerExtra,
  createSnapshotResult,
  EntityBuilder,
  EntityTestHelper,
};
```

**Key rules:**
- Import the correct API client getter from `src/umbraco-api/api/generated/`
- `initializeUmbracoFetch` MUST be called before `configureApiClient` — it sets up the authenticated fetch layer
- Do NOT set `USE_MOCK_API` — these tests run against the real Umbraco instance
- Export `createSnapshotResult` for snapshot testing
- Re-export builders and helpers so test files have a single import

**Compile after creating:** `npm run compile`. Fix errors before continuing.

**Read-only collections:** If the collection has no create or delete operations (e.g. analytics — only GET/query tools), skip steps 4-6 (builder, helper, builder tests). These steps create test data lifecycle management which isn't needed for read-only collections. Proceed directly to step 7 (integration tests).

### Step 4: Per Collection — Create Test Builder

Use the `test-builder-helper-creator` agent.

Create `src/umbraco-api/tools/{collection}/__tests__/helpers/{entity}-builder.ts`:

```typescript
import { getYourAPI } from "../../../../api/generated/yourApi.js";
import { CAPTURE_RAW_HTTP_RESPONSE } from "@umbraco-cms/mcp-server-sdk";

const TEST_ENTITY_NAME = "_Test Entity";

interface EntityModel {
  name: string;
  // ... fields matching the POST body schema from the generated *.zod.ts
}

export class EntityBuilder {
  private model: EntityModel = {
    name: TEST_ENTITY_NAME,
  };

  private createdId?: string;

  withName(name: string): this {
    this.model.name = name;
    return this;
  }

  build(): EntityModel {
    return { ...this.model };
  }

  async create(): Promise<this> {
    const client = getYourAPI();
    // Call the API client's POST method directly (NOT the tool handler)
    const response: any = await client.postEntity(
      this.model as any,
      CAPTURE_RAW_HTTP_RESPONSE,
    );

    if (response.status !== 201) {
      const errorBody = await response.data?.detail || `HTTP ${response.status}`;
      throw new Error(`Failed to create entity: ${errorBody}`);
    }

    // Extract ID from Location header (Umbraco convention: /api/v1/entity/{id})
    const location = response.headers?.get?.("location") || response.headers?.location;
    this.createdId = location?.split("/").pop();

    return this;
  }

  async delete(): Promise<void> {
    if (!this.createdId) return;
    const client = getYourAPI();
    try {
      await client.deleteEntityById(this.createdId, CAPTURE_RAW_HTTP_RESPONSE);
    } catch {
      // Ignore delete failures in cleanup
    }
    this.createdId = undefined;
  }

  getId(): string {
    if (!this.createdId) {
      throw new Error("Entity not created yet. Call create() first.");
    }
    return this.createdId;
  }
}
```

**Key rules:**
- Fluent interface — all `withX` methods return `this`
- `build()` returns the model, `create()` calls the API
- Store created ID for use in tests
- Use `TEST_` prefix for constants
- Match the create tool's input schema for the model fields

**Compile after creating:** `npm run compile`. Fix errors before continuing.

### Step 5: Per Collection — Create Test Helper

Use the `test-builder-helper-creator` agent.

Create `src/umbraco-api/tools/{collection}/__tests__/helpers/{entity}-test-helper.ts`:

```typescript
export class EntityTestHelper {
  static async findByName(name: string): Promise<any | undefined> {
    // Use list tool or API client to find entity
  }

  static async cleanup(namePrefix: string): Promise<void> {
    // List entities and delete those matching prefix
  }

  static normalizeIds(data: any): any {
    // Replace UUIDs with zeroed placeholder for snapshots
    if (Array.isArray(data)) {
      return data.map(item => this.normalizeIds(item));
    }
    if (data && typeof data === "object") {
      const normalized = { ...data };
      if (normalized.id) {
        normalized.id = "00000000-0000-0000-0000-000000000000";
      }
      for (const key of Object.keys(normalized)) {
        if (typeof normalized[key] === "object") {
          normalized[key] = this.normalizeIds(normalized[key]);
        }
      }
      return normalized;
    }
    return data;
  }
}
```

**Compile after creating:** `npm run compile`. Fix errors before continuing.

### Step 6: Per Collection — Create Builder Tests

Create `src/umbraco-api/tools/{collection}/__tests__/helpers/{entity}-builder.test.ts`:

```typescript
import {
  setupTestEnvironment,
  createMockRequestHandlerExtra,
  EntityBuilder,
  EntityTestHelper,
} from "../setup.js";

const TEST_NAME = "_Test Builder Entity";

describe("EntityBuilder", () => {
  setupTestEnvironment();

  let builder: EntityBuilder;

  afterEach(async () => {
    // Always clean up created entities to prevent conflicts with other test files
    if (builder) await builder.delete();
    await EntityTestHelper.cleanup(TEST_NAME);
  });

  it("should create entity with builder", async () => {
    const builder = await new EntityBuilder()
      .withName(TEST_NAME)
      .create();

    expect(builder.getId()).toBeDefined();

    const found = await EntityTestHelper.findByName(TEST_NAME);
    expect(found).toBeDefined();
    expect(found?.name).toBe(TEST_NAME);
  });
});
```

**After creating:**
1. Compile: `npm run compile`
2. Run: `npm test -- __tests__/{collection}/{entity}-builder.test.ts`
3. Fix any failures before continuing

### Step 7: Per Collection — Create Integration Tests

Use the `integration-test-creator` agent.

Create **one test file per tool**. Each tool gets its own `.test.ts` file. Create and run each sequentially.

#### Build order within a collection

Each stage depends on the previous one working, so build them in this order and don't move on
until the current stage passes:

| # | Stage | Why it comes here |
|---|-------|-------------------|
| 1 | Builder (+ its own builder test) | Nothing else can create test data until this works |
| 2 | The single `create-{entity}` test | Proves the POST payload and the ID extraction from the Location header |
| 3 | The rest of CRUD — `get`, `list`, `update`, `delete` | These consume the builder's ID; a failure here is the tool, not the data |
| 4 | Items/tree tools — `ancestors`, `children`, `root` | Need a parent/child pair, so the builder must already support `withParentId()` |
| 5 | Folder tools | Need both a folder and an entity inside it — the most test data of any stage |

Skipping ahead (e.g. writing tree tests before `create` passes) means debugging two unknowns at
once. If a stage is genuinely blocked, note it and continue — don't fabricate data to get past it.

#### Snapshot test pattern (preferred for success cases)

```typescript
import {
  setupTestEnvironment,
  createMockRequestHandlerExtra,
  createSnapshotResult,
  EntityBuilder,
} from "./setup.js";
import getEntityTool from "../get/get-entity.js";

describe("get-entity", () => {
  setupTestEnvironment();

  let builder: EntityBuilder;

  afterEach(async () => {
    // Clean up test data after each test to prevent conflicts
    if (builder) await builder.delete();
  });

  it("should return entity by ID", async () => {
    const context = createMockRequestHandlerExtra();
    builder = await new EntityBuilder()
      .withName("_Test Get Entity")
      .create();

    const result = await getEntityTool.handler(
      { id: builder.getId() },
      context
    );

    expect(
      createSnapshotResult(result, builder.getId())
    ).toMatchSnapshot();
  });

  it("should return error for non-existent ID", async () => {
    const context = createMockRequestHandlerExtra();

    const result = await getEntityTool.handler(
      { id: "00000000-0000-0000-0000-000000000000" },
      context
    );

    expect(result.isError).toBe(true);
  });
});
```

**Key rules:**
- **ALWAYS** use `createSnapshotResult(result, id)` for success responses — it normalizes IDs, dates, and dynamic values
- Pass the created entity's ID as second argument to `createSnapshotResult` so it gets normalized
- Use `toMatchSnapshot()` — not `toMatchInlineSnapshot()`
- Only use assertion testing (`expect(x).toBe(y)`) for error cases
- Use builders to create test data when the test needs existing entities

**NEVER access result properties directly.** The following patterns are WRONG and will fail:
```typescript
// WRONG — result.content may be undefined
const data = JSON.parse(result.content[0].text);

// WRONG — result structure varies by output mode
expect(result.content).toContain("something");
```

Always use the snapshot helper:
```typescript
// CORRECT — handles all output modes
expect(createSnapshotResult(result, builder.getId())).toMatchSnapshot();
```

#### Cursor pagination in tests

Paginated tools (those with `skip`/`take` in their input schema) are converted to cursor pagination by `withCursorPagination`, which `withStandardDecorators` **already applies** — and every tool file exports `withStandardDecorators(tool)`. So an imported tool's handler already accepts `cursor` and no longer accepts `skip`/`take`.

**Do not wrap an imported tool again.** `withCursorPagination` only transforms a tool whose `inputSchema` still has both `skip` and `take`; given an already-decorated tool it returns it unchanged. That makes `withCursorPagination({ ...tool, pageSize: N })` a no-op on imported tools — the page-size override is silently ignored, and a "second page" test then passes or fails depending on whether live data happens to exceed the baked-in default page size.

Control the page size through the cursor instead. `encodeCursor({ s, t })` takes `s` (skip) and `t` (take):

```typescript
import { encodeCursor } from "@umbraco-cms/mcp-server-sdk";
import { validateToolResponse, type CursorPaginatedResult } from "@umbraco-cms/mcp-server-sdk/testing";
import listEntitiesTool from "../get/list-entities.js";

it("should list entities", async () => {
  const result = await listEntitiesTool.handler({}, createMockRequestHandlerExtra());

  // Cast to CursorPaginatedResult for nextCursor access
  const data = validateToolResponse(listEntitiesTool, result) as CursorPaginatedResult;
  expect(data.items.length).toBeGreaterThan(0);
});

it("should paginate with cursor to a second page", async () => {
  const context = createMockRequestHandlerExtra();

  // Two per page from offset 0 — forces a nextCursor whatever the data volume.
  const page1 = await listEntitiesTool.handler(
    { cursor: encodeCursor({ s: 0, t: 2 }) },
    context
  );
  const data1 = validateToolResponse(listEntitiesTool, page1) as CursorPaginatedResult;
  expect(data1.nextCursor).toBeDefined();

  const page2 = await listEntitiesTool.handler({ cursor: data1.nextCursor }, context);
  const data2 = validateToolResponse(listEntitiesTool, page2) as CursorPaginatedResult;
  expect(data2.items[0]).not.toEqual(data1.items[0]);
});
```

This works for read-only collections too (cultures, languages, server info), where `beforeEach` can't seed rows — you set the page size rather than relying on how much data exists.

**`nextCursor` requires a `total` in the response.** The decorator only computes it when `structuredContent` carries a numeric `total` alongside `items` — Umbraco's paged Management API endpoints do. If a tool's response has no `total`, `nextCursor` is always `undefined`, so don't write a second-page test for it: assert on the first page instead. Check the tool's `outputSchema` before assuming pagination is testable.

**Key rules:**
- **NEVER** pass `skip` or `take` to handlers — the decorated schema doesn't accept them
- Pass `{}` for the first page at the tool's default page size
- Force a small page with `encodeCursor({ s: 0, t: N })` — **not** `{ ...tool, pageSize: N }`, which does nothing to an imported tool
- Only call `withCursorPagination(tool)` yourself on a **raw** tool definition that hasn't been through `withStandardDecorators`. Tool files export the decorated form, so tests almost never need it
- For a `toBe(N)` assertion on page size, seed at least N+1 rows in `beforeEach`
- No `total` in the response means no `nextCursor` — skip the second-page test rather than asserting a value that can never appear
- Cast `validateToolResponse` results to `CursorPaginatedResult` from `@umbraco-cms/mcp-server-sdk/testing`
- For edge cases (skip past end), use `encodeCursor({ s: 10000, t: 10 })` as cursor value

#### File naming — one file per tool

| Tool file | Test file |
|-----------|-----------|
| `get/get-{entity}.ts` | `__tests__/get-{entity}.test.ts` |
| `get/list-{entities}.ts` | `__tests__/list-{entities}.test.ts` |
| `post/create-{entity}.ts` | `__tests__/create-{entity}.test.ts` |
| `put/update-{entity}.ts` | `__tests__/update-{entity}.test.ts` |
| `delete/delete-{entity}.ts` | `__tests__/delete-{entity}.test.ts` |

#### Test scope per tool

- **1 happy path test** — use snapshot testing
- **1 error test** — use assertion testing (`isError`)
- **Maximum 2-3 tests per tool**

#### Sequential process per file

For each test file:
1. Write the test
2. Compile: `npm run compile`
3. Run: `npm test -- __tests__/{collection}/{test-file}.test.ts`
4. Fix any failures before creating the next test file

### Step 8: Validate with `integration-test-validator`

After all test files pass for a collection, run the `integration-test-validator` agent. The agent will check:

- `setupTestEnvironment()` used in every describe block
- `configureApiClient()` called in setup.ts
- `createMockRequestHandlerExtra()` used for all handler calls
- Snapshot testing used for success responses (`createSnapshotResult` + `toMatchSnapshot`)
- One test file per tool (no combined test files)
- Builder and helper files in `__tests__/helpers/` directory
- Builder test file exists and passes
- Constants at file head with `TEST_` prefix
- No mock mode (`USE_MOCK_API` is NOT set — tests hit the real API)
- Test count reasonable (2-3 per tool)
- Password constants meet the policy (10+ chars, a digit, a symbol)
- Tools with a flattened parent/target param have a test that supplies it

Flag any issues but continue to the next collection.

### Step 9: Next Collection

Repeat steps 3-8 for the next collection in `.discover.json`.

### Step 10: Final Verification

After all collections have tests:

```bash
npm run compile    # Full type check
npm test           # All integration tests
```

Then run `/count-mcp-tools` to confirm all collections have tests. All collections should show "yes" in the Tests column. If any show "no", report which collections are missing integration tests.

Report what was generated:
- Number of collections with tests
- Number of test files per collection (should be: 1 builder test + 1 per tool)
- Any quality issues found
- Any collections skipped (already had tests)

## File Structure

After running, each collection should have:

```
src/umbraco-api/tools/{collection}/
└── __tests__/
    ├── setup.ts                        # Shared setup, re-exports
    ├── helpers/
    │   ├── {entity}-builder.ts         # Fluent builder
    │   ├── {entity}-builder.test.ts    # Tests the builder itself
    │   └── {entity}-test-helper.ts     # Find, cleanup, normalizeIds
    ├── get-{entity}.test.ts            # One file per tool
    ├── list-{entities}.test.ts
    ├── create-{entity}.test.ts
    ├── update-{entity}.test.ts
    └── delete-{entity}.test.ts
```