Build Tools Tests

Build Tests

Generate integration tests for MCP tool collections created by /build-tools. This skill reads .discover.json and the existing tool files, then builds test infrastructure and test files one collection at a time.

IMPORTANT: This skill ONLY creates files inside __tests__/ directories — test files, setup, builders, and helpers. Do NOT create or modify ANY other files. This means: no tool files, no collection indexes, no registrations, no API client files (src/umbraco-api/api/), no generated code (src/umbraco-api/api/generated/), no mock handlers (src/mocks/). If existing code doesn't support what you need, work within the constraints — do not modify it.

Prerequisites

Before running, ensure:

1. You have run /build-tools (tool collections exist in src/umbraco-api/tools/)
2. The project compiles: npm run compile
3. The Umbraco instance is running
4. An API user exists — remind the user: "You need to create an API user via the Umbraco backoffice UI: Settings > Users, with Client ID umbraco-back-office-mcp and Client Secret 1234567890"

Arguments

• No arguments: build tests for all collections from .discover.json
• Single collection name: build tests only for that collection (e.g. /build-tools-tests form)

Agents

This skill orchestrates the following agents — use them for the relevant steps:

Critical Rules

ONE FILE AT A TIME. This applies to ALL files — builders, helpers, builder tests, and tool tests. After creating any file:

1. Compile: npm run compile
2. If it has tests, run them: npm test -- path/to/file.test.ts
3. Fix any failures
4. Only then create the next file

RUN COMMANDS SEPARATELY. Always run compile and test as separate Bash calls. Never chain them with &&.

NEVER:

• Create multiple files at once
• Move on while a compile error or test failure exists
• Skip the compile step
• Assume anything works without running it
• Chain commands with && (run each command separately)

ONE TEST FILE PER TOOL. Every tool gets its own test file. Never combine tests for multiple tools into one file.

SNAPSHOT TESTING PREFERRED. Use createSnapshotResult from @umbraco-cms/mcp-server-sdk/testing with toMatchSnapshot() for success responses. Only use assertion testing (expect(x).toBe(y)) for error cases where isError is checked.

REAL API — NO MOCKING. These are integration tests that run against a real Umbraco instance. Do NOT set USE_MOCK_API. Do NOT create, modify, or reference anything in src/mocks/. Do NOT import server from mocks. Do NOT add MSW handlers. Do NOT use any mocking framework. The tests call tool handlers directly and those handlers call the real API. If a test fails, the fix is in the test or the tool — never add mock infrastructure.

Test Data Gotchas

These fail in ways that look like tool bugs but are really test-data bugs. Check here first when a builder can't create its entity.

Password fields. Umbraco's default password policy rejects weak values, and the API returns a validation ProblemDetails that reads like a broken tool. Any builder creating users, members, or anything else with a password must use a value that is:

• at least 10 characters
• contains at least one digit
• contains at least one non-alphanumeric symbol

const TEST_PASSWORD = "_TestPassword1!";

Put it in a TEST_ constant at the head of the builder — never inline a short password, and never weaken the test to work around the rejection.

Email and login uniqueness. Users and members are unique by email/username. Derive them from the TEST_ name constant (_test-user-1@example.invalid) so cleanup by prefix reliably removes them, otherwise a failed run leaves data that breaks the next one.

Flattened parent/target params. Tools that flatten a nested reference (parentId → parent: { id }, path → parent: { path }) need at least one test that actually supplies it. The API accepts parent: null happily, so root-only coverage passes even when the transform is broken. See the schema flattening section of /mcp-patterns.

Trees and folders need real ancestry. ancestors/children/root and folder tools can't be tested against a single flat entity — the builder must be able to create a child under a known parent. If the builder has no withParentId(), add it before writing those tests.

Umbraco Instance Management

If testing hits roadblocks — builders can't create data, APIs reject requests due to missing configuration, or features aren't available — you are able to manipulate the Umbraco instance to your needs. You can add connection strings, change settings, install packages, or even write C# code in demo-site/. Read instance-management.md in this skill directory for the full process and concrete examples.

Workflow

Process one collection at a time. Complete each collection fully before starting the next.

Step 0: Read Discovery Manifest

Read .discover.json from the project root:

{
  "apiName": "Umbraco Forms Management API",
  "swaggerUrl": "https://localhost:44324/umbraco/swagger/forms-management/swagger.json",
  "baseUrl": "https://localhost:44324",
  "collections": ["form", "form-template", "field-type", "folder"]
}

If an argument was provided, filter to only that collection. If .discover.json doesn't exist, tell the user to run npx @umbraco-cms/create-umbraco-mcp-server discover first.

Step 1: Understand Existing Tools

For each collection, read:

• src/umbraco-api/tools/{collection}/index.ts — to get the list of tools
• Each tool file — to understand input schemas, handler logic, and entity names
• src/umbraco-api/api/generated/ — to identify the API client function and Zod schemas

If src/umbraco-api/tools/{collection}/index.ts doesn't exist, skip — tell the user to run /build-tools first.

Step 2: Check for Existing Tests

Skip if src/umbraco-api/tools/{collection}/__tests__/setup.ts already exists — tests have already been created for this collection.

Step 3: Per Collection — Create Test Setup

Create src/umbraco-api/tools/{collection}/__tests__/setup.ts:

import {
  setupTestEnvironment,
  createMockRequestHandlerExtra,
  createSnapshotResult,
} from "@umbraco-cms/mcp-server-sdk/testing";
import { configureApiClient, initializeUmbracoFetch } from "@umbraco-cms/mcp-server-sdk";
import { getYourAPI } from "../../../../api/generated/yourApi.js";
import { EntityBuilder } from "./helpers/{entity}-builder.js";
import { EntityTestHelper } from "./helpers/{entity}-test-helper.js";

// Initialize fetch with credentials — required for integration tests hitting the real API
initializeUmbracoFetch({
  baseUrl: process.env.UMBRACO_BASE_URL!,
  clientId: process.env.UMBRACO_CLIENT_ID!,
  clientSecret: process.env.UMBRACO_CLIENT_SECRET!,
});

configureApiClient(() => getYourAPI());

export {
  setupTestEnvironment,
  createMockRequestHandlerExtra,
  createSnapshotResult,
  EntityBuilder,
  EntityTestHelper,
};

Key rules:

• Import the correct API client getter from src/umbraco-api/api/generated/
• initializeUmbracoFetch MUST be called before configureApiClient — it sets up the authenticated fetch layer
• Do NOT set USE_MOCK_API — these tests run against the real Umbraco instance
• Export createSnapshotResult for snapshot testing
• Re-export builders and helpers so test files have a single import

Compile after creating: npm run compile. Fix errors before continuing.

Read-only collections: If the collection has no create or delete operations (e.g. analytics — only GET/query tools), skip steps 4-6 (builder, helper, builder tests). These steps create test data lifecycle management which isn't needed for read-only collections. Proceed directly to step 7 (integration tests).

Step 4: Per Collection — Create Test Builder

Use the test-builder-helper-creator agent.

Create src/umbraco-api/tools/{collection}/__tests__/helpers/{entity}-builder.ts:

import { getYourAPI } from "../../../../api/generated/yourApi.js";
import { CAPTURE_RAW_HTTP_RESPONSE } from "@umbraco-cms/mcp-server-sdk";

const TEST_ENTITY_NAME = "_Test Entity";

interface EntityModel {
  name: string;
  // ... fields matching the POST body schema from the generated *.zod.ts
}

export class EntityBuilder {
  private model: EntityModel = {
    name: TEST_ENTITY_NAME,
  };

  private createdId?: string;

  withName(name: string): this {
    this.model.name = name;
    return this;
  }

  build(): EntityModel {
    return { ...this.model };
  }

  async create(): Promise<this> {
    const client = getYourAPI();
    // Call the API client's POST method directly (NOT the tool handler)
    const response: any = await client.postEntity(
      this.model as any,
      CAPTURE_RAW_HTTP_RESPONSE,
    );

    if (response.status !== 201) {
      const errorBody = await response.data?.detail || `HTTP ${response.status}`;
      throw new Error(`Failed to create entity: ${errorBody}`);
    }

    // Extract ID from Location header (Umbraco convention: /api/v1/entity/{id})
    const location = response.headers?.get?.("location") || response.headers?.location;
    this.createdId = location?.split("/").pop();

    return this;
  }

  async delete(): Promise<void> {
    if (!this.createdId) return;
    const client = getYourAPI();
    try {
      await client.deleteEntityById(this.createdId, CAPTURE_RAW_HTTP_RESPONSE);
    } catch {
      // Ignore delete failures in cleanup
    }
    this.createdId = undefined;
  }

  getId(): string {
    if (!this.createdId) {
      throw new Error("Entity not created yet. Call create() first.");
    }
    return this.createdId;
  }
}

Key rules:

• Fluent interface — all withX methods return this
• build() returns the model, create() calls the API
• Store created ID for use in tests
• Use TEST_ prefix for constants
• Match the create tool's input schema for the model fields

Compile after creating: npm run compile. Fix errors before continuing.

Step 5: Per Collection — Create Test Helper

Use the test-builder-helper-creator agent.

Create src/umbraco-api/tools/{collection}/__tests__/helpers/{entity}-test-helper.ts:

export class EntityTestHelper {
  static async findByName(name: string): Promise<any | undefined> {
    // Use list tool or API client to find entity
  }

  static async cleanup(namePrefix: string): Promise<void> {
    // List entities and delete those matching prefix
  }

  static normalizeIds(data: any): any {
    // Replace UUIDs with zeroed placeholder for snapshots
    if (Array.isArray(data)) {
      return data.map(item => this.normalizeIds(item));
    }
    if (data && typeof data === "object") {
      const normalized = { ...data };
      if (normalized.id) {
        normalized.id = "00000000-0000-0000-0000-000000000000";
      }
      for (const key of Object.keys(normalized)) {
        if (typeof normalized[key] === "object") {
          normalized[key] = this.normalizeIds(normalized[key]);
        }
      }
      return normalized;
    }
    return data;
  }
}

Compile after creating: npm run compile. Fix errors before continuing.

Step 6: Per Collection — Create Builder Tests

Create src/umbraco-api/tools/{collection}/__tests__/helpers/{entity}-builder.test.ts:

import {
  setupTestEnvironment,
  createMockRequestHandlerExtra,
  EntityBuilder,
  EntityTestHelper,
} from "../setup.js";

const TEST_NAME = "_Test Builder Entity";

describe("EntityBuilder", () => {
  setupTestEnvironment();

  let builder: EntityBuilder;

  afterEach(async () => {
    // Always clean up created entities to prevent conflicts with other test files
    if (builder) await builder.delete();
    await EntityTestHelper.cleanup(TEST_NAME);
  });

  it("should create entity with builder", async () => {
    const builder = await new EntityBuilder()
      .withName(TEST_NAME)
      .create();

    expect(builder.getId()).toBeDefined();

    const found = await EntityTestHelper.findByName(TEST_NAME);
    expect(found).toBeDefined();
    expect(found?.name).toBe(TEST_NAME);
  });
});

After creating:

1. Compile: npm run compile
2. Run: npm test -- __tests__/{collection}/{entity}-builder.test.ts
3. Fix any failures before continuing

Step 7: Per Collection — Create Integration Tests

Use the integration-test-creator agent.

Create one test file per tool. Each tool gets its own .test.ts file. Create and run each sequentially.

Build order within a collection

Each stage depends on the previous one working, so build them in this order and don't move on until the current stage passes:

Skipping ahead (e.g. writing tree tests before create passes) means debugging two unknowns at once. If a stage is genuinely blocked, note it and continue — don't fabricate data to get past it.

Snapshot test pattern (preferred for success cases)

import {
  setupTestEnvironment,
  createMockRequestHandlerExtra,
  createSnapshotResult,
  EntityBuilder,
} from "./setup.js";
import getEntityTool from "../get/get-entity.js";

describe("get-entity", () => {
  setupTestEnvironment();

  let builder: EntityBuilder;

  afterEach(async () => {
    // Clean up test data after each test to prevent conflicts
    if (builder) await builder.delete();
  });

  it("should return entity by ID", async () => {
    const context = createMockRequestHandlerExtra();
    builder = await new EntityBuilder()
      .withName("_Test Get Entity")
      .create();

    const result = await getEntityTool.handler(
      { id: builder.getId() },
      context
    );

    expect(
      createSnapshotResult(result, builder.getId())
    ).toMatchSnapshot();
  });

  it("should return error for non-existent ID", async () => {
    const context = createMockRequestHandlerExtra();

    const result = await getEntityTool.handler(
      { id: "00000000-0000-0000-0000-000000000000" },
      context
    );

    expect(result.isError).toBe(true);
  });
});

Key rules:

• ALWAYS use createSnapshotResult(result, id) for success responses — it normalizes IDs, dates, and dynamic values
• Pass the created entity's ID as second argument to createSnapshotResult so it gets normalized
• Use toMatchSnapshot() — not toMatchInlineSnapshot()
• Only use assertion testing (expect(x).toBe(y)) for error cases
• Use builders to create test data when the test needs existing entities

NEVER access result properties directly. The following patterns are WRONG and will fail:

// WRONG — result.content may be undefined
const data = JSON.parse(result.content[0].text);

// WRONG — result structure varies by output mode
expect(result.content).toContain("something");

Always use the snapshot helper:

// CORRECT — handles all output modes
expect(createSnapshotResult(result, builder.getId())).toMatchSnapshot();

Cursor pagination in tests

Paginated tools (those with skip/take in their input schema) are converted to cursor pagination by withCursorPagination, which withStandardDecorators already applies — and every tool file exports withStandardDecorators(tool). So an imported tool's handler already accepts cursor and no longer accepts skip/take.

Do not wrap an imported tool again. withCursorPagination only transforms a tool whose inputSchema still has both skip and take; given an already-decorated tool it returns it unchanged. That makes withCursorPagination({ ...tool, pageSize: N }) a no-op on imported tools — the page-size override is silently ignored, and a "second page" test then passes or fails depending on whether live data happens to exceed the baked-in default page size.

Control the page size through the cursor instead. encodeCursor({ s, t }) takes s (skip) and t (take):

import { encodeCursor } from "@umbraco-cms/mcp-server-sdk";
import { validateToolResponse, type CursorPaginatedResult } from "@umbraco-cms/mcp-server-sdk/testing";
import listEntitiesTool from "../get/list-entities.js";

it("should list entities", async () => {
  const result = await listEntitiesTool.handler({}, createMockRequestHandlerExtra());

  // Cast to CursorPaginatedResult for nextCursor access
  const data = validateToolResponse(listEntitiesTool, result) as CursorPaginatedResult;
  expect(data.items.length).toBeGreaterThan(0);
});

it("should paginate with cursor to a second page", async () => {
  const context = createMockRequestHandlerExtra();

  // Two per page from offset 0 — forces a nextCursor whatever the data volume.
  const page1 = await listEntitiesTool.handler(
    { cursor: encodeCursor({ s: 0, t: 2 }) },
    context
  );
  const data1 = validateToolResponse(listEntitiesTool, page1) as CursorPaginatedResult;
  expect(data1.nextCursor).toBeDefined();

  const page2 = await listEntitiesTool.handler({ cursor: data1.nextCursor }, context);
  const data2 = validateToolResponse(listEntitiesTool, page2) as CursorPaginatedResult;
  expect(data2.items[0]).not.toEqual(data1.items[0]);
});

This works for read-only collections too (cultures, languages, server info), where beforeEach can't seed rows — you set the page size rather than relying on how much data exists.

nextCursor requires a total in the response. The decorator only computes it when structuredContent carries a numeric total alongside items — Umbraco's paged Management API endpoints do. If a tool's response has no total, nextCursor is always undefined, so don't write a second-page test for it: assert on the first page instead. Check the tool's outputSchema before assuming pagination is testable.

Key rules:

• NEVER pass skip or take to handlers — the decorated schema doesn't accept them
• Pass {} for the first page at the tool's default page size
• Force a small page with encodeCursor({ s: 0, t: N }) — not { ...tool, pageSize: N }, which does nothing to an imported tool
• Only call withCursorPagination(tool) yourself on a raw tool definition that hasn't been through withStandardDecorators. Tool files export the decorated form, so tests almost never need it
• For a toBe(N) assertion on page size, seed at least N+1 rows in beforeEach
• No total in the response means no nextCursor — skip the second-page test rather than asserting a value that can never appear
• Cast validateToolResponse results to CursorPaginatedResult from @umbraco-cms/mcp-server-sdk/testing
• For edge cases (skip past end), use encodeCursor({ s: 10000, t: 10 }) as cursor value

File naming — one file per tool

Test scope per tool

• 1 happy path test — use snapshot testing
• 1 error test — use assertion testing (isError)
• Maximum 2-3 tests per tool

Sequential process per file

For each test file:

1. Write the test
2. Compile: npm run compile
3. Run: npm test -- __tests__/{collection}/{test-file}.test.ts
4. Fix any failures before creating the next test file

Step 8: Validate with integration-test-validator

After all test files pass for a collection, run the integration-test-validator agent. The agent will check:

• setupTestEnvironment() used in every describe block
• configureApiClient() called in setup.ts
• createMockRequestHandlerExtra() used for all handler calls
• Snapshot testing used for success responses (createSnapshotResult + toMatchSnapshot)
• One test file per tool (no combined test files)
• Builder and helper files in __tests__/helpers/ directory
• Builder test file exists and passes
• Constants at file head with TEST_ prefix
• No mock mode (USE_MOCK_API is NOT set — tests hit the real API)
• Test count reasonable (2-3 per tool)
• Password constants meet the policy (10+ chars, a digit, a symbol)
• Tools with a flattened parent/target param have a test that supplies it

Flag any issues but continue to the next collection.

Step 9: Next Collection

Repeat steps 3-8 for the next collection in .discover.json.

Step 10: Final Verification

After all collections have tests:

npm run compile    # Full type check
npm test           # All integration tests

Then run /count-mcp-tools to confirm all collections have tests. All collections should show "yes" in the Tests column. If any show "no", report which collections are missing integration tests.

Report what was generated:

• Number of collections with tests
• Number of test files per collection (should be: 1 builder test + 1 per tool)
• Any quality issues found
• Any collections skipped (already had tests)

File Structure

After running, each collection should have:

src/umbraco-api/tools/{collection}/
└── __tests__/
    ├── setup.ts                        # Shared setup, re-exports
    ├── helpers/
    │   ├── {entity}-builder.ts         # Fluent builder
    │   ├── {entity}-builder.test.ts    # Tests the builder itself
    │   └── {entity}-test-helper.ts     # Find, cleanup, normalizeIds
    ├── get-{entity}.test.ts            # One file per tool
    ├── list-{entities}.test.ts
    ├── create-{entity}.test.ts
    ├── update-{entity}.test.ts
    └── delete-{entity}.test.ts