# Open Fusion Test Strategy

> Design the right testing approach for Open Fusion. Use when Codex needs to choose, write, restructure, or review unit, integration, contract, e2e, streaming, provider-adapter, config, orchestration, or security tests for the Open Fusion LLM gateway.

- Skill: `vinilana/open-fusion-test-strategy` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add vinilana/open-fusion-test-strategy`
- Raw SKILL.md: https://api.skillmd.com/api/skills/vinilana/open-fusion-test-strategy/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: vinilana (https://skillmd.com/u/vinilana)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/vinilana/open-fusion-test-strategy

---


# Open Fusion Test Strategy

## Required Context

Read the owning spec before designing tests:

- API behavior: `docs/specs/001-openai-compatible-api.md`
- Orchestration: `docs/specs/002-llm-orchestration-routing.md`
- Config: `docs/specs/003-single-json-configuration.md`
- Providers: `docs/specs/004-provider-adapters-openrouter.md`
- Streaming/tools: `docs/specs/005-streaming-tools-response-normalization.md`
- Ops/security: `docs/specs/007-observability-resilience-security.md`

## Test Pyramid

Prefer this order:

1. Unit tests for pure validation, mapping, policy, and normalization logic.
2. Integration tests for NestJS modules, guards, filters, config loading, and service wiring.
3. Contract tests for public OpenAI-compatible envelopes, provider adapter behavior, and streaming chunks.
4. E2E tests for critical client flows through HTTP.
5. Live provider tests only when explicitly configured and cost-controlled.

## What To Mock

Mock or fake by default:

- Vercel AI SDK model calls;
- OpenRouter/network calls;
- time, ids, and environment variables;
- streaming token sources;
- provider failures and timeouts.

Do not mock the code under test. Mock at external boundaries.

## Required Coverage Areas

API:

- valid chat completion request;
- invalid request envelope;
- non-finite numeric fields (`NaN`, `Infinity`, `-Infinity`);
- message count, message content, and payload limit failures;
- unknown public model;
- auth failure;
- OpenAI-style error body;
- streaming chunk format and `[DONE]`.

Config:

- valid minimal config;
- unknown provider type;
- missing env secret;
- invalid model/provider reference;
- invalid route/orchestrator reference;
- secret redaction.

Orchestration:

- direct orchestrator response;
- allowed delegation;
- blocked delegation;
- max delegation limit;
- blocked, failed, and timed out attempts counted against `maxDelegations`;
- delegate timeout;
- finish reason propagation;
- `depends_on: []` does not become a pre-final task by itself;
- parallel pre-final task failure cancels in-flight work or explicitly ignores late results;
- final-only streaming.

Providers:

- non-streaming success;
- streaming success;
- tool-capable path;
- malformed provider tool payloads;
- malformed provider-supplied delegate messages;
- unsupported capability;
- unknown internal model id maps to internal error without exposing the unresolved id in the public error body;
- provider timeout;
- provider error normalization.

Ops:

- request id propagation;
- structured log fields;
- `chat_completion.failed` for validation/model/tools policy errors before SSE;
- redaction;
- generic public `internal_error` messages for internal configuration failures, with diagnostic identifiers kept out of the OpenAI-compatible response body;
- payload limits;
- health checks without paid provider calls.

Test hygiene:

- Restore `process.env` by mutating the existing object, not by assigning a new object to `process.env`.
- Assert behavior and public contracts first; use mock call assertions only to clarify boundary interactions.

## Test Quality

- Assert observable behavior, not private implementation.
- Keep fixtures small and named.
- Avoid snapshots for dynamic OpenAI-compatible envelopes unless normalized.
- Use builders/helpers only after duplication is real.
- Make failures diagnostic: include model/route/status expectations clearly.