# Apple Integration Testing

> Use when writing integration tests for Apple platforms (Swift / XCTest) covering service layers, repositories, parsers, data pipelines, or any code whose correctness is verifiable through observable state (file contents, returned values, DB rows, exit codes) without launching the UI. Use when consolidating small unit tests into scenario-based integration tests, when a test passes but the feature is broken (vacuous assertion), or when an existing test silently skips a criterion.

- Skill: `sunfmin/apple-integration-testing` (Agent Skill)
- Install (CLI): `npx skillmds@latest add sunfmin/apple-integration-testing`
- Raw SKILL.md: https://api.skillmd.com/api/skills/sunfmin/apple-integration-testing/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Integrations & APIs
- Author: sunfmin (https://skillmd.com/u/sunfmin)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/sunfmin/apple-integration-testing

---


# Apple Integration Testing

Assertion-based integration tests for Apple platforms. Real dependencies, real files, real frameworks. One scenario test per pipeline, step-by-step assertions, no mocks of the thing under test.

This skill covers **observable-state verification only** — file contents, returned values, DB rows, exit codes. UI verification (layout, toasts, modals) is out of scope and belongs to a UI-driving skill.

## When to Use

- Service layer, repositories, parsers, value logic, data pipelines on iOS / macOS
- Multi-stage pipelines (A → B → C → D) where a silent failure between stages is the bug class you fear
- External-dependency code: C/FFI, vendored libs, model loading, ScreenCaptureKit, AVFoundation, CoreData, OpenAPI repositories
- File I/O chains: input file → processing → output file (existence ≠ correctness)
- Consolidating many small unit tests into one integrated scenario per pipeline

## When NOT to Use

- UI-visible criteria (layout, toasts, modals, focus, dialogs) — those need a UI driver, not XCTest
- Pure isolated value-object logic where one tiny unit test is genuinely sufficient
- Code that can't be instantiated without launching the full app — refactor first (extract logic, inject dependencies), then test

## Core Principle

**Test the pipeline, not the methods.** One integrated test that exercises A → B → C → D catches real interaction bugs that isolated tests miss. When it fails, step-by-step assertions tell you exactly WHERE.

```swift
func test_fullScenario_featureName() throws {
    // STEP 1: Setup — create real test data
    let input = try makeRealInput()
    XCTAssertGreaterThan(input.count, 0, "Step 1: setup produced empty input")

    // STEP 2: Component A processes input
    let a = ServiceA().process(input)
    XCTAssertFalse(a.isEmpty, "Step 2: ServiceA produced empty output for non-empty input")

    // STEP 3: Component B receives A's output
    let b = ServiceB().consume(a)
    XCTAssertTrue(b.contains(expectedToken), "Step 3: ServiceB output missing expected token \(expectedToken)")

    // STEP 4: End-to-end verification — parse & validate content
    let final = try Parser.parse(b)
    XCTAssertEqual(final.kind, .success, "Step 4: final parse did not yield success")
}
```

**Rules:** One test per pipeline, not per method. Each step asserts with a unique failure message. Fail fast — don't run Step 3 if Step 2 failed. Real dependencies, no mocks. Clean up in `tearDown`. Remove redundant small tests already covered by the scenario. Keep small tests ONLY for error paths, boundary conditions, and pure logic not exercised by scenarios.

## Integration Test Contract

Before writing tests, write a contract — even a short one in your PR description. It forces you to name what each test must prove.

```markdown
# Integration Test Contract: <feature>

## Pipeline
A (input parser) → B (transform) → C (encoder) → D (output writer)

## Silent failure risks
- B may drop fields when input contains nested arrays
- C may produce a non-empty file with corrupt header bytes

## SCENARIO 1: full pipeline, happy path
- SETUP: temp dir, fixture file `input.json`
- ACTION: run pipeline end-to-end
- ASSERT_TYPE: behavioral
- ASSERT_CONTAINS: output file exists, size > 0, header == "MAGIC", parsed AST has N nodes
- FAIL_IF_BLOCKED: "FAIL: cannot verify SCENARIO 1 — fixture input.json missing"

## EDGE 1: input missing required field
- SCOPE: error path only
- ASSERT: throws ParseError.missingField("name")
```

**Assertion types:**
- `behavioral` — the **mandatory before/after pattern** (capture state before, perform action, capture after, assert state CHANGED, assert new state CONTAINS expected content). A change check alone passes for errors too. A content check alone doesn't prove the action caused the change. Both are required.
- `state` — verify a property matches an expected value (disabled, checked, count == N).
- `existence` — element is present. Only acceptable for "is the thing visible" criteria. **Forbidden as a substitute for behavioral.**

## Behavioral Assertion Pattern (Swift / XCTest)

```swift
let before = service.currentState()
try service.performAction(input)
let after = service.currentState()

XCTAssertNotEqual(before, after, "AC2: state must change after performAction")
XCTAssertTrue(after.contains(expectedToken),
              "AC2: new state must contain \(expectedToken), got \(after)")
```

Both lines required for `behavioral`. One alone is vacuous.

## Forbidden Guard Patterns (Swift)

Every contract criterion must either assert (success) or fail (blocked) — **never silently skipped**.

**FORBIDDEN — silently skips when condition is false:**

```swift
guard let result = action() else { return }              // assertion never runs
if let dir = findDirectory() { XCTAssertTrue(...) }      // skipped on nil
```

**ALLOWED — fails loudly when prerequisite missing:**

```swift
guard let result = action() else {
    XCTFail("action() returned nil — prerequisite for AC3 not met")
    return
}
let dir = findDirectory()
XCTAssertFalse(dir.isEmpty, "findDirectory() must return a non-empty path")
```

If a prerequisite cannot be established, FAIL explicitly with a `FAIL_IF_BLOCKED` message verbatim from the contract. Never paraphrase.

## Single-Flow State Machine

When the contract has Phases that depend on each other, implement them in **a single test function** that flows Phase 1 → Phase 2 → … → Phase N.

**Why:** Splitting Phases into separate functions means each function starts from scratch and loses state Phase 3 needs from Phase 1.

**Exceptions** — separate test functions are allowed only for:
- A criterion requiring **process relaunch** (persistence across restart)
- A criterion requiring **contradictory preconditions** (e.g., feature OFF after feature ON)

Each exception function must re-establish its own preconditions from scratch.

## Bypass Flag Ban

Flags that bypass real processing are **banned in tests**:

```
-generateTestTranscript
-useTestDownloads
-useFakeData
```

Only **state config** flags are allowed (e.g., `-hasCompletedSetup YES` to skip an onboarding wizard). State flags configure the app; they do not replace the code under test.

Equivalents to scan for: `Simulated*`, `Fake*`, `Mock*` in production targets — see "Never Simulate" below.

## Real Test Content

Generate real inputs with system tools, not hardcoded canned outputs.

| Feature | How |
|---------|-----|
| Audio processing | `say "test content" -o /tmp/test.aiff` then convert if needed |
| Transcription | `say` known text → feed to service → assert output `contains` known text |
| Window enumeration | real `ScreenCaptureKit` query of the running test process |
| Persistence | real CoreData store on disk in a temp dir; tear down after |
| Network repositories | real local server fixture or recorded responses replayed by URLProtocol — never a mock of the repository |

Small but real: 2-second audio clips, minimal valid files, tiny models. Each test under 30s where possible (full-pipeline tests may be longer). Use temp directories and clean up in `tearDown`.

## Never Simulate

Never put `Simulated{Feature}Repository`, `Fake{Feature}`, or `Mock{Feature}` in **production code**. Every service must use real framework APIs:

- Window enumeration → ScreenCaptureKit (not a hardcoded window list)
- Model inference → real ML framework (not `Thread.sleep()` + canned output)
- Media playback → AVPlayer (not a static image)

If a real API requires permissions or hardware that blocks progress, treat it as a blocker to escalate, not to stub. The only acceptable test doubles live in test targets — never in the running app.

## Adding New Test Files (XcodeGen)

When adding a new `.swift` file to a test target, run `xcodegen generate` to regenerate `.xcodeproj`. The `sources:` directive in `project.yml` auto-discovers `.swift` files in the directory, but only after regeneration.

Never edit `.pbxproj` manually. Never use the Xcode GUI to change Build Settings on an XcodeGen-managed project — `xcodegen generate` overwrites those edits.

## Running Tests

Run **only the tests related to the current change** — specific test class or file. Do NOT run the full test suite while iterating.

```bash
xcodebuild test \
  -project MyApp.xcodeproj \
  -scheme MyApp \
  -destination 'platform=macOS' \
  -only-testing:MyAppTests/PipelineIntegrationTests
```

Split build and test commands when the platform supports it so build errors surface immediately.

If output is too verbose, spawn a sub-agent (or pipe to a file you read separately) — do not pipe `xcodebuild` through `| tail` / `| grep` while the build is running, that hides failures.

## Compliance Scans (run before declaring done)

Run these greps against the test files and the production code they exercise. Any non-CLEAN result needs a fix before merge.

```bash
# 1. Bypass flags in tests
grep -rn "generateTestTranscript\|useTestDownloads\|useFakeData" . --include="*.swift" || echo "CLEAN"

# 2. Stub functions in production (return "" / return [] as the only path)
grep -rn 'return ""$\|return \[\]$' . --include="*.swift" \
  | grep -v "Tests\|guard\|else\|catch\|//" || echo "CLEAN"

# 3. Test data generators in production code
grep -rn "testSentences\|generateTest\|hardcodedSegments" . --include="*.swift" \
  | grep -v "Tests" || echo "CLEAN"

# 4. Vacuous assertions (OR'd booleans always pass)
grep -rn "XCTAssertTrue.*||" . --include="*.swift" | grep "Tests" || echo "CLEAN"

# 5. Silent-skip guards in test files (if-let around assertions, no XCTFail)
TEST_FILE=path/to/IntegrationTests.swift
grep -n 'if let.*= .*{' "$TEST_FILE" \
  | grep -v "// optional\|cleanup\|Cleanup\|delete\|Delete" || echo "CLEAN"

# 6. Empty output artifacts (adjust paths/sizes per feature)
find ~/<AppName> -name "audio.m4a" -size -1k 2>/dev/null
find ~/<AppName> -name "transcript.jsonl" -empty 2>/dev/null
find ~/<AppName> -name "video.mp4" -size -10k 2>/dev/null
```

For every assertion in the test file, ask the **emptied-handler test**: *if I deleted the body of the function under test, would this assertion still pass?* If yes, the assertion is vacuous — strengthen it.

## Quick Reference

| Symptom | Cause | Fix |
|---------|-------|-----|
| Test passes but feature is broken | Vacuous assertion (existence-only or change-only) | Apply behavioral pattern: assert change AND content |
| `guard let` around the assertion, test "passes" with no output | Silent-skip guard | Replace `else { return }` with `else { XCTFail(...); return }` |
| File exists at expected path → test green; user sees garbage | Existence check, no content validation | Parse the file; assert size, header, structure |
| Phase 3 fails with "no state" when run with all tests | Phases split across separate test functions | Combine into one test that runs Phase 1 → 2 → 3 in order |
| Test green; manual run shows broken pipeline | Mock somewhere in the chain | Remove mocks for the components under test; use real framework |
| Test compiles, won't run after adding new file | Forgot `xcodegen generate` | Run it; clean DerivedData if stale |
| Build setting reverted next run | Edited `.xcodeproj` directly under XcodeGen | Move setting into `project.yml`, regenerate |

## Common Mistakes

| Mistake | Fix |
|---------|-----|
| Using `XCTAssertNotEqual(before, after)` alone for behavioral | Add `XCTAssertTrue(after.contains(expected))` — both required |
| `if let result = service.run() { XCTAssertTrue(result.ok) }` | `guard let result = ... else { XCTFail(...); return }` |
| Many small `test_*` per service, none exercising the pipeline | One scenario test exercising A → B → C → D with step-by-step asserts |
| Mocking the repository the test is meant to verify | Use a real on-disk store / real local server in temp dir |
| Hardcoded fixture audio without speech content | `say "known phrase" -o tmp.aiff`, assert transcript contains "known phrase" |
| `Thread.sleep(forTimeInterval: 3)` waiting for async work | Use `XCTestExpectation` or `await` — no fixed sleeps |
| Test file added to repo, ignored by Xcode | Run `xcodegen generate`, re-clean DerivedData |
| Running entire suite on every change | Run only `-only-testing:Module/ClassName` while iterating |

## Adopting in a New Project

1. Decide which pipelines need integration coverage (the data flows where silent failures hurt most).
2. Write the **integration test contract** for one pipeline (pipeline name, silent failure risks, scenarios with ASSERT_CONTAINS).
3. Implement one scenario test that follows the contract literally — step assertions, behavioral pattern, real dependencies.
4. Run the compliance scans above; clean every flag.
5. Delete the redundant small unit tests the scenario now covers.
6. Repeat per pipeline.