Wax Performance Audit
Overview
Use this skill to benchmark Wax changes, isolate the hottest code path, and separate real regressions from noisy samples.
The repo builds with Swift 6.1 and StrictConcurrency enabled, so audit for Swift 6.2 concurrency risks without assuming 6.2-only language mode.
Workflow
- Name the symptom precisely: latency regression, memory growth, file bloat, or Swift concurrency diagnostics.
- Pick the narrowest benchmark or test file that exercises the path.
- Run a baseline and candidate with the same environment and scale.
- Collect wall time plus memory or file-growth metrics when the issue is not purely CPU-bound.
- Inspect the smallest relevant actor boundary, task group, cache, or I/O path.
- Report the evidence, the bottleneck, and the smallest safe fix.
Benchmark Selection
Use references/benchmark-workflow.md for the command matrix, environment flags, and benchmark map.
Prefer these repo entry points when they match the symptom:
Tests/WaxIntegrationTests/RAGBenchmarkSupport.swiftTests/WaxIntegrationTests/RememberDedupBenchmarks.swiftTests/WaxIntegrationTests/StoreBloatBenchmarks.swiftTests/WaxIntegrationTests/RAGBenchmarks.swiftTests/WaxIntegrationTests/RAGBenchmarksMiniLM.swiftTests/WaxIntegrationTests/BatchEmbeddingBenchmark.swiftTests/WaxIntegrationTests/SessionRuntimeStatsBenchmarks.swiftTests/WaxIntegrationTests/WALCompactionBenchmarks.swiftTests/WaxIntegrationTests/HandoffLookupBenchmarks.swiftTests/WaxIntegrationTests/PayloadLivenessBenchmarks.swiftTests/WaxIntegrationTests/SurrogateSourceBenchmarks.swiftTests/WaxIntegrationTests/AccessStatsBootstrapBenchmarks.swiftTests/WaxIntegrationTests/ConcurrencyStressTests.swiftTests/WaxIntegrationTests/MemoryOrchestratorTests.swiftTests/WaxArcticTests/ArcticPerformanceBenchmark.swiftTests/WaxCoreTests/ReadWriteLockTests.swiftTests/WaxCoreTests/AsyncMutexTests.swift
Bottleneck Triage
- CPU: look for repeated serialization, unnecessary sorting, extra actor hops, and oversized batch work.
- Memory: compare RSS, allocated bytes, dead payload bytes, TOC growth, and frame count.
- I/O: inspect WAL compaction, reopen cost, and close-time rewrite work.
- Embeddings: check compute unit selection, batch sizing, and warmup or prewarm behavior.
- Noise: rerun if caches are cold, an external compiler/service is active, or the benchmark has low sample counts.
- Gated skips: confirm the env flag actually enabled the lane before treating a skip or pass as evidence.
- Harness plumbing:
measureAsync in RAGBenchmarkSupport.swift uses DispatchSemaphore plus Task because XCTest measurement is synchronous; do not confuse that with production concurrency.
- ANE/GPU: CPU-only benchmark paths are intentional in some suites, and warm p95/p99 values can be noisy when
ANECompilerService or similar background work is active.
Swift 6.2 Concurrency
Use references/concurrency-checklist.md when the change touches actors, task groups, Sendable, or @unchecked Sendable.
Default checks:
- Trace every value that crosses an actor boundary.
- Prefer
@Sendable closures that capture immutable values.
- Treat
@unchecked Sendable as a deliberate exception, not a default.
- Watch task groups for hidden fan-out that increases memory pressure.
- Keep blocking I/O off actor executors.
- Verify
@MainActor crossings in UI-adjacent or Photos code.
- Treat
@preconcurrency interop and @unchecked Sendable around CoreML, GRDB, Photos, and tokenizer internals as review hotspots, not automatic bugs.
Reporting
When you finish, state:
- the benchmark or test you ran,
- the before/after evidence,
- whether the regression was CPU, memory, I/O, or concurrency-related,
- and the exact file or subsystem that caused it.
1---2name: wax-performance-audit3description: Benchmarking and performance auditing for the Wax repo. Use when running or interpreting Wax benchmarks, diagnosing CPU, memory, or I/O bottlenecks, or investigating Swift 6.2 concurrency issues such as Sendable, actor isolation, `@unchecked Sendable`, task-group fan-out, and data races.4---56# Wax Performance Audit78## Overview910Use this skill to benchmark Wax changes, isolate the hottest code path, and separate real regressions from noisy samples.1112The repo builds with Swift 6.1 and `StrictConcurrency` enabled, so audit for Swift 6.2 concurrency risks without assuming 6.2-only language mode.1314## Workflow15161. Name the symptom precisely: latency regression, memory growth, file bloat, or Swift concurrency diagnostics.172. Pick the narrowest benchmark or test file that exercises the path.183. Run a baseline and candidate with the same environment and scale.194. Collect wall time plus memory or file-growth metrics when the issue is not purely CPU-bound.205. Inspect the smallest relevant actor boundary, task group, cache, or I/O path.216. Report the evidence, the bottleneck, and the smallest safe fix.2223## Benchmark Selection2425Use [`references/benchmark-workflow.md`](references/benchmark-workflow.md) for the command matrix, environment flags, and benchmark map.2627Prefer these repo entry points when they match the symptom:2829- `Tests/WaxIntegrationTests/RAGBenchmarkSupport.swift`30- `Tests/WaxIntegrationTests/RememberDedupBenchmarks.swift`31- `Tests/WaxIntegrationTests/StoreBloatBenchmarks.swift`32- `Tests/WaxIntegrationTests/RAGBenchmarks.swift`33- `Tests/WaxIntegrationTests/RAGBenchmarksMiniLM.swift`34- `Tests/WaxIntegrationTests/BatchEmbeddingBenchmark.swift`35- `Tests/WaxIntegrationTests/SessionRuntimeStatsBenchmarks.swift`36- `Tests/WaxIntegrationTests/WALCompactionBenchmarks.swift`37- `Tests/WaxIntegrationTests/HandoffLookupBenchmarks.swift`38- `Tests/WaxIntegrationTests/PayloadLivenessBenchmarks.swift`39- `Tests/WaxIntegrationTests/SurrogateSourceBenchmarks.swift`40- `Tests/WaxIntegrationTests/AccessStatsBootstrapBenchmarks.swift`41- `Tests/WaxIntegrationTests/ConcurrencyStressTests.swift`42- `Tests/WaxIntegrationTests/MemoryOrchestratorTests.swift`43- `Tests/WaxArcticTests/ArcticPerformanceBenchmark.swift`44- `Tests/WaxCoreTests/ReadWriteLockTests.swift`45- `Tests/WaxCoreTests/AsyncMutexTests.swift`4647## Bottleneck Triage4849- CPU: look for repeated serialization, unnecessary sorting, extra actor hops, and oversized batch work.50- Memory: compare RSS, allocated bytes, dead payload bytes, TOC growth, and frame count.51- I/O: inspect WAL compaction, reopen cost, and close-time rewrite work.52- Embeddings: check compute unit selection, batch sizing, and warmup or prewarm behavior.53- Noise: rerun if caches are cold, an external compiler/service is active, or the benchmark has low sample counts.54- Gated skips: confirm the env flag actually enabled the lane before treating a skip or pass as evidence.55- Harness plumbing: `measureAsync` in `RAGBenchmarkSupport.swift` uses `DispatchSemaphore` plus `Task` because XCTest measurement is synchronous; do not confuse that with production concurrency.56- ANE/GPU: CPU-only benchmark paths are intentional in some suites, and warm p95/p99 values can be noisy when `ANECompilerService` or similar background work is active.5758## Swift 6.2 Concurrency5960Use [`references/concurrency-checklist.md`](references/concurrency-checklist.md) when the change touches actors, task groups, `Sendable`, or `@unchecked Sendable`.6162Default checks:6364- Trace every value that crosses an actor boundary.65- Prefer `@Sendable` closures that capture immutable values.66- Treat `@unchecked Sendable` as a deliberate exception, not a default.67- Watch task groups for hidden fan-out that increases memory pressure.68- Keep blocking I/O off actor executors.69- Verify `@MainActor` crossings in UI-adjacent or Photos code.70- Treat `@preconcurrency` interop and `@unchecked Sendable` around CoreML, GRDB, Photos, and tokenizer internals as review hotspots, not automatic bugs.7172## Reporting7374When you finish, state:7576- the benchmark or test you ran,77- the before/after evidence,78- whether the regression was CPU, memory, I/O, or concurrency-related,79- and the exact file or subsystem that caused it.
christopherkarani