Visionos Dev

Building native visionOS apps

Build native visionOS apps — Swift + SwiftUI + RealityKit + ARKit, authored in Xcode with Reality Composer Pro. Native is the right default: it gets the full system look-and-feel, the lowest latency, every OS feature, and the smoothest App Store path. Unity's PolySpatial is the alternative for teams with an existing Unity codebase or a fully-immersive game; treat it as out of scope here.

This skill ships a buildable starter project and focused reference files. Read the references on demand — do not dump them all into context. The map is the decision tree below.

Start here: the mental model

visionOS apps are SwiftUI apps whose App.body is composed of scenes. There are three scene types, and choosing among them is the single most important design decision:

Shared Space vs. Full Space is the other axis. By default apps run in the Shared Space (windows + volumes coexist with other apps). Opening an ImmersiveSpace moves the app into the Full Space, where content can be placed freely and other apps optionally hidden. Immersion has three styles:

• .mixed — app content blended with passthrough of the real room (AR).
• .progressive — a partial portal; the user dials immersion with the Digital Crown.
• .full — passthrough fully replaced by the app's environment (VR).

RealityKit's 3D content lives in entities (an Entity–Component–System world) and is presented to SwiftUI through RealityView. 3D assets are authored in Reality Composer Pro and loaded from a Swift package by name. ARKit (ARKitSession + data providers) supplies hand tracking, world sensing, etc., and only delivers data while an immersive space is open.

That is the whole architecture. Everything else is detail in the references.

Decision tree → which reference to read

• Setting up the project / Xcode / permissions / Swift 6 concurrency? → references/project-setup.md, then scaffold with the template (below).
• Deciding/working with windows, volumes, immersive spaces, ornaments, opening/dismissing scenes? → references/app-structure.md
• Anything with 3D content — entities, components, systems, RealityView, materials, animation, audio, attachments, portals, particles? → references/realitykit.md
• Authoring/importing 3D assets, USD/USDZ, shader graphs, the content package? → references/reality-composer-pro.md
• Hand tracking, world/plane/scene/image/object/room tracking, world anchors, permissions? → references/arkit.md
• Gestures, taps/drags/rotate/zoom on 3D, eye+hand targeting, hover, accessibility? → references/interaction.md
• Deep immersion, passthrough control, fully-custom Metal rendering (Compositor Services), spatial & immersive video/photos? → references/immersive-and-rendering.md
• Shared experiences (SharePlay, Spatial Personas, GroupActivities) or visionOS Enterprise APIs (camera access, etc.)? → references/shareplay-and-enterprise.md
• Designing it well (spatial HIG, ergonomics, comfort, materials, typography)? → references/design-hig.md
• Performance, profiling (Instruments/RealityKit Trace), testing, App Store submission? → references/performance-testing-shipping.md
• What changed across visionOS 1→2→26→27, or need the primary-source citations? → references/versions-and-sources.md

When the latest API details matter, verify against current Apple documentation (developer.apple.com — the ARKit, RealityKit, and SwiftUI references, plus the "What's new" updates pages) using whatever web-search or docs tools are available — visionOS moves fast and training data may lag a release.

The build workflow

1. Confirm prerequisites (see end of this file). If full Xcode + the visionOS SDK are not installed, say so up front — building or running a visionOS app is impossible without them; the simulator ships inside Xcode.

2. Scaffold from the template rather than hand-rolling project files:

   python3 <skill>/scripts/new_visionos_app.py "AppName" \
       --bundle-id com.yourco.appname --dest ~/Developer/AppName
   

   This produces a verified source tree (a window, a volume, an immersive space, and a Reality Composer Pro package), renames everything, and runs xcodegen generate if XcodeGen is installed. See assets/templates/VisionApp/README.md. Then open AppName.xcodeproj, pick the Apple Vision Pro simulator, and Run.

3. Build features by reading the relevant reference(s) and following their patterns. Keep AppModel (@Observable, @MainActor) as the shared state passed through the SwiftUI environment.

4. Verify behavior before claiming success. Never assume the user can validate the code by reading it — run it. In the simulator: launch the app, open each scene, exercise the feature. Capture a screenshot or describe the observed behavior. When only code edits are possible (e.g. no Xcode here), say so plainly and provide exact steps for the user to verify on their Mac.

5. Profile and tidy before shipping — frame rate, entity/draw-call budgets, memory — using the guidance in references/performance-testing-shipping.md.

High-leverage rules (the things that most often go wrong)

The "why" matters more than the rule — understand it and the rest generalizes.

• ARKit data only flows inside an open immersive space. Hand/world tracking returns nothing from a plain window. Gate provider setup on the space being open, and request authorization (session.requestAuthorization / implicit-on-run) before relying on data. Add the matching usage-description key (NSHandsTrackingUsageDescription, NSWorldSensingUsageDescription) only when actually adopting the provider.

• Stay on the main actor for RealityKit + SwiftUI. Under Swift 6 strict concurrency, entity mutation, RealityView closures, and UI updates belong on @MainActor. Do heavy asset/IO work off-main, then hop back to mutate the scene.

• Immersive-space state is async and can change on its own. The system can dismiss the space (Digital Crown, focus loss). Drive UI from real lifecycle (onAppear/onDisappear of the immersive view), not from the return of openImmersiveSpace, and guard against double open/dismiss. The template's ToggleImmersiveSpaceButton shows the pattern.

• Load content by name from the content bundle, not by file path: try await Entity(named: "Scene", in: realityKitContentBundle). Asset names come from Reality Composer Pro, and loaders are async throws — handle both.

• Position is in meters, the origin is the user, +Y is up, −Z is forward/away. A SIMD3 of [0, 1.5, -1.5] is roughly eye height, 1.5 m in front. Tiny numbers are correct; if something "disappears," it is usually centimeters from the user's face or kilometers away.

• An entity needs InputTargetComponent + a CollisionComponent to receive gestures, and HoverEffectComponent to react to where the user looks. No collision shape → taps/drags silently do nothing.

• The simulator cannot do everything. No real eye/hand input, no passthrough camera, limited tracking. Design for it, but validate genuinely spatial input on device. Note this when only the simulator is available.

• Prefer system materials and standard components (glass backgrounds, hover effects, standard ornaments) — they get correct lighting, accessibility, and ergonomics for free, and keep the app feeling native.

Platform status (verify before relying on it — visionOS moves fast)

As of mid-2026 (research-dated 2026-06-19):

• visionOS 26 is the current shipping line (year-based numbering jumped 2 → 26 at WWDC 2025 to align with iOS/macOS 26); point releases through 26.6.
• visionOS 27 was announced at WWDC 2026 (session 287), shipping fall 2026 — treat its APIs as pre-GA and subject to change.
• App Store: since April 28, 2026, uploads must be built with Xcode 26+ using the visionOS 26 SDK or later.
• Hardware: the M5 Apple Vision Pro ($3,499, Oct 2025) is current; no cheaper/newer headset has shipped. Hand tracking is 90Hz (was 30Hz on visionOS 1).

references/versions-and-sources.md has the full visionOS 1→2→26→27 delta table and the primary-source citation index. Re-confirm "latest" claims against Apple's docs (developer.apple.com, via available web-search tools) before stating them.

Prerequisites & environment

• macOS with Xcode 26+ and the visionOS SDK (Xcode ▸ Settings ▸ Components). The visionOS Simulator is bundled with Xcode.
• Swift 6 language mode (the template enables SWIFT_STRICT_CONCURRENCY).
• Optional but recommended: XcodeGen (brew install xcodegen) for the scaffold script's project generation.
• A physical Apple Vision Pro is needed for true input/passthrough testing but not for development; most work happens in the simulator.

If a required piece is missing, surface it immediately and offer to help install it — do not generate code and imply it ran when it could not.

What's in this skill

visionos-dev/
├── SKILL.md                      (this file — orientation + workflow + map)
├── references/                   (read on demand; see the decision tree)
├── scripts/new_visionos_app.py   (scaffold a renamed copy of the template)
└── assets/templates/VisionApp/   (verified buildable starter: window + volume
                                    + immersive space + RealityKitContent package)