On-Device AI for Apple Platforms
Architect and deploy private on-device machine learning and LLMs on Apple Silicon. Covers framework selection between Apple Foundation Models, Core ML, MLX Swift, and llama.cpp, plus model optimization and quantization.
Scope Boundary: Swift-side Core ML inference and model caching live in coreml. This skill owns backend selection, architecture, model conversion, and LLM runtimes.
Contents
Framework Selection Router
| Framework |
Best For |
Pros |
Constraints |
Apple Foundation Models (iOS 26+; iOS 27 adds images, Private Cloud Compute, custom LanguageModel) |
Text generation, structured output, tool calling |
Zero app download footprint, native system UI integration |
Requires Apple Intelligence eligible device |
| Core AI (iOS 27+) |
Running your own models in Swift; LLMs through a Foundation Models session |
Memory-safe Swift API, specialization cache, Neural Engine/GPU/CPU selection |
iOS 27 only; background Neural Engine needs an entitlement |
| Core ML |
Computer vision, audio, classification, custom transformer models |
Best Neural Engine offload, lowest power consumption |
Fixed compute graph; slower autoregressive text generation |
| MLX Swift |
Custom open-weights LLMs/VLMs (Llama, Mistral, Gemma) |
Full control over model architecture and generation parameters |
Consumes user storage and unified RAM (high memory pressure) |
| llama.cpp |
Cross-platform C++ runtime, CPU fallback |
Broadest quantization support (GGUF), battle-tested |
Higher power usage than Neural Engine pipelines |
Apple Foundation Models
Leverage system-provided generative models on iOS 26+ without bundling model weights:
import FoundationModels
let session = LanguageModelSession()
let response = try await session.respond(to: "Summarize today's highlights in 3 bullet points.")
print(response.text)
Use @Generable to extract structured data directly from model prompts.
iOS 27 keeps the same LanguageModelSession API but lets you swap the model: SystemLanguageModel, PrivateCloudComputeLanguageModel (server model, managed entitlement, quota), CoreAILanguageModel (your own exported weights), or any custom LanguageModel. It also adds image Attachment prompts and DynamicProfile for switching model, tools, and instructions mid-session. Gate with #available(iOS 27, *). Details: Foundation Models in iOS 27.
Core ML & Model Conversion
Convert PyTorch and Hugging Face models using Python coremltools:
import coremltools as ct
import torch
model = MyPyTorchModel().eval()
traced = torch.jit.trace(model, torch.randn(1, 3, 224, 224))
mlmodel = ct.convert(
traced,
inputs=[ct.TensorType(name="input", shape=(1, 3, 224, 224))],
compute_units=ct.ComputeUnit.ALL
)
mlmodel.save("Model.mlpackage")
MLX Swift & Open Weights
Run open-source LLMs leveraging Apple Silicon unified memory:
import MLXLLM
let model = try await LLMModelFactory.shared.load(modelName: "mlx-community/Llama-3.2-3B-Instruct-4bit")
let output = try await model.generate(prompt: "Explain relativity in simple terms.")
Model Compression & Optimization
- Quantization: Compress weights to 4-bit (int4) or 8-bit (int8) via
coremltools.optimize.coreml to reduce memory bandwidth bottlenecks.
- Palettization: Cluster weights into lookup tables for smaller download sizes.
- Pruning: Zero out non-critical weights to accelerate inference.
Common Mistakes
- Using iOS 27 Foundation Models or Core AI types on an iOS 26 target without availability checks: gate with
#available(iOS 27, *) and keep the iOS 26 path.
- Bundling massive LLMs in app bundles: Causes app review rejection or slow downloads. Use Background Assets or system Foundation Models.
- Ignoring device memory limits: Loading a 4-bit 7B LLM requires >4GB RAM, risking immediate OOM termination on devices with 6GB or 8GB unified memory.
- Running autoregressive LLMs without KV-cache: Re-evaluating the full token prompt at each step slows generation exponentially.
- Neglecting thermal throttling: Sustained GPU/CPU inference causes thermal throttling within minutes. Leverage the Neural Engine for sustained workloads.
Review Checklist
References
- Foundation Models API -- LanguageModelSession,
@Generable, tool calling, prompt design
- Foundation Models in iOS 27 --
LanguageModel protocol, Private Cloud Compute, multimodal, Dynamic Profiles, Core AI in a session
- Core AI -- AIModel, InferenceFunction, NDArray, Neural Engine rules
- Core ML Conversion -- Model conversion from PyTorch, TensorFlow, other frameworks
- Core ML Optimization -- Quantization, palettization, pruning, performance tuning
- MLX Swift & llama.cpp -- MLX Swift patterns, llama.cpp integration, memory management
1---2name: apple-on-device-ai3description: "Designs private on-device AI for Apple platforms with Foundation Models (including iOS 27 LanguageModel providers, Private Cloud Compute, multimodal prompts, Dynamic Profiles), Core AI, Core ML, MLX Swift, or llama.cpp. Use for local LLM runtime selection, Apple Intelligence chat or tool use, Apple Silicon inference, model conversion/compression, or backend comparison; route Core ML prediction code to coreml."4metadata:5 source: https://skillmd.com/skills/thiennc-tesoglobal/apple-on-device-ai6---78# On-Device AI for Apple Platforms910Architect and deploy private on-device machine learning and LLMs on Apple Silicon. Covers framework selection between Apple Foundation Models, Core ML, MLX Swift, and llama.cpp, plus model optimization and quantization.1112> **Scope Boundary:** Swift-side Core ML inference and model caching live in `coreml`. This skill owns backend selection, architecture, model conversion, and LLM runtimes.1314## Contents1516- [Framework Selection Router](#framework-selection-router)17- [Apple Foundation Models](#apple-foundation-models)18- [Core ML & Model Conversion](#core-ml--model-conversion)19- [MLX Swift & Open Weights](#mlx-swift--open-weights)20- [Model Compression & Optimization](#model-compression--optimization)21- [Common Mistakes](#common-mistakes)22- [Review Checklist](#review-checklist)23- [References](#references)2425## Framework Selection Router2627| Framework | Best For | Pros | Constraints |28|---|---|---|---|29| **Apple Foundation Models** (iOS 26+; iOS 27 adds images, Private Cloud Compute, custom `LanguageModel`) | Text generation, structured output, tool calling | Zero app download footprint, native system UI integration | Requires Apple Intelligence eligible device |30| **Core AI** (iOS 27+) | Running your own models in Swift; LLMs through a Foundation Models session | Memory-safe Swift API, specialization cache, Neural Engine/GPU/CPU selection | iOS 27 only; background Neural Engine needs an entitlement |31| **Core ML** | Computer vision, audio, classification, custom transformer models | Best Neural Engine offload, lowest power consumption | Fixed compute graph; slower autoregressive text generation |32| **MLX Swift** | Custom open-weights LLMs/VLMs (Llama, Mistral, Gemma) | Full control over model architecture and generation parameters | Consumes user storage and unified RAM (high memory pressure) |33| **llama.cpp** | Cross-platform C++ runtime, CPU fallback | Broadest quantization support (GGUF), battle-tested | Higher power usage than Neural Engine pipelines |3435## Apple Foundation Models3637Leverage system-provided generative models on iOS 26+ without bundling model weights:3839```swift40import FoundationModels4142let session = LanguageModelSession()43let response = try await session.respond(to: "Summarize today's highlights in 3 bullet points.")44print(response.text)45```4647Use `@Generable` to extract structured data directly from model prompts.4849iOS 27 keeps the same `LanguageModelSession` API but lets you swap the model: `SystemLanguageModel`, `PrivateCloudComputeLanguageModel` (server model, managed entitlement, quota), `CoreAILanguageModel` (your own exported weights), or any custom `LanguageModel`. It also adds image `Attachment` prompts and `DynamicProfile` for switching model, tools, and instructions mid-session. Gate with `#available(iOS 27, *)`. Details: [Foundation Models in iOS 27](references/foundation-models-ios27.md).5051## Core ML & Model Conversion5253Convert PyTorch and Hugging Face models using Python `coremltools`:5455```python56import coremltools as ct57import torch5859model = MyPyTorchModel().eval()60traced = torch.jit.trace(model, torch.randn(1, 3, 224, 224))6162mlmodel = ct.convert(63 traced,64 inputs=[ct.TensorType(name="input", shape=(1, 3, 224, 224))],65 compute_units=ct.ComputeUnit.ALL66)67mlmodel.save("Model.mlpackage")68```6970## MLX Swift & Open Weights7172Run open-source LLMs leveraging Apple Silicon unified memory:7374```swift75import MLXLLM7677let model = try await LLMModelFactory.shared.load(modelName: "mlx-community/Llama-3.2-3B-Instruct-4bit")78let output = try await model.generate(prompt: "Explain relativity in simple terms.")79```8081## Model Compression & Optimization8283- **Quantization**: Compress weights to 4-bit (int4) or 8-bit (int8) via `coremltools.optimize.coreml` to reduce memory bandwidth bottlenecks.84- **Palettization**: Cluster weights into lookup tables for smaller download sizes.85- **Pruning**: Zero out non-critical weights to accelerate inference.8687## Common Mistakes8889- **Using iOS 27 Foundation Models or Core AI types on an iOS 26 target without availability checks**: gate with `#available(iOS 27, *)` and keep the iOS 26 path.90- **Bundling massive LLMs in app bundles**: Causes app review rejection or slow downloads. Use Background Assets or system Foundation Models.91- **Ignoring device memory limits**: Loading a 4-bit 7B LLM requires >4GB RAM, risking immediate OOM termination on devices with 6GB or 8GB unified memory.92- **Running autoregressive LLMs without KV-cache**: Re-evaluating the full token prompt at each step slows generation exponentially.93- **Neglecting thermal throttling**: Sustained GPU/CPU inference causes thermal throttling within minutes. Leverage the Neural Engine for sustained workloads.9495## Review Checklist9697- [ ] iOS 27-only APIs (PCC model, multimodal, Dynamic Profiles, Core AI) gated by availability, with fallback98- [ ] Appropriate framework selected for latency, power, and storage constraints99- [ ] System Foundation Models prioritized when Apple Intelligence features suffice100- [ ] Memory footprint measured on the lowest supported physical device target101- [ ] Models quantized to 4-bit or 8-bit to fit within memory budgets102- [ ] Neural Engine dispatch verified using `MLComputePlan` or Instruments103104## References105106- [Foundation Models API](references/foundation-models.md) -- LanguageModelSession, `@Generable`, tool calling, prompt design107- [Foundation Models in iOS 27](references/foundation-models-ios27.md) -- `LanguageModel` protocol, Private Cloud Compute, multimodal, Dynamic Profiles, Core AI in a session108- [Core AI](references/core-ai.md) -- AIModel, InferenceFunction, NDArray, Neural Engine rules109- [Core ML Conversion](references/coreml-conversion.md) -- Model conversion from PyTorch, TensorFlow, other frameworks110- [Core ML Optimization](references/coreml-optimization.md) -- Quantization, palettization, pruning, performance tuning111- [MLX Swift & llama.cpp](references/mlx-swift.md) -- MLX Swift patterns, llama.cpp integration, memory management