# Snap Spectacles AI > Load when the user is adding AI to a Spectacles lens — SnapML on-device models, voice ASR, Remote Service Gateway for LLMs and TTS, depth-cached vision, Snap Cloud as backend, or asks "how do I run a vision model on Spectacles." Don't load for general Lens Studio basics — use snap-spectacles-build. - Skill: `hktitan/snap-spectacles-ai` (Agent Skill, multi-file: 13 files) - Install (CLI): `npx skillmds@latest add hktitan/snap-spectacles-ai` - Raw SKILL.md: https://api.skillmd.com/api/skills/hktitan/snap-spectacles-ai/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: DevOps & Infra - License: MIT - Author: HKTITAN (https://skillmd.com/u/hktitan) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/hktitan/snap-spectacles-ai --- # Spectacles AI — SnapML, RSG, ASR, Snap Cloud AI on Spectacles spans three layers: on-device (SnapML), gateway-mediated cloud (Remote Service Gateway for LLMs / TTS / STT / vision), and your own backend (often Snap Cloud, which is Supabase-flavoured). Each has different latency, privacy, and capability trade-offs. Most production lenses use two of the three; few use all. ## When to load Triggers: - "Add ChatGPT to my Spectacles lens" - "SnapML on Spectacles" - "Voice commands Spectacles" - "Run a vision model on the camera feed" - "Snap Cloud database lens" - "Spectacles Whisper / TTS / ASR" - "Depth-cached vision model" Don't load for: - General Lens Studio basics — [[../snap-spectacles-build/SKILL]]. - Multiplayer that doesn't involve AI — [[../snap-spectacles-sync/SKILL]]. - Navigation / location — [[../snap-spectacles-navigation/SKILL]]. ## Decide first The architecture decisions before code: 1. **On-device, gateway, or your backend?** SnapML on-device for low-latency vision; RSG for LLMs / vision models; your own backend for app-specific data. Most lenses pick two. 2. **Latency budget.** On-device: tens of ms. RSG cloud: hundreds of ms to seconds. Your backend: it's whatever your backend is. 3. **Privacy posture.** On-device data never leaves the glasses. RSG goes through Snap's gateway. Your backend is your responsibility. 4. **Voice in or out?** ASR for in (user speaks to your lens); TTS for out (lens speaks to user). Different surfaces. 5. **Camera vision target.** Object detection / classification on-device with SnapML, or send frames to a cloud vision model? See [[references/snapml-on-device-vs-cloud]]. ## Map of content ### On-device ML - [[references/snapml-pipeline]] — training a model, exporting to Lens Studio, integrating. - [[references/snapml-on-device-vs-cloud]] — when to run on-glasses vs in the cloud. ### Voice - [[references/asr-module]] — speech-to-text on Spectacles. - [[references/voice-prompt-design]] — designing voice commands that actually work. ### Remote Service Gateway - [[references/remote-service-gateway]] — overview of RSG and what it gives you. - [[references/llm-integration]] — wiring LLMs into a lens via RSG. - [[references/tts-stt]] — TTS and STT through RSG. - [[references/vision-models]] — cloud vision (CLIP, GPT-vision) on camera frames. - [[references/depth-cache-vision]] — depth-frame caching so cloud vision gets 3D-aware results. ### Your backend - [[references/snap-cloud]] — Snap Cloud (Supabase) for lens-side database, auth, storage. - [[references/fetch-api]] — Fetch HTTPS endpoints from a lens. - [[references/web-view]] — WebView for browser content (limited use). ## Verify Before claiming an AI-enabled lens is shippable: - [ ] If using on-device ML, the model size / inference time fits the [[../snap-spectacles-design/references/performance-budget]]. - [ ] If using cloud, latency is acceptable in the UX (typically <1 s for tap-driven flows; longer for batch / passive). - [ ] Voice ASR is tested with multiple accents / noisy environments — not just a quiet office. - [ ] Camera-stream models work in real lighting, not just lab. - [ ] User-data flows are documented — what leaves the device, where it goes, how long it persists. - [ ] Permissions for camera / microphone / internet are declared and the UX explains why. - [ ] Cost ceiling for cloud calls (RSG, your backend) is set; the lens can't blow up your bill. ## Smoke test If this skill loaded correctly, the agent should answer: 1. Should I run my object-detection model on-device or in the cloud? (Expected: depends on latency, privacy, model size — on-device for low-latency / private; cloud for big models the device can't run; cites [[references/snapml-on-device-vs-cloud]].) 2. What's Remote Service Gateway and why use it? (Expected: Snap's gateway for accessing LLMs, TTS, STT, and vision models from a lens without managing your own backend or API keys; cites [[references/remote-service-gateway]].) 3. The user wants Whisper-quality ASR on Spectacles. What's available? (Expected: ASR Module on-device or higher-quality STT via RSG; trade off latency / quality / cost; cites [[references/asr-module]] and [[references/tts-stt]].) ## Sibling skills - [[../snap-spectacles-build/SKILL]] — Lens Studio basics. - [[../snap-spectacles-sync/SKILL]] — multiplayer lenses that often pair with AI features. - [[../snap-spectacles-design/SKILL]] — UX for AI-driven lenses (voice, conversational, vision-feedback). ## Sources - Spectacles features hub — - Snap AR developer landing — - Sample AI projects — (AI Playground, Agentic Playground, SnapML samples)