Vision Framework

Vision Framework

Detect text, faces, barcodes, objects, contours, and poses in images and live video using Apple's on-device computer vision frameworks (Vision and VisionKit). Targets Swift 6.3 / iOS 26+.

Contents

• Two API Generations
• Modern Request Architecture
• Normalized Coordinates vs Pixel Space
• Core ML Integration
• Route by Task
• Common Mistakes
• Review Checklist
• References

Two API Generations

Prefer modern Swift-native request structs for new code. Maintain legacy handlers only when supporting deployment targets below iOS 18.

Modern Request Architecture

All modern requests conform to ImageProcessingRequest:

import Vision

var request = RecognizeTextRequest()
request.recognitionLevel = .accurate
request.recognitionLanguages = [Locale.Language(identifier: "en-US")]

let observations = try await request.perform(on: cgImage)
for observation in observations {
    print("Found text: \(observation.topCandidates(1).first?.string ?? "")")
}

Normalized Coordinates vs Pixel Space

Vision observations express bounding boxes in normalized coordinates (0.0...1.0) with origin at the bottom-left corner (Cartesian), while UIKit/SwiftUI places origin at the top-left.

To convert to UIKit/SwiftUI coordinates:

let rect = VNImageRectForNormalizedRect(observation.boundingBox, Int(viewWidth), Int(viewHeight))
// Flip Y-axis: y = viewHeight - rect.origin.y - rect.size.height

Core ML Integration

Run custom Core ML vision models using CoreMLModelContainer:

1. Compile model (.mlpackage or .mlmodelc).
2. Wrap with CoreMLModelContainer(model: compiledModel).
3. Create image request: var request = ImageFeaturePrintRequest() or custom Core ML classification request.

Route by Task

• For OCR text recognition, barcode scanning, face detection, person segmentation, and object tracking, read Vision Requests and Detectors.
• For live camera scanner UI, document scanning, and optical barcode flows with DataScannerViewController, read VisionKit Scanner.

Common Mistakes

• Forgetting to invert the Y-axis when projecting normalized Vision bounding boxes onto UIKit/SwiftUI views.
• Running heavy Vision requests (.accurate text recognition) synchronously on the main thread.
• Passing UIImage directly without extracting cgImage or preserving image orientation metadata.
• Using DataScannerViewController without checking DataScannerViewController.isSupported and isAvailable.
• Retaining stateful tracking requests (TrackObjectRequest) across unrelated image sequences.

Review Checklist

• Modern request types (RecognizeTextRequest) preferred on iOS 18+ targets
• Image orientation properly passed to perform(on:orientation:)
• Vision bounding box coordinates converted and Y-flipped for SwiftUI rendering
• Camera scanner checks DataScannerViewController.isSupported before presentation
• Heavy processing executed on background tasks off @MainActor
• Bounding boxes clamped to image bounds before cropping

References

• Vision requests, legacy VNRequest patterns, and Core ML integration
• VisionKit DataScannerViewController and document scanner
• Vision documentation
• VisionKit documentation
• RecognizeTextRequest