tools-only Whisper CoreML Acceleration
Technical documentation for CoreML acceleration in VoiceMode's Whisper implementation.
Overview
CoreML acceleration provides significant performance improvements for Whisper speech-to-text on Apple Silicon Macs, delivering 2-3x faster transcription speeds compared to Metal-only acceleration.
Performance Comparison
| Acceleration Type | Processing Speed | Relative Performance | CPU Usage |
|---|---|---|---|
| CPU Only | ~1x (baseline) | 100% | High |
| Metal (GPU) | ~3-4x | 300-400% | Medium |
| CoreML + Metal | ~8-12x | 800-1200% | Low |
Example: Processing 7.6 seconds of audio in 0.2 seconds (38x real-time) with base.en model
Technical Architecture
CoreML Integration
VoiceMode uses whisper.cpp with CoreML support through:
- Model Conversion: GGML models are converted to CoreML format during installation
- Runtime Selection: whisper.cpp automatically selects CoreML when available
- Fallback Support: Falls back to Metal acceleration if CoreML fails
Compilation Requirements
CoreML acceleration requires specific build-time dependencies:
- Full Xcode Installation: Command Line Tools alone insufficient
- Core ML Tools: For model conversion (coremltools Python package)
- PyTorch: Required for model conversion pipeline (~2.5GB download)
Installation Process
- Dependency Check: Verify Xcode and coremlc availability
- PyTorch Installation: Download and install PyTorch via pip
- Model Download: Fetch GGML model from Hugging Face
- CoreML Conversion: Convert GGML → CoreML using coremltools
- Verification: Test CoreML model loading and performance
Current Implementation Status
Why CoreML is Currently Disabled
As of the current installer, CoreML installation is commented out due to:
# Lines 841-842 and 871-872 in install script
# DISABLED: CoreML build issues - users getting errors at 3:30am
# setup_coreml_acceleration
Issues Encountered
- Late-night Installation Failures: Users reported errors during off-hours
- Xcode Detection Problems: Inconsistent Xcode installation detection
- PyTorch Download Timeouts: Large download size causing timeouts
- Dependency Conflicts: Version conflicts between system and pip packages
Re-enabling CoreML (Technical Guide)
1. Prerequisites Verification
# Check for full Xcode installation
if [[ -f "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/coremlc" ]]; then
echo "✓ Full Xcode detected"
else
echo "✗ Full Xcode required (not just Command Line Tools)"
fi
# Verify Xcode license acceptance
xcodebuild -checkFirstLaunchStatus
2. Model Installation Commands
CLI Method
# Install base model with CoreML conversion
voicemode whisper model install base.en --skip-core-ml=false --install-torch=true
# Force reinstallation with CoreML
voicemode whisper model install base.en --force-download --install-torch=true
MCP Tool Method
whisper_model_install(
model="base.en",
force_download=True,
skip_core_ml=False,
install_torch=True,
auto_confirm=True
)
3. Troubleshooting Common Issues
"CoreML not available" Error
- Verify Xcode installation path:
/Applications/Xcode.app/Contents/Developer - Check coremlc binary exists and is executable
- Ensure xcode-select points to full Xcode, not Command Line Tools
PyTorch Installation Failures
- Use system Python instead of conda/pyenv
- Clear pip cache:
pip cache purge - Install with specific index:
--extra-index-url https://download.pytorch.org/whl/cpu
Model Conversion Failures
- Verify adequate disk space (>5GB free)
- Check /tmp directory permissions
- Monitor conversion logs for specific error messages
File Locations
Models Directory
~/.voicemode/services/whisper/models/
├── ggml-base.en.bin # Original GGML model
├── ggml-base.en-encoder.mlmodel # CoreML encoder
└── ggml-base.en-decoder.mlmodel # CoreML decoder (if applicable)
Configuration Files
- Service config:
~/.voicemode/services/whisper/config.json - Model preferences:
~/.voicemode/whisper-models.txt - Installation logs:
~/.voicemode/services/whisper/logs/
Performance Tuning
Model Selection for CoreML
| Model Size | GGML Size | CoreML Size | Accuracy | Speed | Use Case |
|---|---|---|---|---|---|
| tiny.en | 39MB | ~150MB | Good | Fastest | Real-time |
| base.en | 142MB | ~400MB | Better | Fast | Balanced |
| small.en | 466MB | ~1.2GB | Best | Good | Accuracy |
Runtime Optimization
# Set optimal thread count (usually CPU cores)
export WHISPER_CPP_THREADS=8
# Enable CoreML explicitly (if detection fails)
export WHISPER_COREML=1
# Set model path explicitly
export WHISPER_MODEL_PATH="~/.voicemode/services/whisper/models/ggml-base.en.bin"
Development Notes
Code Locations
- MCP Tool:
voice_mode/tools/services/whisper.py - Model Management:
voice_mode/tools/whisper_model_install.py - Configuration:
voice_mode/config.py - Service Integration:
voice_mode/providers.py
Future Improvements
- Better Error Handling: More specific error messages for common failures
- Partial Installation: Allow Metal-only fallback during CoreML failures
- Model Validation: Verify CoreML model integrity after conversion
- Size Optimization: Investigate quantization for smaller CoreML models
- Auto-retry Logic: Automatic retry with fallback options
Security Considerations
- CoreML models run in Apple's secure sandbox
- No additional network access required after installation
- Models stored in user directory with standard permissions
- No elevation required for CoreML runtime (unlike some CUDA setups)
Monitoring and Logging
Performance Metrics
# Check actual acceleration being used
voicemode whisper status
# View performance statistics
voicemode statistics
# Monitor real-time processing
tail -f ~/.voicemode/services/whisper/logs/performance.log
Debug Information
# Enable debug logging
export VOICEMODE_DEBUG=true
# Test CoreML functionality
voicemode whisper model test base.en
# Benchmark different acceleration methods
voicemode whisper benchmark --models base.en --acceleration metal,coreml
This document should be updated as CoreML support evolves and installation issues are resolved.