AI Assistant Documentation

AI Assistant Documentation

Overview

The AI Assistant add-on brings enterprise-grade AI to your Home Assistant instance. It provides a web-based chat interface with multi-provider AI support (Anthropic Claude, OpenAI GPT, Google Gemini, NVIDIA NIM, GitHub Models), file access capabilities, persistent memory, document analysis, and more.

The add-on integrates seamlessly with Home Assistant's Supervisor API, requiring no long-lived tokens—just your AI provider API keys.

Features

Core Features

• Streaming chat UI with real-time responses
• Multi-provider support: Anthropic, OpenAI, Google Gemini, NVIDIA NIM, GitHub Models (40+ models)
• Model switching: Change AI providers and models on-the-fly without restarting
• Persistent model selection: Your chosen agent is saved and restored after restart
• Multi-language UI: English, Italian, Spanish, French
• Home Assistant integration: Read device states, call services directly from chat

Advanced Features (Experimental)

• File Upload & Analysis (v3.3.0+): Upload PDF, DOCX, TXT, MD, YAML files for AI analysis
• Persistent Memory (v3.3.0+): AI remembers past conversations across sessions
• RAG (Retrieval-Augmented Generation) (v3.3.0+): Semantic search over uploaded documents
• File Access: Optional read/write access to /config directory with automatic snapshots

AI Providers

Anthropic Claude

• Models: Claude 3.5 Sonnet, Haiku, Opus
• Cost: ~$3/$0.80/$15 per 1M tokens
• Setup: Get API key from console.anthropic.com
• Best for: Complex reasoning, creative tasks, long context

OpenAI

• Models: GPT-4o, GPT-4 Turbo, GPT-4, GPT-3.5 Turbo, o1-preview, o3-mini
• Cost: $5-$20 per 1M tokens (varies by model)
• Setup: Create API key at platform.openai.com
• Best for: Balanced performance, variety of models, tools integration

Google Gemini

• Models: Gemini 2.0 Flash, Pro, Pro Vision
• Cost: Free tier available, ~$7.50 per 1M tokens (paid)
• Setup: Get API key from ai.google.dev
• Best for: Fast responses, vision capabilities, free tier users

NVIDIA NIM

• Models: NVIDIA Llama 3.1 405B, Mistral, Mixtral, etc.
• Cost: Free tier (rate-limited)
• Setup: Get API key from build.nvidia.com
• Thinking Mode: Available on supported models (opt-in)
• Best for: Open-source models, free inference, high throughput

GitHub Models

• Models: OpenAI o1-preview, o3-mini, GPT-4o, Llama, Phi, etc.
• Cost: Free for GitHub users (token limits apply)
• Setup: Use GitHub PAT (fine-grained, no special permissions needed)
• Best for: GitHub users, experimental models, free access

Installation

1. Add Repository:

   • Settings → Add-ons & Backups → Add-on Store → ⋮ → Repositories
   • Add: https://github.com/Bobsilvio/ha-claude

2. Install Add-on:

   • Search for "AI Assistant"
   • Click AI Assistant for Home Assistant
   • Click Install

3. Configure & Start:

   • Open the Configuration tab
   • Add at least one provider API key
   • Click Save and Start

Configuration

Required

At least one AI provider API key is required. Choose based on:

• Anthropic API Key: Best all-rounder, most reliable
• OpenAI API Key: Variety of models, popular choice
• Google API Key: Free options available
• NVIDIA API Key: Open-source models, free tier
• GitHub Token: Free for GitHub users

Optional Features

Using the Chat

First Launch

1. Open AI Assistant from the Home Assistant sidebar
2. Click the model dropdown (top left of chat area)
3. Select an agent/model (e.g., "OpenAI → GPT-4o")
4. Start chatting

Model Switching

• Click the model dropdown to switch providers/models instantly
• Selection is saved automatically and persists across restarts

Home Assistant Integration

Ask the AI about your smart home:

• Device states: "What's the current garage door status?"
• Services: "Turn on the living room lights"
• Automations: "Show me my evening routine automation"

The AI reads states and can trigger actions directly from chat.

File Upload (Experimental)

If enable_file_upload: true:

1. Click the file upload button (orange, in input area)
2. Select a document (PDF, DOCX, TXT, MD, YAML)
3. Documents are auto-injected into AI context

Files are cleaned up after use. Upload limit: 10MB per file.

Persistent Memory (Experimental)

If enable_memory: true:

• AI remembers past conversations across sessions
• Memory searches by keywords and message content
• Old conversations are kept (never deleted)
• Memory is local (no cloud sync)

Document Search (RAG)

If enable_rag: true and enable_file_upload: true:

• Upload documents to build a knowledge base
• AI performs semantic search over documents
• Results automatically injected into prompts

Advanced Configuration

File Access

Requires enable_file_access: true in config.

Snapshot & Restore Mechanism:

• When you edit /config via chat, the add-on creates an automatic backup
• If edit fails or causes issues, you can restore the last snapshot
• Snapshots are kept in /config/.claude_backups/

Example Uses:

• Edit configuration.yaml to add automations
• Update secrets in secrets.yaml
• Write custom Python scripts to python_scripts/

Logging & Debugging

Set log_level to control verbosity:

• normal (default): Core messages only (clean logs)
• verbose: Includes all API request/response logs
• debug: Maximum detail, including internal state

Use via YAML config or environment variable LOG_LEVEL.

Troubleshooting

Chat UI doesn't load

1. Verify add-on is Running (check green status in Add-ons)
2. Check add-on logs for errors
3. Hard-refresh browser (Ctrl+F5 / Cmd+Shift+R)
4. Restart Home Assistant if persists

API errors (401, 403, 429)

• 401: API key is invalid → Check provider account and key format
• 403: Permissions issue → Verify API key has chat/inference permissions
• 429: Rate limited → Wait or upgrade to higher tier with provider

Home Assistant integration not working

• Check /api/status endpoint (add /ha-claude/ to HA URL)
• Verify HA connection status shows ok
• Restart add-on if status shows error

File Upload not working

• Ensure enable_file_upload: true andSave config
• Verify file is < 10MB
• Check supported formats: PDF, DOCX, DOC, TXT, MD, YAML, YML, ODT
• Restart add-on if issues persist

Memory feature not saving

• Ensure enable_memory: true in config
• Check /config/.claude_memory/ folder exists
• Restart add-on
• Check add-on logs for permission errors

Module import errors

• Example: ModuleNotFoundError: No module named 'PyPDF2'
• Solution: Restart the add-on (dependencies are installed on start)

API Reference

The add-on exposes a REST API at /ha-claude/api/ (or direct port 5010 if not using Ingress).

Chat Endpoints

POST /api/chat/stream

• Stream-based chat API
• Request body: {"message": "...", "conversation_id": "..." (optional)}
• Response: Server-Sent Events (SSE) with streamed tokens

GET /api/models

• List available providers and models
• Returns: {"providers": {...}, "models": [...]}

POST /api/set_model

• Change active provider/model
• Request body: {"provider": "openai", "model": "gpt-4o"}
• Returns: {"success": true}

GET /api/status

• System status (HA connection, feature flags, version)
• Returns: {"ha_connection_ok": true, "version": "3.3.0", ...}

Document Endpoints (File Upload/RAG)

POST /api/documents/upload

• Upload document for analysis
• Multipart form-data with file
• Returns: {"success": true, "filename": "...", "pages": N}

DELETE /api/documents/{filename}

• Remove uploaded document
• Returns: {"success": true}

GET /api/documents

• List uploaded documents
• Returns: {"documents": [...]}

Memory Endpoints

GET /api/memory/search

• Search conversation history
• Query params: q=<query>
• Returns: {"results": [...]}

DELETE /api/memory/clear

• Clear all memory (use with caution!)
• Returns: {"success": true}

For full API details, visit the /api/ endpoint directly.

Home Assistant Integration Examples

Read Device State

"What's the current temperature in the living room?"

• AI reads the temperature sensor state from HA

Call Service

"Turn off the bedroom lights"

• AI calls the light.turn_off service on bedroom lights

Complex Automation

"Create an automation that turns on kitchen lights when motion is detected after sunset"

• AI reads existing automations and can write new ones to configuration.yaml

Performance Tips

1. Streaming: Modern models stream responses (GPT-4o, Claude 3.5) → faster perceived performance
2. Model Size: Smaller models (o3-mini, Llama 3.1 70B) are faster but less capable
3. Timeout Setting: Increase if using complex reasoning or long documents
4. Document Size: Keep uploaded files < 5MB for best performance
5. Memory Size: Limit active memory by clearing old conversations occasionally

Security Notes

• API Keys: Stored in HA configuration, never exposed to UI
• File Access: Only reads/writes under /config directory
• Ingress: All traffic through HA Ingress by default (no direct internet exposure)
• Memory: Local only, no cloud sync
• Documents: Deleted after use, not persisted

Support

• Issues: https://github.com/Bobsilvio/ha-claude/issues
• Discussions: https://github.com/Bobsilvio/ha-claude/discussions
• Repository: https://github.com/Bobsilvio/ha-claude

Changelog

See CHANGELOG.md for version history and updates.