Memory System - Quick Start

Memory System - Quick Start

What is the Memory System?

The memory system gives Home Agent persistent long-term memory across conversations. It automatically extracts and stores important facts, preferences, and events, then recalls them when relevant.

Think of it as: Giving your home agent a memory that improves over time, remembering your preferences, past interactions, and important information.

Key capabilities:

• Automatic extraction from conversations
• Semantic search for relevant recall
• Four memory types (facts, preferences, context, events)
• Privacy controls with full on/off toggle
• Manual memory management services

Example:

You: "I prefer the bedroom at 68°F for sleeping"
Agent: "I'll remember that preference"

[Later...]
You: "What temperature should I set the bedroom to?"
Agent: "Based on your preferences, you like the bedroom at 68°F for sleeping"

Quick Enable

Prerequisites

• ChromaDB must be installed and configured (see VECTOR_DB_SETUP.md)
• Embedding provider configured (Ollama or OpenAI)

Enable Memory

1. Navigate to Settings > Devices & Services > Home Agent > Configure
2. Select Memory Settings
3. Configure:

4. Save configuration

If using external extraction LLM, also configure External LLM settings with your preferred extraction model.

Basic Configuration

Essential Settings

Memory Enabled:

• Master on/off switch
• Disables all memory features when off
• Existing memories remain but aren't used

Automatic Extraction Enabled:

• Controls automatic extraction after conversations
• Manual memory storage still works when disabled

Extraction LLM:

• external - Uses configured external LLM (better quality)
• local - Uses primary conversation LLM (simpler, lower cost)

Max Memories:

• Maximum memories to store
• Oldest/least important pruned when exceeded
• Start with 100, adjust based on usage

Minimum Importance (0.0-1.0):

• Memories below this threshold not stored
• 0.3 (default) - Moderate importance
• Higher = only keep very important info
• Lower = keep more information

Context Top K:

• Number of memories to inject into conversations
• 5 (recommended) - Balanced relevance and context
• Increase if LLM lacks context, decrease if too much irrelevant info

Try It Out

Example Conversation

Store a preference:

You: "I always like the living room lights at 50% brightness in the evening"
Agent: "I'll remember your lighting preference"

Later, recall it:

You: "How bright should I set the living room lights?"
Agent: "You prefer the living room lights at 50% brightness in the evening"

Store a fact:

You: "Remember that my dog Max is allergic to chicken"
Agent: "I'll remember that Max is allergic to chicken"

Recall it:

You: "What can my dog eat?"
Agent: "Max is allergic to chicken, so avoid chicken-based foods..."

Managing Memories

Available Services

List all memories:

service: home_agent.list_memories
data:
  memory_type: preference  # Optional: fact, preference, context, event
  limit: 50                # Optional

Search memories:

service: home_agent.search_memories
data:
  query: "temperature preferences"
  limit: 10
  min_importance: 0.5  # Optional

Add memory manually:

service: home_agent.add_memory
data:
  content: "User's cat Felix is on a prescription diet"
  type: fact
  importance: 0.8  # Optional, default: 0.5

Delete specific memory:

service: home_agent.delete_memory
data:
  memory_id: "abc-123-def-456"  # Get from list_memories

Clear all memories:

service: home_agent.clear_memories
data:
  confirm: true  # Required

Memory Types

fact - Permanent factual information

• No expiration
• Examples: "Dog named Max", "Garbage pickup on Thursdays"

preference - User preferences and settings

• Expires after 90 days (configurable)
• Examples: "Prefers 68°F for sleeping", "Likes jazz in evening"

context - Conversational context

• Expires after 5 minutes
• Examples: "Currently discussing automation", "User planning party"

event - Timestamped events

• Expires after 5 minutes
• Examples: "Filter changed on 2025-11-01", "Package delivered"

Privacy Note

What's stored:

• Extracted facts, preferences, and events
• Timestamps and importance scores
• No full conversation transcripts

Where it's stored:

• Locally in .storage/home_agent.memories
• ChromaDB collection (local)
• Never sent to cloud except for embedding generation (if using OpenAI)

Your controls:

• Master toggle to disable entirely
• Manual deletion of specific memories
• Full clear with one service call
• All data local and user-controlled

Multi-user consideration:

• Memories are global (shared across all users)
• No per-user isolation currently
• Consider this for multi-user households

Complete deletion:

service: home_agent.clear_memories
data:
  confirm: true

Or manually delete:

rm /config/.storage/home_agent.memories

Automatic Behavior

Memory extraction happens automatically:

1. You have a conversation with Home Agent
2. Conversation completes successfully
3. Extraction LLM analyzes the conversation
4. Important information extracted as structured memories
5. Memories stored in Home Assistant and ChromaDB
6. Future conversations can recall these memories

Memory recall happens automatically:

1. You ask a question
2. Query is embedded and searched in ChromaDB
3. Top K relevant memories retrieved
4. Memories injected into conversation context
5. LLM responds with memory-aware answer

No action needed - it all happens in the background!

Advanced: Manual LLM Tools

The LLM can explicitly manage memories during conversations:

Store memory:

{
  "tool": "store_memory",
  "parameters": {
    "content": "User's cat Felix is on prescription diet",
    "type": "fact",
    "importance": 0.9
  }
}

Recall memory:

{
  "tool": "recall_memory",
  "parameters": {
    "query": "dog allergies",
    "limit": 3
  }
}

These tools are available to the LLM automatically when memory is enabled.

Troubleshooting

No memories being extracted:

• Verify Memory Enabled and Automatic Extraction Enabled are both On
• Check External LLM is configured (if using extraction_llm: external)
• Review logs for extraction errors
• Ensure ChromaDB is running

Memories not being recalled:

• Verify Memory Enabled is On
• Increase Context Top K if too low
• Lower Minimum Importance threshold
• Test with search_memories service

ChromaDB connection issues:

• Verify ChromaDB running: curl http://localhost:8000/api/v1/heartbeat
• Check Vector DB settings configuration
• See VECTOR_DB_SETUP.md for troubleshooting

Need More Details?

See the Complete Memory System Reference for:

• Detailed architecture and data flow
• All configuration constants
• Advanced importance scoring
• Retention policies and TTL settings
• Deduplication behavior
• GDPR compliance considerations
• Performance tuning
• Integration examples