# 1\. Core Concepts & Architecture > This guide provides a deep dive into the architecture and implementation of the conversational memory features within WilmerAI. - Skill: `tools-only/1-core-concepts-and-architecture-3` (Agent Skill, multi-file: 3 files) - Install (CLI): `npx skillmds@latest add tools-only/1-core-concepts-and-architecture-3` - Raw SKILL.md: https://api.skillmd.com/api/skills/tools-only/1-core-concepts-and-architecture-3/raw - Safety review: pending (external: skill-scanner PASS, skillspector PASS) - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: tools-only (https://skillmd.com/u/tools-only) - Updated: 2026-09-29 - Page: https://skillmd.com/skills/tools-only/1-core-concepts-and-architecture-3 --- ### **Developer Guide: WilmerAI Memory System** This guide provides a deep dive into the architecture and implementation of the conversational memory features within WilmerAI. It has been updated to reflect the powerful workflow-based memory generation system, the specifics of the vector search implementation, and the complete data schemas as verified by the current codebase. ----- ## 1\. Core Concepts & Architecture WilmerAI's memory system is a sophisticated, multi-layered feature set implemented through specialized nodes within the workflow engine. This design allows for different types of memory operations—creation, retrieval, and summarization—to be strategically placed within workflows. #### **Key Architectural Principles:** * **Activation via `discussionId`**: All persistent, stateful memory features are tied to a `discussionId`. Its presence activates persistent storage mechanisms; its absence causes memory nodes to fall back to stateless, in-memory operations using the current chat history. * **Separation of Concerns: Creators vs. Retrievers**: The system is fundamentally split into two categories of operations, handled by distinct services: * **Memory Creation (Write)**: These are computationally expensive processes that analyze conversation history, generate new summarized memories, and write them to persistent storage. This is the exclusive responsibility of the **`SlowButQualityRAGTool`**. This tool can generate memories either by executing a full sub-workflow for complex logic or by making a direct LLM call. * **Memory Retrieval (Read)**: These are inexpensive, fast operations that read from storage or the current chat history to provide context for an LLM. This is the primary responsibility of the **`MemoryService`**. * **Node-Based Implementation**: Each memory operation is defined as a specific `node_type` in a workflow's JSON configuration. This gives developers explicit control over when to perform expensive write operations versus cheap read operations. * **Centralized Routing and Registration**: * The **`MemoryNodeHandler`** acts as a central router, directing requests for different memory `node_type`s to the appropriate service (`MemoryService` or `SlowButQualityRAGTool`). * The **`WorkflowManager`** acts as the system registrar where new memory `node_type`s must be mapped to the `MemoryNodeHandler`. * **Persistent Storage**: When a `discussionId` is active, the system maintains state in a user-specific directory using a set of discussion-specific files: 1. **Memory File (`_memories.jsonl`)**: Stores discrete, summarized chunks of the conversation for file-based memory. Each chunk is saved with a hash of the last message it's based on, creating a traceable, append-only ledger. 2. **Chat Summary File (`_summary.jsonl`)**: Stores a single, continuously updated "rolling summary" of the entire conversation. It's linked via a hash to the last memory chunk from the memory file that it incorporated. 3. **Vector Memory Database (`_vector_memory.db`)**: A **discussion-specific SQLite database** created on-demand for each `discussionId`. It uses the FTS5 extension for powerful, weighted, full-text search across two main tables: * **`memories` table**: Stores the ground-truth data. For vector memories, the `memory_text` column holds the * *LLM-generated summary**, not the raw conversation chunk. It also stores the full `metadata_json`. * **`memories_fts` table**: A virtual table that indexes the metadata for fast searching. The indexed columns are `title`, `summary`, `entities`, `key_phrases`, and the original `memory_text`. Search relevance is determined by the **`bm25`** ranking function. * **Recency Scoring**: The database connection is initialized with a custom SQL function, `recency_score`, which can calculate a time-decay boost for memories. While the default search query uses only `bm25` ranking, this function is available for developers to implement time-sensitive search ranking logic. 4. **Vector Memory Tracker (`vector_memory_tracker` table)**: Located inside the `_vector_memory.db`, this table stores the hash of the last message processed for vector memory creation. This crucial feature prevents the system from re-processing the same conversation history on subsequent runs. ----- ## 2\. Anatomy of Memory Node Types Understanding the creator/retriever pattern clarifies the role of each node type. #### **Memory Creation & Persistence (Write)** These nodes perform the "heavy lifting" of creating and saving memories. They are powered by **`SlowButQualityRAGTool` **. * **`QualityMemory`**: This is the primary **memory creator** node. It's designed to be run periodically in a workflow to keep persistent memory up-to-date. * **Process Flow**: 1. The node triggers `SlowButQualityRAGTool.handle_discussion_id_flow`. 2. The tool checks if new memories are needed by comparing the current conversation history against the last processed point. 3. **(Correction Start)** It checks the `useVectorForQualityMemory` flag from the config to determine its behavior: if `true`, it generates **vector memories**; if `false`, it generates **file-based memories**. This flag controls the behavior of the `QualityMemory` node specifically. Other nodes (like `RecentMemory`) can trigger the creation of file-based memories independently of this flag's value. **(Correction End)** 4. For the chosen memory type, the tool generates memories using one of two methods, based on its configuration: * **A) Workflow-Based Generation (Recommended)** ✨: If the config specifies a `fileMemoryWorkflowName` or `vectorMemoryWorkflowName`, the tool executes that sub-workflow. * **File-Based Workflow**: Passes the raw text chunk, recent memories, full memories, and the current chat summary as `scoped_inputs`. The workflow's final output **must be a single summarized text block **. * **Vector-Based Workflow**: Passes only the raw text chunk as a `scoped_input`. The workflow's final output **must be a JSON string representing a single object or an array of objects**. * **B) Direct LLM Call (Legacy)**: If a workflow name is not provided, the system falls back to a direct LLM call using prompts defined in the config. #### **Memory Retrieval (Read)** These nodes perform fast, inexpensive "read" operations and are powered by **`MemoryService`**. * **`RecentMemory` & `RecentMemorySummarizerTool`**: The primary **file-based memory retriever**. * **With `discussionId` (Stateful)**: Calls `MemoryService.get_recent_memories`, which reads the last `N` memory chunks from `_memories.jsonl`. * **Without `discussionId` (Stateless)**: Falls back to an in-memory operation, grabbing the last `N` turns from the current `messages` list. * **`VectorMemorySearch`**: The primary **RAG search node**. * **Purpose**: Performs a highly relevant, keyword-based search against the discussion-specific vector memory database. * **Process Flow**: This node takes a string of keywords. The keywords **must be separated by semicolons (`;`)**. The `MemoryService` calls `vector_db_utils.search_memories_by_keyword`. * **Keyword Handling**: Internally, this process is robust: * **Sanitization**: Each keyword is passed through the `_sanitize_fts5_term` function, which wraps it in double quotes (`"`) to handle multi-word phrases and prevent FTS5 syntax errors. * **Query Construction**: The sanitized terms are combined into a `MATCH` query using **`OR` logic**. * **Limits**: The system truncates the keyword list to a maximum of `60` (`MAX_KEYWORDS_FOR_SEARCH`) to avoid exceeding SQLite's expression depth limit. * **Ranking**: The final results are ranked by relevance using the `bm25` algorithm. * **`FullChatSummary`**: Retrieves the holistic, rolling summary of the conversation from `_summary.jsonl`. ----- ## 3\. Critical Files for Development To modify or extend the memory system, you will primarily work with these five files: 1. **`Middleware/workflows/tools/slow_but_quality_rag_tool.py`**: The heart of **memory creation**. Modify this file to change how memories are generated, including the logic for choosing between vector/file and workflow/LLM-call methods. 2. **`Middleware/services/memory_service.py`**: The core of **memory retrieval**. Modify this file to change how memories are read from `.jsonl` files or retrieved from the vector database. 3. **`Middleware/utilities/vector_db_utils.py`**: The **database abstraction layer**. This contains all logic for interacting with the discussion-specific SQLite databases, including table creation, data insertion, FTS5 search query construction, and term sanitization. 4. **`Middleware/workflows/handlers/impl/memory_node_handler.py`**: The **central router**. You must update this file's `handle` method to route any new `node_type` you create to the correct service or tool. 5. **`Middleware/workflows/managers/workflow_manager.py`**: The **system registrar**. You must register your new `node_type` in the `node_handlers` dictionary within this manager's constructor. This manager is also responsible for executing the memory generation sub-workflows. ----- ## 4\. Memory Generation Triggering Logic This section details the internal logic that determines when new file-based memories are generated. Understanding this is essential for debugging memory timing issues. ### Key Configuration Fields The triggering logic is governed by four fields in the discussion ID workflow configuration file: * **`chunkEstimatedTokenSize`** (default: 1000): The estimated token count threshold. When the accumulated new messages reach or exceed this many tokens, memory generation triggers. * **`maxMessagesBetweenChunks`** (default: 5): The message count threshold. When this many new messages have accumulated, memory generation triggers. This threshold only applies when the memory file exists on disk (standard mode). It is disabled when the file does not exist (consolidation mode). * **`maxResponseSizeInTokens`** (default: 400): The maximum number of tokens in the LLM's summarization response. * **`lookbackStartTurn`** (default: 3): The number of most recent conversation turns to exclude from memory processing. This prevents recent, potentially incomplete exchanges (such as frontend command messages) from being included in memories. ### Triggering Modes The system operates in two modes depending on whether the memory file exists on disk: #### Standard Mode (Memory File Exists) Both thresholds are active. Memory generation triggers when **either** threshold is reached first: * `estimated_tokens >= chunkEstimatedTokenSize`, OR * `message_count >= maxMessagesBetweenChunks` This "whichever comes first" logic allows the system to generate memories based on either volume (tokens) or frequency (messages), depending on which condition is met first. The memory file does not need to contain any chunks for standard mode to apply — it just needs to exist on disk. In a new conversation, the file is created (as an empty `[]`) on the first check (around message 3, after the early return for conversations with fewer than 3 messages). From that point on, standard mode applies and both thresholds are active. #### Consolidation Mode (Memory File Does Not Exist) Only the token threshold applies. The message count threshold is disabled. This mode activates only when the memory file has been deleted from disk — for example, when a user wants to regenerate memories with a larger chunk size to produce fewer, larger chunks. ### Implementation Details The triggering logic lives in `SlowButQualityRAGTool.handle_discussion_id_flow()`, within the file-based memory path (the `else` branch after the `use_vector_memory_from_config` check). 1. **File existence check**: Before reading, the system checks `os.path.exists(filepath)` to determine whether the memory file exists on disk. This check must happen before `read_chunks_with_hashes()`, which creates the file as an empty `[]` if it does not exist (see "File Creation Side-Effect" below). 2. **New message detection**: The method uses hash-based tracking (`find_last_matching_hash_message`) to identify messages that have been added since the last memory generation run. Only unprocessed messages are considered. If the file has no chunks (new chat or freshly created file), all messages minus the lookback are treated as new. 3. **Token estimation**: The content of all new messages is joined and passed through `rough_estimate_token_length()` to get an approximate token count. This function intentionally overestimates by using the higher of a word-based estimate (1.35 tokens/word) and a character-based estimate (3.5 chars/token), then applying a `safety_margin` multiplier (default 1.10). See `text_utils.py` for details. 4. **Trigger evaluation**: The two boolean conditions are evaluated. `trigger_by_messages` is gated on `file_exists` (from step 1). Diagnostic logging reports the exact counts, thresholds, and which trigger(s) fired (or why none did). ### File Creation Side-Effect The `read_chunks_with_hashes()` function calls `ensure_json_file_exists()`, which creates the memory file as an empty JSON array (`[]`) if it does not exist on disk. This is why the `file_exists` check must happen before that call — otherwise the file would always appear to exist. The side-effect is intentional: it means that on the first memory check of a new conversation (around message 3), the file is created, and from message 4 onward the system operates in standard mode with both thresholds active. ----- ## 5\. How to Add a New Memory Feature The system is designed for extension. Below are two common scenarios. #### **Example A: Creating a New Memory *Retriever*** **Scenario**: Create a new retriever node called `FirstFiveMemories` that always gets the *first* five memories of a conversation to provide context on its origin. 1. **Implement the Logic (in the Retriever)**: Open `Middleware/services/memory_service.py` and add the new method. ```python # In MemoryService class def get_first_five_memories(self, discussion_id: str) -> str: filepath = get_discussion_memory_file_path(discussion_id) hashed_chunks = read_chunks_with_hashes(filepath) if not hashed_chunks: return "No memories have been generated yet" chunks = extract_text_blocks_from_hashed_chunks(hashed_chunks) return '--ChunkBreak--'.join(chunks[:5]) ``` 2. **Create and Route the Node Type (in the Router)**: Open `Middleware/workflows/handlers/impl/memory_node_handler.py` and add the new type to the `handle` method's router logic. ```python # In MemoryNodeHandler.handle() # ... elif node_type == "FirstFiveMemories": return self.memory_service.get_first_five_memories(context.discussion_id) # ... ``` 3. **Register the Node Type (in the Registrar)**: Open `Middleware/workflows/managers/workflow_manager.py` and add the new `node_type` to the `node_handlers` dictionary in the constructor. ```python # In WorkflowManager.__init__() self.node_handlers = { # ... "FirstFiveMemories": memory_node_handler, # ... } ``` #### **Example B: Using a Workflow for Memory *Creation*** **Scenario**: Use a multi-step workflow to generate higher-quality vector-based memories. The first step will identify key topics, and the second step will write a structured JSON memory object for each topic. 1. **Configure the Memory System**: In your discussion ID config file, set `useVectorForQualityMemory` to `true` and specify the name of the workflow to run. ```json { "useVectorForQualityMemory": true, "vectorMemoryWorkflowName": "my-vector-memory-workflow", "fileMemoryWorkflowName": "my-file-memory-workflow" } ``` 2. **Understand the Injected Context**: The system automatically passes context into your workflow as `scoped_inputs`, which are then available in the workflow prompts as `{agent1Input}`, `{agent2Input}`, etc. The order is critical. * For a **file-based** memory workflow: * `{agent1Input}`: The raw text chunk to be summarized. * `{agent2Input}`: The most recent memory chunks. * `{agent3Input}`: The full history of all memory chunks. * `{agent4Input}`: The current rolling chat summary. * For a **vector-based** memory workflow: * `{agent1Input}`: The raw text chunk to be processed. 3. **Create the Memory Workflow**: Create a new workflow file (e.g., `Public/Configs/Workflows/my-vector-memory-workflow.json`). The final output of this workflow (from the last node where `returnToUser` is `true`) will become the new memory. *Workflow Definition (`.../my-vector-memory-workflow.json`):* ```json [ { "id": "agent1", "type": "Standard", "prompt": "You are a topic analyzer. Read the following conversation chunk and list the 3 most important topics discussed. Chunk: {agent1Input}", "returnToUser": false }, { "id": "agent2", "type": "Standard", "prompt": "You are a summarizer. The original conversation chunk is: {agent1Input}. The topics identified were: {agent1Output}. Generate a JSON array of memory objects, with one object for each topic. Each object must have 'title', 'summary', 'entities', and 'key_phrases' fields.", "returnToUser": true } ] ``` *Final Output from Workflow:* For a vector memory workflow, this **must be a JSON string** representing a single object or an array of objects. For example: ```json [ { "title": "Topic A Summary", "summary": "A detailed summary about the first topic discussed in the chunk.", "entities": ["Entity1", "Entity2"], "key_phrases": ["key phrase 1", "key phrase 2"] }, { "title": "Topic B Summary", "summary": "A detailed summary about the second topic discussed in the chunk.", "entities": ["Entity3"], "key_phrases": ["key phrase 3"] } ] ```