# Memory Complete

> Complete memory system with semantic vectors, session compaction, and automatic knowledge extraction. Use when building AI agents that need persistent memory across sessions with intelligent compression and mandatory extraction of lessons/decisions before token reset. Includes LanceDB (local vector storage), Hugging Face embeddings (384-dim, offline), session compaction (160K→40K tokens), and heuristic extraction of valuable knowledge. Zero cloud dependencies. Clone from GitHub and it's ready to use.

- Skill: `aspiranteintus/memory-complete` (Agent Skill, multi-file: 16 files)
- Install (CLI): `npx skillmds@latest add aspiranteintus/memory-complete`
- Raw SKILL.md: https://api.skillmd.com/api/skills/aspiranteintus/memory-complete/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: aspiranteintus (https://skillmd.com/u/aspiranteintus)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/aspiranteintus/memory-complete

---


# Complete Memory System Skill

A production-ready memory system for persistent AI agent context: semantic vectors + compaction + extraction rules.

## Overview

This skill provides a **complete, production-ready memory system** that you can:

1. **Clone from GitHub** → Get entire configuration ready
2. **Run setup.py** → Initialize in 30 seconds
3. **Start using** → No additional configuration needed (already optimized)
4. **Customize later** → Adapt rules/agents to your specific needs

Everything is pre-configured based on real production experience.

## ⚡ Quick Start (3 Steps)

### Step 1: Clone or Copy

```bash
# Option A: Git clone
git clone https://github.com/YOUR_USERNAME/memory-complete.git
cd memory-complete

# Option B: Copy entire folder
cp -r /path/to/memory-complete ./my-project/
cd my-project
```

### Step 2: Install & Setup

```bash
# Install dependencies
pip install -r requirements.txt

# Initialize system (one-time)
python3 code/setup.py

# Expected output:
# ✓ Memory system initialized
# ✓ Vector database ready
# ✓ Configuration loaded
# ✓ Ready to use
```

### Step 3: Start Using

```python
from openclaw_bridge_v2 import OpenClawBridge

# Create bridge (uses config.json automatically)
bridge = OpenClawBridge()

# Create your first agent (any name)
bridge.create_agent("my_context")

# Save a conversation
bridge.save_conversation(
    agent="my_context",
    role="user",
    content="I need to remember this important insight"
)

# Recall context when needed
context = bridge.recall_context(
    agents=["my_context"],
    query="What important insights do I have?"
)

print(context)
```

**Done!** Your memory system is running. ✓

## 📦 What's Inside

### Pre-Configured Settings

Everything is optimized from day 1:

```json
{
  "session": {
    "max_tokens": 160000,        ← Session capacity
    "compact_trigger": 160000,   ← When to compress
    "compact_reserve": 40000     ← Keep after compression
  },
  "extraction": {
    "before_compact": true,      ← MANDATORY before compressing
    "auto_extract": true,        ← Automatic knowledge capture
    "save_to_vectors": true      ← Persist extracted knowledge
  }
}
```

### Extraction Rules (Built-In)

Knowledge is automatically extracted using proven patterns:

**Lessons learned:**
```
"I learned that X is faster"
"Discovered that Y works better"
"Key insight: Z approach is more efficient"
```

**Decisions made:**
```
"Decided to use LanceDB instead"
"Chosen approach: vector-first architecture"
"We agreed to implement X pattern"
```

**Projects & goals:**
```
"Building a memory system"
"Working on improving performance"
"Project: system optimization"
```

### Compaction Strategy

When session grows too large (160K tokens):

1. **Extract** all lessons/decisions/projects
2. **Save** to dedicated vector agents
3. **Compact** session (discard conversations, keep 40K reserve)
4. **Continue** with fresh token budget

**Result:** No knowledge loss, just cleaned-up conversation history.

## 🎯 Core Components

### Memory Manager
```python
from memory_manager_v2 import MemoryManager

manager = MemoryManager()  # Uses config.json
manager.create_agent("finance")
manager.add_memory_to_agent(
    agent="finance",
    content="Bitcoin reached new ATH"
)

results = manager.search_agent_memory(
    agent="finance",
    query="Bitcoin price history"
)
```

### OpenClaw Bridge
```python
from openclaw_bridge_v2 import OpenClawBridge

bridge = OpenClawBridge()

# Save conversation turns
bridge.save_conversation(agent="context", role="user", content="...")
bridge.save_conversation(agent="context", role="assistant", content="...")

# Recall relevant context
context = bridge.recall_context(
    agents=["context", "decisions"],
    query="What did we decide about X?"
)
```

### Session Compactor
```python
from session_compactor import SessionCompactor

compactor = SessionCompactor()

# Check if compaction needed
if compactor.should_compact(current_tokens=165000):
    # Automatically extracts lessons/decisions before compacting
    result = compactor.compact("my_agent", token_count=165000)
    print(f"Extracted {result['lessons_extracted']} lessons")
```

## ⚙️ Configuration Guide

### config.json (Main Settings)

**What it controls:**
- Session size (how much memory per session)
- Compaction triggers (when to compress)
- Extraction settings (what to extract)
- Search defaults (query limits)
- Persistence (backups, logging)

**Pre-configured values** are production-ready. Only change if you have specific needs:

```json
{
  "session": {
    "max_tokens": 160000,          // Increase for longer sessions
    "compact_trigger": 160000,     // Can be lower to compact earlier
    "compact_reserve": 40000       // Token budget after compaction
  },
  "extraction": {
    "before_compact": true,        // CRITICAL - don't disable
    "auto_extract": true,          // Automatic extraction
    "save_to_vectors": true        // Persist extracted knowledge
  },
  "search": {
    "default_limit": 5,            // Results per search
    "confidence_threshold": 0.7    // Min similarity score (0-1)
  }
}
```

### rules/extraction-rules.json (What to Extract)

Customizable patterns for extracting knowledge:

```json
{
  "lessons": {
    "keywords": ["learned", "discovered", "insight", "found"],
    "patterns": [
      "I learned that {content}",
      "Key takeaway: {content}",
      "Insight: {content}"
    ],
    "confidence": 0.9,
    "enabled": true
  },
  "decisions": {
    "keywords": ["decided", "chose", "agreed", "committed"],
    "patterns": [
      "Decided to {action}",
      "We chose {option}",
      "Agreement: {decision}"
    ],
    "confidence": 0.85,
    "enabled": true
  },
  "projects": {
    "keywords": ["building", "working on", "project", "developing"],
    "patterns": [
      "Building {name}",
      "Project: {description}",
      "Working on {item}"
    ],
    "confidence": 0.8,
    "enabled": true
  }
}
```

**How to customize:**
1. Add new keywords
2. Add new patterns (use `{content}` or `{action}` placeholders)
3. Adjust confidence levels (0.0-1.0)
4. Enable/disable categories

### rules/compaction-rules.json (Compression Strategy)

Control how compaction happens:

```json
{
  "strategy": "aggressive",      // "conservative", "balanced", "aggressive"
  "token_limits": {
    "max_session": 160000,       // Maximum before mandatory compaction
    "preserve_after": 40000,     // Token budget after compaction
    "warning_threshold": 140000  // Warn when approaching limit
  },
  "extraction": {
    "mandatory": true,           // CRITICAL - always extract before compacting
    "save_extracted_to_agents": true,
    "update_memory_md": true
  },
  "deletion": {
    "policy": "oldest_first",    // How to select what to delete
    "keep_recent_count": 50,     // Keep most recent N memories
    "preserve_high_confidence": true
  },
  "scheduling": {
    "auto_compact": true,        // Automatic trigger at max_session
    "compact_on_startup": false, // Check on system start
    "notify_before_compact": true
  }
}
```

### rules/agent-templates.json (Agent Definitions)

Define agent templates users can create from:

```json
{
  "conversation": {
    "description": "Store conversations and chat history",
    "type": "general",
    "extract_lessons": true,
    "extract_decisions": true,
    "retention_days": 30,
    "example": "bridge.create_agent('conversation')"
  },
  "decisions": {
    "description": "Critical decisions made",
    "type": "decisions",
    "extract_lessons": false,
    "extract_decisions": true,
    "retention_days": 90,
    "example": "bridge.create_agent('decisions')"
  },
  "lessons": {
    "description": "Lessons learned and insights",
    "type": "lessons",
    "extract_lessons": true,
    "extract_decisions": false,
    "retention_days": 365,
    "example": "bridge.create_agent('lessons')"
  },
  "temporary": {
    "description": "Temporary context (cleared on compaction)",
    "type": "temporary",
    "extract_lessons": false,
    "extract_decisions": false,
    "retention_days": 1,
    "example": "bridge.create_agent('temp_research')"
  }
}
```

## 📊 How It Works

### Data Flow: Save & Recall

```
Your Code
    ↓
save_conversation(agent, role, content)
    ↓
Text → Auto-Embed (384-dim vector)
    ↓
Insert into LanceDB (agent's vector space)
    ↓
✓ Saved

---

recall_context(agents=["x", "y"], query="?")
    ↓
Query → Auto-Embed (same 384-dim space)
    ↓
Semantic search across agents
    ↓
Return top-5 similar memories + content
    ↓
Ready to use in LLM prompt
```

### Compaction Flow: Mandatory Extraction

```
Session Token Count > 160K
    ↓
MANDATORY EXTRACTION PHASE
    ├─ Extract lessons using patterns
    ├─ Extract decisions using patterns
    ├─ Extract projects using patterns
    └─ Save to vector agents
    ↓
COMPACTION PHASE
    ├─ Delete old conversation memories
    ├─ Keep 40K token reserve
    └─ Preserve extracted vectors
    ↓
SESSION RESET
    ├─ Fresh 160K token budget
    ├─ Can still recall lessons/decisions from previous sessions
    └─ Ready to continue
```

**Key:** Extraction happens BEFORE deletion. No knowledge loss.

## 🔧 Customization Examples

### Example 1: Change Token Limits

Want bigger/smaller memory?

```json
// config.json
{
  "session": {
    "max_tokens": 200000,        // ← Bigger (more conversation history)
    "compact_trigger": 200000,
    "compact_reserve": 60000     // ← More preserved after compact
  }
}
```

### Example 2: Add Custom Extraction Pattern

Want to extract something specific?

```json
// rules/extraction-rules.json
{
  "bug_reports": {
    "keywords": ["bug", "error", "crash", "issue"],
    "patterns": [
      "Bug: {description}",
      "Error encountered: {details}"
    ],
    "confidence": 0.8,
    "enabled": true
  }
}
```

Then update `memory_extractor.py`:

```python
def extract_bugs(self, text: str) -> List[str]:
    """Extract bug reports"""
    patterns = [
        r"Bug:\s*(.+?)(?:\.|,|$)",
        r"Error encountered:\s*(.+?)(?:\.|,|$)"
    ]
    # ... extraction logic
```

### Example 3: Create Domain-Specific Agents

```python
bridge = OpenClawBridge()

# Create agents for your domains
bridge.create_agent("trading_signals")
bridge.create_agent("market_analysis")
bridge.create_agent("risk_management")

# Save to specific agents
bridge.save_conversation(
    agent="trading_signals",
    role="system",
    content="BTC showing bullish divergence on 4h chart"
)

# Search across domain
context = bridge.recall_context(
    agents=["trading_signals", "market_analysis"],
    query="What are current trading signals?"
)
```

## 🚀 Production Deployment

### Checklist

- [x] Dependencies installed: `pip install -r requirements.txt`
- [x] Setup run: `python3 code/setup.py`
- [x] Tests pass: `python3 code/test_full_system.py`
- [x] config.json reviewed (customize if needed)
- [x] rules/ files reviewed (optional customization)
- [x] Agents created: `bridge.create_agent("name")`
- [x] Integration code written (save/recall in your agent)
- [x] Monitoring setup (logs enabled, backups scheduled)

### Logging & Monitoring

```python
import logging

logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
    handlers=[
        logging.FileHandler('memory.log'),
        logging.StreamHandler()
    ]
)

# System will log:
# - Memory additions
# - Searches performed
# - Extractions triggered
# - Compactions executed
# - Performance metrics
```

### Backup Strategy

Automatic backups configured in config.json:

```json
{
  "persistence": {
    "auto_save": true,
    "backup_interval_hours": 24,
    "backup_path": "./backups",
    "compression": "gzip"
  }
}
```

Backups include:
- Vector database snapshot
- Configuration files
- Extracted knowledge (lessons/decisions)
- Metadata

## 📈 Performance

| Operation | Latency | Notes |
|-----------|---------|-------|
| Add memory | 2-5ms | Auto-embed + insert |
| Search | 10-20ms | Vector similarity |
| Batch 100 | 50-100ms | Efficient |
| Compaction | 100-500ms | Full extraction + delete |
| Model load | ~30s | One-time download |

After model loads: Everything runs locally at 1-10ms.

## ❓ FAQ

**Q: Can I change extraction rules?**
A: Yes! Edit `rules/extraction-rules.json` and restart.

**Q: What if I need bigger session?**
A: Increase `max_tokens` in `config.json`.

**Q: Does it work with OpenClaw?**
A: Yes, `OpenClawBridge` designed specifically for OpenClaw.

**Q: Can I backup my memories?**
A: Automatic backups configured. Manual: `code/backup.py`.

**Q: How much disk space?**
A: ~1.5GB per 1 million vectors. Scales linearly.

**Q: Is it production-ready?**
A: Yes. Used in production. See deployment checklist above.

## 📚 File Structure

```
memory-complete/
├── SKILL.md                    ← You are here
├── README.md                   ← Quick overview
├── LICENSE                     ← MIT
├── requirements.txt            ← Dependencies
├── config.json                 ← MAIN CONFIG (pre-configured)
├── rules/
│   ├── extraction-rules.json   ← What to extract
│   ├── compaction-rules.json   ← How to compact
│   └── agent-templates.json    ← Agent definitions
├── code/
│   ├── embeddings.py           ← Hugging Face wrapper
│   ├── lance_memory.py         ← Vector DB
│   ├── memory_manager_v2.py    ← Orchestrator
│   ├── openclaw_bridge_v2.py   ← OpenClaw integration
│   ├── session_compactor.py    ← Compaction logic
│   ├── memory_extractor.py     ← Extraction engine
│   ├── rules_engine.py         ← Load & apply rules
│   ├── setup.py                ← Initialization
│   └── test_full_system.py     ← Tests
├── references/
│   ├── CONFIG_GUIDE.md         ← Detailed config options
│   ├── RULES_GUIDE.md          ← Custom rules tutorial
│   ├── IMPLEMENTATION_GUIDE.md ← Integration patterns
│   └── ARCHITECTURE.md         ← System design
└── memory/
    ├── vector_db/              ← LanceDB (created by setup)
    ├── backups/                ← Auto backups
    └── logs/                   ← System logs
```

## 🎓 Next Steps

1. **Clone & Setup** (3 minutes)
   ```bash
   git clone <repo>
   pip install -r requirements.txt
   python3 code/setup.py
   ```

2. **Integrate** (5-10 minutes)
   - Import `OpenClawBridge`
   - Create agents
   - Add save/recall calls

3. **Customize** (optional)
   - Edit `config.json` for your session sizes
   - Edit `rules/extraction-rules.json` for custom patterns
   - Add domain-specific agents

4. **Deploy** (production)
   - Enable logging
   - Configure backups
   - Monitor token usage
   - Iterate based on feedback

## 🤝 Support

- **SKILL.md** — This file (overview)
- **README.md** — Quick start
- **references/CONFIG_GUIDE.md** — All config options
- **references/RULES_GUIDE.md** — Customization tutorial
- **GitHub Issues** — Report problems

---

**Clone. Setup. Use. Customize if needed.**

That's the entire flow. Everything else is optional optimization.

Ready? → See **README.md** for quick start.