Agent Memory Architecture

Agent Memory Architecture

Complete memory system for AI agents using only files. No APIs. No databases. No external dependencies. Just smart file structures and disciplined practices that give your agent perfect recall.

1. Memory Architecture Overview

workspace/
├── MEMORY.md              ← Long-term curated memory (the brain)
├── ACTIVE-CONTEXT.md      ← Hot working memory (what matters NOW)
├── AGENTS.md              ← Operating manual (how you work)
├── memory/
│   ├── 2026-01-15.md      ← Daily notes (raw event log)
│   ├── 2026-01-16.md
│   ├── heartbeat-state.json  ← Heartbeat tracking state
│   ├── topics/
│   │   ├── project-alpha.md  ← Topic-specific deep context
│   │   ├── client-acme.md
│   │   └── tech-stack.md
│   └── archive/
│       ├── 2025-Q4.md       ← Quarterly archive summaries
│       └── 2025-Q3.md

The 5 Memory Layers

Core Principle: Write It Down

Memory is limited. Files are permanent.

• "Mental notes" don't survive session restarts. Files do.
• If someone says "remember this" → write to a file
• If you learn a lesson → update the relevant file
• If you make a mistake → document it so future-you doesn't repeat it
• Text > Brain 📝

2. Layer 1: Hot Memory (ACTIVE-CONTEXT.md)

Your working scratchpad. What's happening RIGHT NOW.

Template

# ACTIVE-CONTEXT.md — What's Hot

Last updated: 2026-01-15 14:30 GMT

## 🔥 Current Priority
[ONE sentence: what is the most important thing right now?]

## In Progress
- [ ] Task A — status, next step
- [ ] Task B — status, blocker

## Waiting On
- Waiting for [person] to [action] — asked [date]
- Waiting for [system] to [complete] — ETA [time]

## Key Decisions Made Today
- Decided to [X] because [Y] — reversible: yes/no

## Context for Next Session
[What does future-you need to know to pick up where you left off?]

Rules

• Max 50 lines — if it's longer, you're hoarding. Move completed items to daily notes.
• Update before ending session — your gift to future-you
• One priority — if everything is priority, nothing is
• Delete completed items — this is NOT an archive

3. Layer 2: Long-Term Memory (MEMORY.md)

Your curated brain. Distilled knowledge, not raw logs.

Structure Template

# MEMORY.md — Long-Term Memory

## About [Human]
- Name, preferences, timezone, communication style
- What motivates them, what frustrates them
- Key relationships, roles, goals

## About Me [Agent]
- Name, personality, capabilities
- Operating preferences learned over time

## Active Projects
### Project Name
- Status, key decisions, blockers
- Links to relevant topic files

## Key People
- [Name] — role, relationship, communication notes

## Lessons Learned
- [Date] — [What happened] → [What I learned]

## Preferences & Patterns
- [Human prefers X over Y]
- [This approach works better than that one]

## Important Dates
- [Event] — [Date] — [Context]

Curation Rules

1. Only curated insights — not raw events (those go in daily notes)
2. Review weekly — scan daily notes, extract what's worth keeping
3. Prune quarterly — remove outdated info, archive completed projects
4. Max 500 lines — if it's longer, you need topic files
5. Security — never store secrets, API keys, passwords
6. Main session only — don't load MEMORY.md in group chats or shared contexts

What Goes In vs What Doesn't

4. Layer 3: Daily Notes (memory/YYYY-MM-DD.md)

Raw event log. Everything that happened today.

Template

# 2026-01-15 — Daily Notes

## Morning
- [08:15] Started session, reviewed ACTIVE-CONTEXT
- [08:30] Received task from [human]: [summary]
- [09:00] Completed [task] — result: [outcome]

## Afternoon
- [14:00] [Event/conversation summary]
- [15:30] Decision: [what was decided and why]

## Key Takeaways
- [Anything worth remembering beyond today]

## Tomorrow
- [ ] Follow up on [X]
- [ ] Check [Y]

Rules

• One file per day — memory/YYYY-MM-DD.md
• Append-only during the day — don't edit earlier entries
• Timestamps for important events
• Summarize, don't transcribe — capture essence, not every word
• Auto-create the memory/ directory if it doesn't exist
• Retention: Keep 30 days of daily notes. Archive older ones quarterly.

5. Layer 4: Topic Files (memory/topics/*.md)

Deep context on specific subjects that span many days.

When to Create a Topic File

• A project lasts more than 2 weeks
• A client/person comes up frequently
• A technical area needs accumulated knowledge
• You keep searching daily notes for the same information

Template

# [Topic Name]

Created: YYYY-MM-DD
Last updated: YYYY-MM-DD

## Summary
[2-3 sentences: what is this about?]

## Key Facts
- [Fact 1]
- [Fact 2]

## Decision Log
| Date | Decision | Reasoning | Outcome |
|------|----------|-----------|---------|
| | | | |

## Open Questions
- [Question 1]

## Related
- memory/topics/[related-topic].md
- [External link]

Rules

• Name descriptively — project-alpha.md not topic-1.md
• One topic per file — if it covers two things, split it
• Link from MEMORY.md — topic files are extensions of long-term memory
• Update when you learn — don't let them go stale

6. Layer 5: Archive (memory/archive/*.md)

Historical summaries for completed projects and past quarters.

Quarterly Archive Process

Every quarter (or when daily notes exceed 30 files):

1. Read all daily notes older than 30 days
2. Extract key events, decisions, outcomes, lessons
3. Write memory/archive/YYYY-QN.md (e.g., 2025-Q4.md)
4. Delete or move archived daily notes
5. Update MEMORY.md if any long-term insights emerged

Archive Template

# Q4 2025 Archive

## Summary
[3-5 sentences: what defined this quarter?]

## Major Events
- [Event 1] — [outcome]
- [Event 2] — [outcome]

## Projects
### [Project Name]
- Started: [date], Ended: [date]
- Outcome: [result]
- Lesson: [what we learned]

## Metrics
- [Key metric 1]: [value]
- [Key metric 2]: [value]

## Lessons Carried Forward
- [Lesson added to MEMORY.md: yes/no]

7. Session Startup Protocol

What to read at the start of every session, in order:

Main Session (Direct Chat with Human)

1. SOUL.md          — Who am I? (personality, values)
2. USER.md          — Who am I helping? (human context)
3. MEMORY.md        — Long-term memory (full brain)
4. ACTIVE-CONTEXT.md — Hot working memory (current state)
5. memory/today.md  — Today's daily notes (if exists)
6. memory/yesterday.md — Yesterday's notes (recent context)

Shared/Group Session (Discord, Slack, Group Chats)

1. SOUL.md          — Who am I?
2. USER.md          — Who am I helping?
3. ACTIVE-CONTEXT.md — Current priorities only
4. memory/today.md  — Today's notes
⚠️ DO NOT load MEMORY.md — contains private context

Sub-Agent / Isolated Session

1. Task-specific context only
2. Relevant topic file if applicable
3. ACTIVE-CONTEXT.md for current state
⚠️ Minimal context = focused output + lower token cost

8. Memory Write Protocol

When to Write (Triggers)

Write-Ahead Protocol

For critical information, write BEFORE acting:

1. Human gives important instruction
2. IMMEDIATELY write to daily notes or MEMORY.md
3. THEN execute the instruction
4. Update with results after

Why: If the session crashes mid-execution, the instruction is preserved.

Conflict Resolution

When information conflicts between layers:

• ACTIVE-CONTEXT.md wins for current state (most recent)
• MEMORY.md wins for long-term facts (curated)
• Daily notes are evidence — use to resolve disputes
• Topic files win for deep domain knowledge

9. Memory Search Strategy

When you need to find something:

Search Order (Fast to Slow)

1. ACTIVE-CONTEXT.md    — Is it current? (instant)
2. MEMORY.md            — Is it a known fact? (quick scan)
3. memory/today.md      — Did it happen today? (quick)
4. memory/yesterday.md  — Did it happen recently? (quick)
5. memory/topics/*.md   — Is it a deep topic? (targeted)
6. memory_search tool   — Semantic search across all files
7. memory/archive/*.md  — Is it historical? (slow)

Search Tips

• Use memory_search tool for fuzzy/semantic queries
• Use memory_get with line numbers for precise retrieval after search
• Check daily notes in reverse chronological order
• If you can't find it after 3 searches, ask the human

10. Memory Hygiene Schedule

Daily (During Session)

• Read ACTIVE-CONTEXT.md at session start
• Create/append to today's daily notes
• Update ACTIVE-CONTEXT.md before session ends
• Move completed ACTIVE-CONTEXT items to daily notes

Weekly (Pick One Heartbeat)

• Read last 7 daily notes
• Extract significant events/lessons to MEMORY.md
• Prune ACTIVE-CONTEXT.md (remove stale items)
• Check topic files for staleness
• Review MEMORY.md for outdated information

Monthly

• MEMORY.md line count check (target: <500 lines)
• Topic files audit — any need merging or archiving?
• Daily notes older than 30 days → archive
• Check if any topic files should be promoted to MEMORY.md sections

Quarterly

• Full archive process (see Layer 5)
• MEMORY.md deep review — still accurate?
• Topic files — archive completed projects
• Update AGENTS.md with any process improvements learned

11. Heartbeat Integration

Use heartbeats (periodic agent wake-ups) for memory maintenance:

heartbeat-state.json

{
  "last_memory_review": "2026-01-15",
  "last_archive": "2025-12-31",
  "last_active_context_prune": "2026-01-14",
  "daily_notes_count": 12,
  "memory_md_lines": 287,
  "next_scheduled": {
    "weekly_review": "2026-01-19",
    "monthly_audit": "2026-02-01",
    "quarterly_archive": "2026-03-31"
  }
}

Heartbeat Memory Tasks (Rotate)

Heartbeat 1: Check daily notes count, prune ACTIVE-CONTEXT
Heartbeat 2: Scan recent daily notes, update MEMORY.md
Heartbeat 3: Check topic files for staleness
Heartbeat 4: Token guard — how much are memory reads costing?

12. Context Window Management

Token Budget Rules

Smart Loading Strategy

Don't load everything every session. Use progressive disclosure:

Level 1: Always load (every session)
  → ACTIVE-CONTEXT.md (tiny, essential)
  → SOUL.md, USER.md (identity)

Level 2: Load in main sessions
  → MEMORY.md (the brain)
  → Today's daily notes

Level 3: Load on demand
  → Topic files (when topic comes up)
  → Yesterday's notes (if needed)
  → Archive (only on explicit search)

Context Overflow Protocol

When context gets too large mid-session:

1. Write ACTIVE-CONTEXT.md with full current state
2. Write HANDOFF.md with: what was done, in progress, next steps, key decisions, gotchas
3. Start fresh session
4. New session reads HANDOFF.md → picks up seamlessly
5. Delete HANDOFF.md after successful handoff

13. Security Rules

Never Store in Memory Files

• API keys, tokens, passwords, secrets
• Full credit card or bank account numbers
• Social security numbers or government IDs
• Private encryption keys
• Anything that would cause harm if the file were shared

Safe Storage Pattern

# ✅ Safe
- API keys: stored in 1Password vault "MyVault"
- Database password: see secrets manager, item "prod-db"

# ❌ Dangerous
- API key: sk-abc123def456...
- Password: MyS3cretP@ss!

Privacy in Shared Contexts

• MEMORY.md contains personal context — never load in group chats
• Topic files may contain sensitive business data — check before sharing
• Daily notes may reference private conversations — don't share
• When in doubt, ask before exposing any memory content

14. Memory Patterns & Anti-Patterns

✅ Good Patterns

❌ Anti-Patterns

15. Migration Guide

From No Memory System

Day 1: Create MEMORY.md with basic info about human + agent
Day 2: Start daily notes (memory/YYYY-MM-DD.md)
Day 3: Create ACTIVE-CONTEXT.md
Week 2: First weekly review — extract lessons to MEMORY.md
Month 2: Create first topic files for recurring subjects
Quarter 2: First archive cycle

From MEMORY.md-Only System

1. Create memory/ directory
2. Start daily notes — stop putting raw events in MEMORY.md
3. Create ACTIVE-CONTEXT.md — move "current" stuff out of MEMORY.md
4. Review MEMORY.md — what's curated vs what's raw? Move raw to daily notes.
5. Identify topics that deserve their own files — split them out

From External Tool (Database, API, Cloud)

1. Export key data to markdown files
2. Structure into the 5-layer architecture
3. Set up heartbeat maintenance schedule
4. Gradually reduce dependency on external tool
5. Benefits: zero cost, zero dependencies, works offline, no vendor lock-in

16. Natural Language Commands

• /memory-status — Show memory system health: file sizes, line counts, staleness, next maintenance
• /memory-review — Run weekly review: scan daily notes, extract to MEMORY.md, prune active context
• /memory-search [query] — Search across all memory layers for a topic
• /memory-archive — Run quarterly archive: summarize old daily notes, create archive file
• /remember [fact] — Immediately write a fact to MEMORY.md
• /active-context — Show current ACTIVE-CONTEXT.md contents
• /daily-summary — Generate summary of today's daily notes
• /topic-create [name] — Create a new topic file with template
• /memory-prune — Audit all memory files for staleness and bloat
• /handoff — Write HANDOFF.md for session transition
• /memory-migrate — Guided migration from current system to this architecture
• /memory-debug — Diagnose memory issues: missing files, conflicts, outdated info