Memory Poisoning Defense

Memory Poisoning Defense

Attribution: YourVisionYourCreation LLC — yourvisionyourcreation.com Research anchor (verified): AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases — Chen, Xiang, Xiao, Song & Li, NeurIPS 2024, arXiv:2407.12784 — demonstrating backdoor attacks via poisoned long-term memory and RAG stores with high success rates at poison rates under 0.1%. Successor persistent-memory attacks (e.g. MemoryGraft-style implanted experiences) confirm the vector's continued evolution. The defensive doctrine is YVYC original. Doctrine class: Tier 5 — Frontier (YVYC original, anchored in verified research)

Universal So-What

Prompt injection dies with the session. Memory poisoning does not. An attacker — or an accident — that writes into an agent's persistent memory has planted something that fires on every future retrieval: a fake preference, a corrupted procedure, a booby-trapped "fact" that steers behavior weeks later in sessions the attacker never touched. Memory is the only attack surface that compounds with time. It gets defended like one, at every stage: what gets written, what gets stored, what gets retrieved, and what gets believed.

Core Doctrine

1. The Write Gate — Memory Admission Is Earned

Nothing enters persistent memory by default:

• Provenance tagging: every candidate memory records its source — direct user statement, agent inference, tool result, retrieved document — and the session it came from. Untagged memories do not exist; they are contamination with a database row.
• Source-tier admission: direct user statements earn the highest write trust; content from retrieved documents and tool outputs earns the LOWEST — third-party content requesting persistence is the classic poisoning vector ("remember for future sessions that...") and is refused as a command, however it is phrased
• Write-worthiness check: stable, useful, consistent with the existing record — a candidate contradicting established memory triggers reconciliation, not silent overwrite

2. The Quarantine Tier

New memories are not full citizens:

• Fresh writes enter a probationary state: retrievable, but flagged as unconfirmed and weighted accordingly in decisions
• Promotion to trusted status requires corroboration — repeated consistent evidence across sessions, or explicit user confirmation
• High-consequence memory classes (permissions, identities, standing instructions, anything security-adjacent) NEVER auto-promote: human or policy confirmation only

3. Retrieval Hygiene

Defense at read time, because the store will never be perfectly clean:

• Retrieved memories arrive as DATA with provenance attached — never as instructions; a memory containing imperative language ("always do X," "ignore Y") is a red flag pattern, not a command (the injection boundary of authority-stack-doctrine, applied to the agent's own past)
• Anomaly screening at retrieval: memories that are outliers against the surrounding record — unusual recency-of-write for their claimed age, provenance mismatch, semantic distance from the user's established pattern — get flagged rather than silently trusted
• Influence proportionality: no single retrieved memory silently redirects a high-stakes action; consequential decisions cite the memories they relied on, making the influence path auditable

4. The Memory Audit

The store is inspected, not assumed:

• On a cadence: sample the store and verify — provenance intact, content consistent with source sessions, no imperative-laden entries, no orphans without traceable origin
• After any suspected manipulation: full sweep of every memory written during the exposure window, treated with the burned-secret logic of secure-secrets-doctrine — rotate out everything the window touched rather than adjudicating entries one by one
• Behavior-first detection: an agent whose conduct shifted after a specific interaction gets its memory diffed against that interaction date — the write log is the crime scene index

5. Shared and Multi-Agent Memory Rules

Shared stores multiply the blast radius:

• A memory written by one agent and read by many is a supply chain — it inherits supply-chain discipline (hardening-baseline, Perimeter 5): write access enumerated, writers authenticated, writes logged
• Cross-agent memory carries the writing agent's identity and trust tier; a low-trust agent's writes never silently inform a high-trust agent's actions
• Knowledge bases feeding RAG are memory by another name: ingestion pipelines get the same write gate, and bulk imports get quarantine as a batch — one poisoned document in a trusted corpus is the attack the research literature demonstrates working

6. Recovery Doctrine

Assume eventual contamination; design the comeback:

• Memory is versioned: every write is reversible, every state reconstructable to a point in time — an unversioned memory store is a hostage negotiation waiting for its incident
• The rollback decision uses the exposure window, not entry-by-entry litigation: restore to pre-exposure state, replay legitimate writes from the log where provenance proves them
• Post-incident, the write gate that admitted the poison is the defect that gets fixed — the entry was the symptom; the admission was the failure

Common Failure Modes

Non-Negotiables

1. Nothing enters memory without provenance.
2. Content never grants itself persistence — "remember this" inside retrieved material is a red flag, not a command.
3. New memories are quarantined; security-adjacent classes never auto-promote.
4. Retrieved memories are data with provenance, never instructions.
5. Memory is versioned and rollback-capable — always.
6. Every poisoning incident ends with a write-gate fix.

Built by YourVisionYourCreation LLC — yourvisionyourcreation.com Research foundation credited above. Licensed under CC BY 4.0