per-client-memory-router
The problem this solves: an agency running one agent stack across many clients hits
two failures that look unrelated but share one root cause — no separation between "what
this agent knows in general" and "what this agent knows about this one client." Global
prompts, global memory, and a shared job queue.
- Context bleed. A preference, a brand voice note, or a fact from Client A's thread
surfaces in Client B's output, because the agent's memory has no boundary between them.
- Quota starvation. One client with a large or slow-running job (a big crawl, a long
content run) monopolizes the shared model/API quota, and every other client's job
queues behind it or times out.
Both are architecture gaps, not model failures. Fixing them takes two separate pieces
working together: a memory boundary and a concurrency boundary.
Global memory answers "how do we work." Per-client memory answers "what do we know
about this client." A job never reads the wrong one, and no client's queue can starve
another's.
Say this to your agent
"Split our memory into one global store (house style, SOPs, tool configs — shared
across every client) and one isolated store per client (their brand facts, preferences,
history, open items). Every job you run must declare which client it's for before it
reads memory, and it may only read that client's store plus the global one — never
another client's. Then put a concurrency limit in front of the job queue: cap how many
jobs can run at once per client and in total, so one client's large job can't starve
everyone else's queue. Flag any code path that reads memory without a declared client
scope before we ship it."
The two boundaries (both required, neither substitutes for the other)
| Boundary |
What it does |
What breaks without it |
| Memory partition |
Global store for shared knowledge (house style, SOPs, tool wiring); one isolated store per client for their facts/preferences/history |
Client A's brand voice, pricing, or private notes leak into Client B's output — a trust-ending mistake, not a cosmetic bug |
| Concurrency-limited router |
Caps in-flight jobs per client and in total before they hit the model/API layer |
One client's big job (a full crawl, a long batch) monopolizes shared quota; every other client's job queues, times out, or silently drops |
Partitioning memory without limiting concurrency still leaks nothing, but one client can
still starve the rest on shared compute. Limiting concurrency without partitioning memory
still runs jobs fairly, but any of them can read the wrong client's context. Ship both.
Reference architecture
Global memory store Per-client memory store (one per client)
- house style/voice - brand facts, tone, do/don't list
- SOPs, checklists - preferences and standing instructions
- tool/API configs - open items, thread history
- shared skill library - prior deliverables, approved copy
| |
└──────────────┬───────────────┘
v
Job declares client_id
v
Router reads global + client_id's store ONLY
v
Concurrency gate (per-client cap + global cap)
v
Model/API execution
- Every job carries a
client_id before it touches memory. No client_id means the
job cannot read or write any per-client store; it may only use the global one.
- A job for
client_id=A is structurally blocked from reading client_id=B's store —
not a prompt instruction to "keep them separate," an actual access boundary (separate
files, namespaces, or directories, not a shared blob with a filter).
- The concurrency gate sits between the router and the model call, not inside the
agent's own reasoning — a per-client cap (e.g. 2 concurrent jobs) and a global cap (e.g.
10 concurrent jobs across all clients) so no single client can consume the whole pool.
- Global memory is read-heavy, rarely written mid-run. Per-client memory is the one
that changes as the relationship progresses (new preferences, new history, new
deliverables) and is the one to check for drift/staleness first when something reads
wrong.
Build sequence
- Inventory what's currently global-only. Most single-client-turned-multi-client
setups start with everything in one memory blob. List every fact/preference/note
currently in there and mark each one global (applies to all clients) or client-specific.
- Stand up the per-client store shape first, even with one client in it: a folder,
namespace, or table keyed by
client_id. Migrate client-specific facts out of the
global blob into it.
- Require
client_id at the job's entry point, not somewhere deep in the prompt.
Reject or route-to-global-only any job that doesn't declare one.
- Add the concurrency gate last, once partitioning is verified clean. Test that
Client A's job genuinely cannot read Client B's store before you also start running
them in parallel; a race condition on top of a leaky boundary is harder to diagnose
than either problem alone.
- Set caps conservatively and watch real usage before tuning up. Start with a low
per-client cap and a global cap comfortably under your actual rate limit, then raise
both once you've seen a week of real multi-client load.
Watch-outs
| Signal |
Fix |
| A client's fact or preference ever shows up in another client's output |
The memory read had no client_id boundary, or the boundary is a filter on a shared store rather than real isolation. Move to separate stores/namespaces |
| Jobs for one client time out only when another client runs a big job |
No concurrency gate, or the cap is total-only with no per-client sub-cap |
| Global memory keeps accumulating client-specific facts |
The global/client split wasn't enforced at write time. Audit and re-migrate, not just fix it going forward |
A job runs with no declared client_id and nothing rejected it |
The entry point isn't actually gating on client_id. It's advisory, not structural |
Sourced from a live member build shared in the 2026-09-10 AMM cohort session: the
memory/queue architecture (concurrency-limited router, global vs. per-client memory
partitioning) a member is building directly with the program lead for a multi-client
agent stack.
1---2name: per-client-memory-router3description: Stops one agent working N clients from bleeding context between them, and stops N parallel client jobs from starving each other on the same model quota. Partition memory into a global store plus one isolated store per client, then gate execution through a concurrency-limited router so no single client's job queue can starve the rest. Use when one agentic setup serves multiple clients, when a client's context/preferences ever showed up in another client's output, or when parallel client jobs are timing out or burning quota unevenly.4metadata:5 source: https://skillmd.com/skills/search-atlas-group/per-client-memory-router6---78# per-client-memory-router910**The problem this solves:** an agency running one agent stack across many clients hits11two failures that look unrelated but share one root cause — no separation between "what12this agent knows in general" and "what this agent knows about this one client." Global13prompts, global memory, and a shared job queue.1415- **Context bleed.** A preference, a brand voice note, or a fact from Client A's thread16 surfaces in Client B's output, because the agent's memory has no boundary between them.17- **Quota starvation.** One client with a large or slow-running job (a big crawl, a long18 content run) monopolizes the shared model/API quota, and every other client's job19 queues behind it or times out.2021Both are architecture gaps, not model failures. Fixing them takes two separate pieces22working together: a memory boundary and a concurrency boundary.2324> **Global memory answers "how do we work." Per-client memory answers "what do we know25> about this client." A job never reads the wrong one, and no client's queue can starve26> another's.**2728---2930## Say this to your agent3132> "Split our memory into one global store (house style, SOPs, tool configs — shared33> across every client) and one isolated store per client (their brand facts, preferences,34> history, open items). Every job you run must declare which client it's for before it35> reads memory, and it may only read that client's store plus the global one — never36> another client's. Then put a concurrency limit in front of the job queue: cap how many37> jobs can run at once per client and in total, so one client's large job can't starve38> everyone else's queue. Flag any code path that reads memory without a declared client39> scope before we ship it."4041---4243## The two boundaries (both required, neither substitutes for the other)4445| Boundary | What it does | What breaks without it |46|---|---|---|47| **Memory partition** | Global store for shared knowledge (house style, SOPs, tool wiring); one isolated store per client for their facts/preferences/history | Client A's brand voice, pricing, or private notes leak into Client B's output — a trust-ending mistake, not a cosmetic bug |48| **Concurrency-limited router** | Caps in-flight jobs per client and in total before they hit the model/API layer | One client's big job (a full crawl, a long batch) monopolizes shared quota; every other client's job queues, times out, or silently drops |4950Partitioning memory without limiting concurrency still leaks nothing, but one client can51still starve the rest on shared compute. Limiting concurrency without partitioning memory52still runs jobs fairly, but any of them can read the wrong client's context. Ship both.5354---5556## Reference architecture5758```text59Global memory store Per-client memory store (one per client)60 - house style/voice - brand facts, tone, do/don't list61 - SOPs, checklists - preferences and standing instructions62 - tool/API configs - open items, thread history63 - shared skill library - prior deliverables, approved copy64 | |65 └──────────────┬───────────────┘66 v67 Job declares client_id68 v69 Router reads global + client_id's store ONLY70 v71 Concurrency gate (per-client cap + global cap)72 v73 Model/API execution74```7576- **Every job carries a `client_id` before it touches memory.** No `client_id` means the77 job cannot read or write any per-client store; it may only use the global one.78- **A job for `client_id=A` is structurally blocked from reading `client_id=B`'s store** —79 not a prompt instruction to "keep them separate," an actual access boundary (separate80 files, namespaces, or directories, not a shared blob with a filter).81- **The concurrency gate sits between the router and the model call**, not inside the82 agent's own reasoning — a per-client cap (e.g. 2 concurrent jobs) and a global cap (e.g.83 10 concurrent jobs across all clients) so no single client can consume the whole pool.84- **Global memory is read-heavy, rarely written mid-run.** Per-client memory is the one85 that changes as the relationship progresses (new preferences, new history, new86 deliverables) and is the one to check for drift/staleness first when something reads87 wrong.8889---9091## Build sequence92931. **Inventory what's currently global-only.** Most single-client-turned-multi-client94 setups start with everything in one memory blob. List every fact/preference/note95 currently in there and mark each one global (applies to all clients) or client-specific.962. **Stand up the per-client store shape first**, even with one client in it: a folder,97 namespace, or table keyed by `client_id`. Migrate client-specific facts out of the98 global blob into it.993. **Require `client_id` at the job's entry point**, not somewhere deep in the prompt.100 Reject or route-to-global-only any job that doesn't declare one.1014. **Add the concurrency gate last, once partitioning is verified clean.** Test that102 Client A's job genuinely cannot read Client B's store before you also start running103 them in parallel; a race condition on top of a leaky boundary is harder to diagnose104 than either problem alone.1055. **Set caps conservatively and watch real usage** before tuning up. Start with a low106 per-client cap and a global cap comfortably under your actual rate limit, then raise107 both once you've seen a week of real multi-client load.108109---110111## Watch-outs112113| Signal | Fix |114|---|---|115| A client's fact or preference ever shows up in another client's output | The memory read had no `client_id` boundary, or the boundary is a filter on a shared store rather than real isolation. Move to separate stores/namespaces |116| Jobs for one client time out only when another client runs a big job | No concurrency gate, or the cap is total-only with no per-client sub-cap |117| Global memory keeps accumulating client-specific facts | The global/client split wasn't enforced at write time. Audit and re-migrate, not just fix it going forward |118| A job runs with no declared `client_id` and nothing rejected it | The entry point isn't actually gating on `client_id`. It's advisory, not structural |119120---121122*Sourced from a live member build shared in the 2026-09-10 AMM cohort session: the123memory/queue architecture (concurrency-limited router, global vs. per-client memory124partitioning) a member is building directly with the program lead for a multi-client125agent stack.*