Agent Stack

Agent Stack

Deploy scoped AI agents as Docker containers that communicate via Redis queues, each with a defined remit and brain-page-writing discipline. Used when the user wants to build a multi-agent system on a VPS.

Trigger conditions

• User asks to build agents, agent workers, or an agent fleet
• User mentions "company brain" + agents
• User wants Redis-queue-based job routing between containers
• User provides a multi-phase build plan that includes agent containers

Architecture

┌──────────┐   ┌──────────┐   ┌──────────┐   ┌──────────┐
│  Slack   │   │ Ingest   │   │ Analysis │   │ Research │
│  Bot     │   │ Agent    │   │ Agent    │   │ Agent    │
│ (Bolt)   │   │          │   │          │   │          │
└────┬─────┘   └────┬─────┘   └────┬─────┘   └────┬─────┘
     │              │              │              │
     └──────────────┴──────┬───────┴──────────────┘
                           │  LPUSH / BRPOP
                    ┌──────▼──────┐
                    │    Redis    │
                    │  (queues)   │
                    └─────────────┘

Queues (Redis lists):

• hermes:queue:ingest — file ingestion jobs
• hermes:queue:analyze — analysis requests
• hermes:queue:research — research topics
• hermes:queue:brain — brain queries
• hermes:results — responses back to Slack bot

Pattern: Producers (Slack bot) LPUSH jobs. Consumers (agents) BRPOP jobs. All results go to hermes:results for delivery.

Agent container pattern

Every agent follows this structure. See templates/agent.py for the starter template.

# 1. Imports
import os, json, logging, time
from datetime import datetime, timezone
import redis

# 2. Config from env
REDIS_URL = os.environ.get("REDIS_URL", "redis://redis:6379/0")
r = redis.from_url(REDIS_URL, decode_responses=True)
QUEUE_IN = "hermes:queue:<name>"
QUEUE_RESULTS = "hermes:results"

# 3. Core processing
def process_job(job: dict):
    # Do the work
    # Write brain page to /data/brain/<type>/<slug>.md
    # Push result to QUEUE_RESULTS
    pass

# 4. Main loop
def main():
    while True:
        try:
            _, job_json = r.brpop(QUEUE_IN, timeout=5)
            if job_json:
                process_job(json.loads(job_json))
        except Exception as e:
            if "Timeout" not in str(e):  # brpop timeout is normal
                log.error(f"Error: {e}")
            time.sleep(5)

Brain page discipline (mandatory)

Every agent run MUST write a brain page. The page follows this structure:

---
title: '<type>: <summary>'
type: <page_type>
created: <ISO timestamp>
tags: [<domain tags>]
---

# <Title>

- **Date**: <when>
- **Source**: <provenance>
- **Key findings**: ...

<Artifact links or data>

Page directories by agent type:

• brain/ingest/ — ingestion reports
• brain/analyses/ — analysis results
• brain/research/ — research briefs
• brain/ops/ — ops alerts

Docker Compose service pattern

See templates/docker-compose-agent.yml for the service template. Key points:

• Single base image for all agents: build once with common deps (redis, pandas, duckdb, matplotlib)
• Bind-mount individual agent scripts: ./agents/<name>_agent.py:/app/<name>_agent.py
• Use command: not entrypoint: for per-agent script selection — entrypoint with a wrapper script adds a file that must exist in the image
• Volume mount /data so agents can read/write brain pages and processed data
• Depends on redis with condition: service_healthy

Image build

# In agents/ directory
docker build -t hermes-agent:latest .
docker compose up -d

The Dockerfile should include all shared dependencies so individual agents don't need their own images:

FROM python:3.12-slim
RUN pip install --no-cache-dir redis pandas pyarrow duckdb matplotlib
WORKDIR /app

Slack Bot pattern (Bolt + Socket Mode)

Socket Mode means the bot connects OUTBOUND to Slack — no public port needed. See templates/slack-bot.py.

Required env vars:

SLACK_BOT_TOKEN=xoxb-...
SLACK_APP_TOKEN=xapp-...
SLACK_SIGNING_SECRET=...

The bot LPUSHes jobs to agent queues and polls hermes:results (or listens via a separate results-delivery mechanism) to post back to channels.

Verification

After deploying agents, test each one:

# Test ingest
docker compose exec redis redis-cli LPUSH hermes:queue:ingest \
  '{"name":"test.csv","size":100,"user":"test","file":"/data/raw/inbox/test.csv","ts":"2026-01-01"}'

# Test analysis
docker compose exec redis redis-cli LPUSH hermes:queue:analyze \
  '{"query":"show data","channel":"test"}'

# Test research
docker compose exec redis redis-cli LPUSH hermes:queue:research \
  '{"topic":"example topic","channel":"test"}'

# Check logs
docker logs ingest-agent --tail 5

Pitfalls

• sed chain corruption: When using multiple sed -i commands on docker-compose.yml, each sed can match the output of the previous one if patterns overlap. Write the compose file from scratch or use targeted per-service sed blocks (/container_name: X/,/networks:/ s|...|).
• brpop timeout is normal: redis.brpop(key, timeout=5) raises a TimeoutError after 5 seconds of no data. Suppress this in logs — it's expected polling behavior, not an error. Filter with if "Timeout" not in str(e).
• Agent must not crash on missing data: If no datasets exist (e.g., analysis agent starts before any files are ingested), respond with a helpful message in the results queue rather than crashing.
• Slack bot won't start without real tokens: The bot container will crash-loop with placeholder tokens. This is expected. It starts working once .env has real Slack credentials.