Hermes Cost Optimization

Hermes Cost Optimization

Audit a live Hermes Agent setup for token waste and provider cost, then apply safe, reversible optimizations. Covers what to inspect, what's safe to change, what's risky, and expected savings per change class.

When to Use

• User asks to audit their Hermes setup for cost efficiency
• User notices high token usage and wants to reduce it
• User wants to set up tiered provider routing (cheap model for simple tasks, expensive for complex)
• User is approaching context-window limits and needs to trim what loads each turn

Quick Start

# 1. Run the bootstrap script for a backup + size report
./fix-hermes-costs.sh /home/user/.hermes/hermes-agent rtk
# Options: rtk (install RTK compressor), sqz (install sqz), none

# 2. Read the auto-generated audit prompt
cat .hermes-cost-optimization/prompts/hermes-token-cost-audit.md

# 3. Ask Hermes to audit (paste the prompt or reference the file)
# 4. Review the report before applying any changes

What to Inspect (ordered by impact)

1. Skills Index — #1 Token Drain

The skills list is injected into the system prompt every turn. 100+ skills with descriptions can cost 10,000+ tokens.

# Check skill count and estimated token cost
python3 -c "
import json
with open('$HERMES_HOME/.skills_prompt_snapshot.json') as f:
    d = json.load(f)
skills = d.get('skills', [])
total_desc = sum(len(s.get('description','')) for s in skills)
print(f'Skills: {len(skills)}')
print(f'Description chars: {total_desc}')
print(f'Est tokens: ~{len(json.dumps(skills, indent=2)) // 4}')
"

Key file: ~/.hermes/hermes-agent/agent/prompt_builder.py → build_skills_system_prompt() (line 997) Snapshot: `/.hermes/.skills_prompt_snapshot.json**Skill testing landscape**: Seereferences/skill-testing-ecosystem.md` for what exists on GitHub for testing/evaluating skill quality (spoiler: nothing for functional correctness — only identity grading and security scanning).

2. AGENTS.md — Conditional Giant

53KB+ of development guide that loads ONLY when the cwd (or a parent) contains .git AND AGENTS.md. In normal use it doesn't load — but if the user cds into the hermes-agent repo for development work, it dumps 13,000+ tokens into context.

Key file: ~/.hermes/hermes-agent/agent/prompt_builder.py → build_context_files_prompt() (line ~1426) Load order: SOUL.md → AGENTS.md → CLAUDE.md → .cursorrules

3. Memory — Near-Capacity Stores

wc -c ~/.hermes/memories/MEMORY.md ~/.hermes/memories/USER.md

Config limits: memory.memory_char_limit (default 2200), memory.user_char_limit (default 1375).

4. SOUL.md — Agent Persona

cat ~/.hermes/SOUL.md

Empty/default uses the built-in DEFAULT_AGENT_IDENTITY. Custom personas can be large.

5. Compression Settings

hermes config show  # then look for compression section

Key knobs: enabled, threshold (0.50), target_ratio (0.20), protect_last_n (20).

6. Provider Routing

hermes config show  # then look for model and fallback_providers sections

Single-model setups waste money on simple tasks. Check if auxiliary tasks (vision, compression, session_search) are using auto (inherits expensive main model).

7. Tool Definitions

All tool schemas are injected. These are prompt-cached when prompt_caching.cache_ttl is set, so they don't cost tokens every turn — but they still consume context budget.

ls ~/.hermes/hermes-agent/tools/*.py | wc -l
hermes tools list

8. Cron Jobs

Each cron job prompt is loaded into context on every tick. Long prompts add up across many jobs.

hermes cron list
# Check individual job prompts:
python3 -c "
import json
with open('$HERMES_HOME/cron/jobs.json') as f:
    jobs = json.load(f)['jobs']
for j in jobs:
    print(f\"{j['name']}: {len(j.get('prompt',''))} chars\")
"

9. Context Files (cwd-dependent)

# Check if any of these exist in the user's common working directories
find /mnt/c/Users -maxdepth 3 -name "AGENTS.md" -o -name "CLAUDE.md" -o -name ".cursorrules" 2>/dev/null

Audit Methodology

Step 1: Measure the baseline

Run the bootstrap script or manually collect:

• Skills snapshot token estimate
• Memory file sizes
• Config.yaml settings (compression, memory, caching)
• Provider/model setup
• Cron job count and prompt sizes

Step 2: Identify waste sources

Categorize each finding:

Step 3: Propose staged changes

Always gate by risk, from safest to riskiest:

Stage 1 (Safe): Remove irrelevant skills, trim memory, fix config Stage 2 (Low risk): Wire shell-output compressor (RTK or sqz) Stage 3 (Medium): Add tiered provider routing Stage 4 (Ongoing): Memory/retrieval policy tuning Stage 5 (Aggressive): Heavy skill trimming, global compression hooks

Step 4: Show diffs before applying

Never modify files without showing the exact diff first. Every change must include a rollback instruction.

Safe Changes (always propose these)

1. Move irrelevant skills to a parking lot: Create ~/.hermes/skills-disabled/ and move unused skill directories there. This is better than optional-skills/ (which is inside the source repo and gets re-scanned or re-installed on update). Skills remain available — the agent can still load them via skill_view() with a full path if needed, but they won't appear in the system prompt skills index.

   mkdir -p ~/.hermes/skills-disabled/apple
   mv ~/.hermes/skills/apple/apple-notes ~/.hermes/skills-disabled/apple/
   # ... etc for each skill directory
   
   # Verify
   find ~/.hermes/skills -name "SKILL.md" | wc -l      # active
   find ~/.hermes/skills-disabled -name "SKILL.md" | wc -l  # parked
   

   Common candidates: apple/ (non-Mac users), gaming/ (non-gamers), niche creative tools (touchdesigner-mcp, songwriting, manim-video).

   Rollback: mv ~/.hermes/skills-disabled/<category>/* ~/.hermes/skills/<category>/

2. Consolidate memory entries: Merge related entries. Move operational details (API keys, URLs) from memory into skills where they belong. Add a pointer line like API keys/contacts in <skill-name> skill.

3. Remove plaintext credentials from cron job prompts: Scan cron/jobs.json for passwords, API keys, or tokens in the prompt field. Migrate to SSH key auth or .env references. See references/ssh-key-migration.md for the tested step-by-step (sshpass → ssh-copy-id → verify → edit prompt).

4. Remove unused personality definitions from config.yaml: They're in config, not in prompts — zero token savings but cleaner.

5. Set prompt_caching.cache_ttl higher: Default 5m is fine. Don't disable.

6. Verify compression is on: compression.enabled: true with threshold ~0.5.

Risky Changes (flag explicitly)

1. Aggressive skill trimming: Removing 50%+ of skills could drop rarely-used but important ones.
2. Global RTK hooks: Could truncate critical output (stack traces, structured data).
3. Switching main model to cheap: Degraded reasoning on complex tasks. Use tiering instead.
4. Disabling compression: Context overflow on long sessions.
5. Reducing tool_output limits: Could truncate legitimate large outputs.

Files to NEVER Modify

• ~/.hermes/.env — credential store
• ~/.hermes/config.yaml compression settings (already optimal when defaults)
• ~/.hermes/config.yaml tool_output limits (50KB/2000 lines is reasonable)
• Bundled skills (re-installed on update)
• Any file in ~/.hermes/hermes-agent/ without backup

Provider Routing Policy

Recommended tier pattern:

See references/third-party-api-resellers.md for pricing comparison of resellers, verification results, and red flags.

Implementation paths:

1. Config-based: model_routing in config.yaml (if supported)
2. Slash-command: /model gpt-4.1-nano before simple tasks
3. Delegation model: Set cheap model for subagents since they do mechanical work
4. Cron-specific: Set model override per cron job
5. SOUL.md guidance: Add a task-routing table to the agent persona so the agent knows when to switch models. See references/model-routing-guidance.md for the template and wiring instructions.

Shell-Output Compression

Two complementary approaches — use both:

Approach 1: Post-output filter hook (automatic, all terminal output)

A lightweight Python filter (compact-output.py) runs as a post_tool_call hook on the terminal toolset. It strips ANSI, collapses blank lines, deduplicates repeated lines, and truncates long lines — but never touches lines containing error/crash keywords.

Wire it via hermes config set:

hermes config set hooks.post_tool_call \
  '[{"matcher": "terminal", "command": "/home/lenovo/.hermes/scripts/compact-output.py", "timeout": 5}]'
hermes config set hooks_auto_accept true

The filter script lives at scripts/compact-output.py in this skill. Copy it to ~/.hermes/scripts/ before wiring.

Always test the filter on representative output before deploying: ANSI-colored output, error traces, repeated lines, and long lines. Verify error keywords (error, exception, traceback, fail, crash, fatal, OOM) pass through untouched.

Approach 2: RTK command proxy (behavioral, per-command)

RTK is a command proxy — you replace git diff with rtk git diff. It is NOT a pipe filter and cannot sit in a post-output hook. Instead, add guidance to ~/.hermes/SOUL.md telling the agent to prefer RTK-wrapped commands:

rtk ls        (69% smaller than bare ls)
rtk find      (75% smaller than bare find)
rtk git       (for git diff, git status, git log)
rtk grep      (compact, groups by file)
rtk docker    (compact Docker output)
rtk err       (run command, show only errors/warnings)
rtk test      (test runners — shows only failures)

For commands RTK doesn't support, it exits 127 harmlessly. If RTK output looks incomplete, re-run with the bare command.

Never use RTK on: structured JSON/YAML output, git diffs you need to read line-by-line, or test output where pass/fail counts matter.

Rollback

The bootstrap script creates a backup + rollback script:

# Full rollback to pre-audit state
bash .hermes-cost-optimization/rollback-*.sh

# Per-file rollback
cp -a .hermes-cost-optimization/backups/*/files/path/to/file ~/.hermes/path/to/file

Expected Savings Reference

Pitfalls

• The bootstrap script runs against source, not user config: fix-hermes-costs.sh creates .hermes-cost-optimization/ inside whatever path you give it. If you pass ~/.hermes/hermes-agent (the source repo), the size report will measure the repo's node_modules and venv — not the user's actual Hermes setup. The audit must inspect ~/.hermes/ (config.yaml, .env, skills/, memories/, SOUL.md) separately. The source repo size report is still useful for understanding what skills/tools are bundled, but it's not the token-cost picture.
• AGENTS.md is silent: It loads based on cwd, not configuration. A user who cds into the hermes-agent repo for dev work suddenly gets 53KB of context dumped in. Check cwd before starting a session.
• CLAUDE.md name causes model confusion: Users seeing "CLAUDE.md" in output often think Claude models are being used. CLAUDE.md is a project context file (like AGENTS.md), not model selection. When auditing actual model usage, query state.db directly — see references/session-db-model-audit.md for SQL queries.
• Skills re-install on update: Moving built-in skills to optional-skills/ may be undone by hermes update. The skills-disabled/ parking lot (outside the skills directory tree) survives updates.
• RTK is a command proxy, not a pipe filter: You cannot pipe output through RTK. It replaces the command itself (rtk ls instead of ls). For post-output filtering, use the compact-output.py script as a post_tool_call hook.
• Hook config syntax: hermes config set hooks.post_tool_call stores values as JSON strings in config.yaml. Use the CLI — don't hand-edit YAML for hooks. The config file is protected from direct writes anyway.
• hooks_auto_accept is required for non-interactive runs: Gateway, cron, and non-TTY sessions cannot prompt for hook consent. Without hooks_auto_accept: true, newly-added hooks silently never fire. Always set it when wiring a new hook, then verify the hook runs by checking session output for the [compacted: ...] header or similar marker.
• Prompt caching masks the real cost: Tool schemas are cached and don't appear in per-turn token counts, but they still consume context budget. The skills list and memory are the variable costs.
• Cron job prompts are each a full system prompt: Each LLM-based cron job run assembles its own system prompt including the full skills index. Multiply your per-turn estimate by cron frequency for true daily cost.
• Plaintext credentials in cron prompts: Scan cron/jobs.json for passwords, API keys, or tokens. Migrate to SSH keys or .env references. See references/ssh-key-migration.md for the step-by-step pattern. Test key auth BEFORE removing the password from the prompt — ssh-copy-id first, verify with ssh -o BatchMode=yes, then edit.