Hermes State Sync Via Git

Hermes State Sync Across Machines (via private Git repo)

User wants the same Hermes Agent config/skills/memories/cron on multiple machines (e.g. macOS master + Windows mirror). The pattern: a private GitHub repo acts as canonical store; each machine runs a periodic sync that pulls, refreshes a staging dir from local Hermes state, and pushes if changed.

Architecture

┌─────────────────┐                ┌─────────────────┐
│  macOS (master) │  ←──push/pull──│ Windows (mirror)│
│  Hermes state   │                │  Hermes state   │
└────────┬────────┘                └────────┬────────┘
         │ pull/push (cron)                │ pull/push (cron)
         └──────────────┬───────────────────┘
                        │
                ┌───────▼────────┐
                │  GitHub private │
                │  hermes-sync    │
                │  (canonical)    │
                └────────────────┘

Critical pattern: staging dir, not symlink. Never symlink the working Hermes data dir into a Git repo. Hermes holds open file handles (SQLite WAL, .lock files) — git add -A will read inconsistent state and pushes will fail. Always copy into a separate staging dir, commit, push, then have the other machine's sync pull and copy back over.

What to sync

Default paths

• macOS Hermes dir: ~/Library/Application Support/hermes/
• Windows Hermes dir: %LOCALAPPDATA%\hermes (e.g. C:\Users\<user>\AppData\Local\hermes)
• Staging dir (any machine): anywhere outside the Hermes dir, e.g. ~/hermes-sync/

Reference scripts and templates

• templates/sync.sh — bash sync script (the one that actually runs)
• templates/sync.py — pure-Python equivalent, more portable
• templates/.gitignore — block-everything-then-allowlist pattern
• references/setup-walkthrough.md — step-by-step PAT + repo creation + first push
• references/windows-pitfalls.md — every Windows-specific gotcha we hit (MSYS, CRLF, python3 stub, etc.)
• references/mac-setup.md — what to do on the other machine after Windows is up

Quick setup flow

1. Create a GitHub PAT with repo scope (fine-grained or classic both work; classic is simpler for this use case).
2. Create the private repo via curl -X POST .../user/repos with private: true and auto_init: true.
3. Set git identity on the master machine:

   git config --global user.name "okya1"
   git config --global user.email "<user@email>"
   

4. Build the staging dir by copying the sync items from Hermes dir (see references/setup-walkthrough.md).
5. Write .gitignore that blocks everything then allowlists the items — see templates/.gitignore. Critical: explicitly list !sync.sh and !README.md in the allowlist, otherwise they're blocked by the catch-all * rule and the sync script never gets committed.
6. First push: init repo, add the (unrelated) remote, git pull --allow-unrelated-histories, then push.
7. Wire up cron (every 5 min) to call bash sync.sh.

sync.sh essentials

The script MUST auto-detect its OS rather than hardcode HERMES_DIR. See pitfall #8 below for the cross-contamination failure mode this prevents.

#!/usr/bin/env bash
set -e
SYNC_DIR="$(cd "$(dirname "$0")" && pwd)"

# === Auto-detect platform at runtime ===
detect_platform() {
  case "$OSTYPE" in
    msys*|win32*|cygwin*)
      # Windows. $USER is often empty in Git Bash — fall back to USERPROFILE
      # and convert via cygpath (preferred) or sed.
      if [ -n "$USERPROFILE" ]; then
        if command -v cygpath >/dev/null 2>&1; then
          WIN_USER_PATH="$(cygpath -u "$USERPROFILE")"
        else
          WIN_USER_PATH="$(echo "$USERPROFILE" | sed -E 's|^([A-Za-z]):|/\L\1|; s|\\|/|g')"
        fi
      elif [ -n "$USERNAME" ]; then
        WIN_USER_PATH="/c/Users/$USERNAME"
      else
        WIN_USER_PATH="$(echo "$HOME" | sed -E 's|.*/Users/([^/]+).*|\1|' | xargs -I{} echo "/c/Users/{}")"
      fi
      HERMES_DIR="$WIN_USER_PATH/AppData/Local/hermes"
      if [ -x "$WIN_USER_PATH/AppData/Local/hermes/hermes-agent/venv/Scripts/python" ]; then
        PYTHON_BIN="$WIN_USER_PATH/AppData/Local/hermes/hermes-agent/venv/Scripts/python"
      else
        PYTHON_BIN="python"
      fi
      PLATFORM_LABEL="windows"
      ;;
    darwin*)
      HERMES_DIR="$HOME/Library/Application Support/hermes"
      PYTHON_BIN="/usr/bin/python3"
      PLATFORM_LABEL="macos"
      ;;
    linux*)
      HERMES_DIR="$HOME/.local/share/hermes"
      PYTHON_BIN="/usr/bin/python3"
      PLATFORM_LABEL="linux"
      ;;
    *)
      echo "ERROR: unknown OS type: $OSTYPE" >&2
      echo "  Set HERMES_DIR_OVERRIDE and PYTHON_BIN_OVERRIDE env vars, then re-run." >&2
      exit 1
      ;;
  esac
  # Allow override via env (e.g. for non-standard installs)
  HERMES_DIR="${HERMES_DIR_OVERRIDE:-$HERMES_DIR}"
  PYTHON_BIN="${PYTHON_BIN_OVERRIDE:-$PYTHON_BIN}"
}
detect_platform
export HERMES_DIR PYTHON_BIN
echo "[platform: $PLATFORM_LABEL] HERMES_DIR=$HERMES_DIR PYTHON_BIN=$PYTHON_BIN"

REMOTE="origin"
BRANCH="main"

# 1) Pull latest — DON'T let a pull failure kill the run
cd "$SYNC_DIR"
if ! git pull --rebase --autostash "$REMOTE" "$BRANCH" 2>&1; then
  echo "  pull failed (will continue with local state)"
fi

# 2) Refresh staging from local Hermes state
echo "[2/3] Refreshing staging from $HERMES_DIR..."
"$PYTHON_BIN" - <<'PYEOF'
import os, shutil
from pathlib import Path
HERMES = Path(os.environ["HERMES_DIR"])
SYNC   = Path(os.environ["SYNC_DIR"])
ITEMS = {
  "config": "config.yaml", "soul": "SOUL.md", "auth": "auth.json",
  "memories": "memories", "skills": "skills", "cron": "cron",
}
# Tolerate Windows file locks (Hermes desktop holds .lock/.db-wal/.db
# open). Single copytree with dirs_exist_ok + ignore callback merges
# into the existing staging without rmtree race. See pitfall #11.
def _ignore_readonly(src_dir, names):
    out = []
    for n in names:
        p = Path(src_dir) / n
        if p.is_file():
            try:
                with open(p, "rb"): pass
            except (PermissionError, OSError):
                out.append(n)
    return out
for name, rel in ITEMS.items():
    src = HERMES / rel
    dst = SYNC / name
    if not src.exists(): continue
    if src.is_dir():
        shutil.copytree(src, dst, dirs_exist_ok=True, ignore=_ignore_readonly)
    else:
        shutil.copy2(src, dst)
print("staging refreshed")
PYEOF

# 3) Stage and push if changed
git add -A
if git diff --staged --quiet; then
  echo "  no changes to commit"
else
  git commit -m "auto-sync $(date '+%Y-%m-%d %H:%M:%S') from $(hostname) [$PLATFORM_LABEL]"
  git push "$REMOTE" "$BRANCH"
  echo "  pushed"
fi
echo "Done."

Critical pitfalls (the ones that actually bit)

1. python3 is broken on Windows — resolves to the Microsoft Store stub which silently fails on heredocs with exit 49. Use python, which resolves to the real venv. (Both are in PATH on this machine.)

2. set -e + git pull — a transient network blip will kill the whole sync run. Wrap in if ! ... ; then echo ... ; fi.

3. .gitignore catch-all * blocks your own script — must explicitly allowlist !sync.sh and !README.md, otherwise git add -A silently skips them and the fix never reaches the remote.

4. Don't -f best on Twitter videos (related but different skill) — Twitter ships video and audio as separate HLS streams; let yt-dlp pick the best of each and merge. (This pitfall belongs in the video skill, not here, but it bites in the same Windows env.)

5. MSYS bash on Windows mangles backslashes in absolute paths passed as args — bash -n 'C:\Users\foo\sync.sh' becomes C:Usersfoosync.sh. Workaround: cd to the dir first and pass a relative path, or copy to /tmp and check there. For subprocess in Python: convert the path yourself with str(p).replace("\\","/") and prepend /c/... form, or pass cwd=<dir> + "script.sh" (relative).

6. subprocess.run(capture_output=True) on Windows drops the first line of bash output — use Popen + communicate with bytes mode and explicit UTF-8 decode (Windows bash often emits Korean codepage, not UTF-8). This bit us hard in the verification script. When WINDIR is set in MSYS, output encoding switches to UTF-16LE — always pass text=False and decode with errors="replace", not errors="strict".

7. First push to a repo with auto-init README: git push is rejected with "fetch first". Fix: git pull --allow-unrelated-histories --no-edit, then push.

8. sync.sh must auto-detect OS — never hardcode HERMES_DIR. If Mac edits HERMES_DIR="$HOME/Library/Application Support/hermes" and pushes, Windows's next pull silently overwrites its own Windows path with the Mac one. bash sync.sh then keeps "succeeding" while refreshing the wrong directory — silent corruption. The fix is to detect $OSTYPE at runtime and pick the right path. See references/cross-platform-safety.md for the full failure mode and the detect_platform() function.

9. $USER is empty in Git Bash on Windows. Use $USERPROFILE (always set by Windows), then convert with cygpath -u (preferred) or sed to MSYS form /c/Users/.... Falling back to $USERNAME only works if it matches the home dir name (e.g. WSL has them divergent).

10. write_file saves with CRLF on Windows. Bash then chokes on $'\r' syntax errors. After any script edit on Windows, strip CR before committing: python -c "p=Path('sync.sh'); p.write_bytes(p.read_bytes().replace(b'\r',b''))". Verify with xxd sync.sh | head -2 — should be 23 21 2f 62 69 6e 2f 62 61 73 68 0a (LF, not CRLF).

11. shutil.copytree(src, dst, ignore_errors=True) does NOT exist as a parameter in Python's stdlib. Despite the name suggesting a flag, copytree only accepts ignore (callable) and onerror (legacy) for error handling. ignore_errors is a parameter on shutil.rmtree only. And the seemingly-safe pattern rmtree(ignore_errors=True) → copytree(dirs_exist_ok=True) has a race: if rmtree leaves locked files behind (e.g. SQLite WAL, .lock files held by the running Hermes process), copytree raises FileExistsError on the next iteration and the staging dir ends up empty/broken. The correct pattern is a single shutil.copytree(src, dst, dirs_exist_ok=True, ignore=callback) that tests each file for read access and skips locked ones, leaving the rest intact. The ignore callback is (src_dir, names) -> list_of_names_to_skip.

12. The .gitignore allowlist must use ** to recurse into whitelisted directories. A line like !skills/ only allows the directory entry to be tracked, not the files inside — git treats the dir as an empty placeholder. You need both: !skills/ (so git sees the dir) AND !skills/** (so the files inside are tracked). Same for !memories/** and !cron/**. Without the **, only the top-level files (config, soul, auth) get tracked and the memory/skills/cron directories appear empty in git ls-files. Verify with git ls-tree -r --name-only origin/main | grep -E '^(memories|skills|cron)/' | wc -l — should be hundreds, not 0.

13. Cross-machine pushes are dangerous: never auto-push from a sandbox / verification script that isn't the real host. A subprocess.Popen bash on Windows spawned from Python reports OSTYPE=linux-gnu (not msys), so sync.sh runs with the Linux branch, copies a fake HERMES_DIR to staging, and pushes that to origin — silently polluting the canonical repo. The fix is two-pronged: (a) sync.sh should detect [platform] and refuse to run if it doesn't match the actual host, or at minimum log the mismatch prominently; (b) verification scripts that need to test sync.sh should run it from a real terminal, not from a Python sandbox. Symptoms: commit history shows [linux] tags from a Windows user, or HERMES_DIR in commit log doesn't match the host.

14. Cron-spawned bash on Windows can flash a visible Git Bash console window every cycle. When the user has sync.sh running every 5 min via Hermes cron, each spawn can briefly pop a terminal window in the foreground. This happens because Git Bash inherits a console handle when launched non-interactively on Windows. The fix is an auto-detach guard at the very top of sync.sh, BEFORE set -e:

    if [ -z "${INTERACTIVE:-}" ] && [ -n "$WINDIR" ]; then
      exec </dev/null >/dev/null 2>&1
    fi
    

    Important gotcha: do NOT use [ ! -t 1 ] in the guard. MSYS bash reports stdout as a TTY even when it's a pipe, especially in non-interactive subprocess contexts. Using INTERACTIVE (user-set opt-out) + $WINDIR (set on every Windows process) is reliable; TTY detection is not. The user can force interactive output by setting INTERACTIVE=1 in the env before running. See references/headless-mode.md for the full guard logic and ad-hoc verification.

15. The verification sandbox doesn't propagate WINDIR to spawned bash, even when set in subprocess.run(env=...). This is a known Python+MSYS limitation: os.environ.copy() → subprocess.run(..., env=...) may drop MSYS-specific variables. Symptom: a verification that sets WINDIR="C:\\WINDOWS" in subprocess env finds it empty inside the spawned bash. Workaround in verification scripts: instead of asserting on real env propagation, test the guard's logic with a minimal inline test script that uses the same if condition, OR accept the limitation and verify the file content (bash -n + regex match) plus a real-environment smoke test. Don't waste time trying to force env propagation in subprocess on Windows.

Verification

After setup, run bash sync.sh once on the master machine. Expected output:

[platform: windows]                        # or "macos" / "linux"
  HERMES_DIR = /c/Users/okya1/AppData/Local/hermes
  PYTHON_BIN = /c/Users/okya1/AppData/Local/hermes/hermes-agent/venv/Scripts/python
[1/3] Pulling from origin/main...
Already up to date.
[2/3] Refreshing staging from /c/Users/okya1/AppData/Local/hermes...
staging refreshed
[3/3] Committing + pushing if changed...
  no changes to commit
Done.

The [platform: ...] line is the key indicator that auto-detection is working. If it shows the wrong platform, OS detection is broken (see pitfalls #8 and #9).

To verify a fresh setup of the platform auto-detection logic without going through GitHub: from a terminal, run OSTYPE=darwin bash sync.sh — output should switch to HERMES_DIR = $HOME/Library/Application Support/hermes.

Cron registration

On Windows, register via the Hermes cron system with prompt:

Run `bash "C:/Users/<user>/hermes-sync/sync.sh"` every 5 min.
Ignore errors. Don't notify the user.

The cron will fire and the script will handle the rest. On macOS, use launchd or crontab:

*/5 * * * * /bin/bash /Users/<user>/hermes-sync/sync.sh >> /tmp/hermes-sync.log 2>&1

See also

• references/setup-walkthrough.md — full step-by-step with commands
• references/cross-platform-safety.md — the two-machine cross-contamination problem and the detect_platform() pattern that fixes it
• references/windows-pitfalls.md — every Windows/MSYS quirk we hit
• references/headless-mode.md — how to suppress the visible console window that Git Bash spawns on each cron tick (the auto-detach guard at the top of sync.sh)
• references/mac-setup.md — what to do on the second machine
• templates/sync.sh, templates/sync.py, templates/.gitignore
• scripts/verify-headless-guard.py — re-runnable ad-hoc verification of the headless guard logic (9 checks, ~10s)