# Hermes State Sync Via Git

> Use when the user wants to sync Hermes Agent state (config, soul, auth, memories, skills, cron) across multiple machines via a private Git repo. Covers bi-directional sync via cron, PAT auth on GitHub, MSYS path issues on Windows, the .gitignore allowlist trick, the python3→python Microsoft Store stub bug, the staging-vs-working-tree pattern that prevents sync from corrupting a running Hermes process, and the **two-machine cross-contamination problem** (Mac editing HERMES_DIR silently breaks Windows) solved by OSTYPE auto-detection in sync.sh.

- Skill: `wcpaka-lgtm/hermes-state-sync-via-git` (Agent Skill, multi-file: 10 files)
- Install (CLI): `npx skillmds@latest add wcpaka-lgtm/hermes-state-sync-via-git`
- Raw SKILL.md: https://api.skillmd.com/api/skills/wcpaka-lgtm/hermes-state-sync-via-git/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: wcpaka-lgtm (https://skillmd.com/u/wcpaka-lgtm)
- Updated: 2026-09-21
- Page: https://skillmd.com/skills/wcpaka-lgtm/hermes-state-sync-via-git

---


# Hermes State Sync Across Machines (via private Git repo)

User wants the same Hermes Agent config/skills/memories/cron on multiple machines (e.g. macOS master + Windows mirror). The pattern: a private GitHub repo acts as canonical store; each machine runs a periodic sync that pulls, refreshes a staging dir from local Hermes state, and pushes if changed.

## Architecture

```
┌─────────────────┐                ┌─────────────────┐
│  macOS (master) │  ←──push/pull──│ Windows (mirror)│
│  Hermes state   │                │  Hermes state   │
└────────┬────────┘                └────────┬────────┘
         │ pull/push (cron)                │ pull/push (cron)
         └──────────────┬───────────────────┘
                        │
                ┌───────▼────────┐
                │  GitHub private │
                │  hermes-sync    │
                │  (canonical)    │
                └────────────────┘
```

**Critical pattern: staging dir, not symlink.**
Never symlink the working Hermes data dir into a Git repo. Hermes holds open file handles (SQLite WAL, .lock files) — `git add -A` will read inconsistent state and pushes will fail. Always copy into a separate staging dir, commit, push, then have the *other* machine's sync pull and copy back over.

## What to sync

| Item | Sync? | Why |
|---|---|---|
| `config.yaml` | ✅ | Cross-machine settings |
| `SOUL.md` | ✅ | Agent identity |
| `auth.json` | ✅ | Auth profiles (per-machine tokens may differ — accept merge conflicts) |
| `memories/` | ✅ | Persistent memory entries |
| `skills/` | ✅ | Installed skills |
| `cron/` | ✅ | Scheduled jobs |
| `.env` | ❌ | Secrets |
| `*.lock`, `*.db-shm`, `*.db-wal` | ❌ | Transient locks, SQLite WAL |
| `state.db*`, `kanban.db*`, `projects.db*` | ❌ | Live process state — would conflict |
| `audio_cache/`, `image_cache/`, `cache/`, `logs/`, `sessions/` | ❌ | Machine-local, large |
| `hermes-agent/` | ❌ | App source — installed separately per machine |
| `models_dev_cache.json`, `provider_models_cache.json`, `ollama_cloud_models_cache.json` | ❌ | Per-machine API cache |

## Default paths

- **macOS Hermes dir:** `~/Library/Application Support/hermes/`
- **Windows Hermes dir:** `%LOCALAPPDATA%\hermes` (e.g. `C:\Users\<user>\AppData\Local\hermes`)
- **Staging dir (any machine):** anywhere outside the Hermes dir, e.g. `~/hermes-sync/`

## Reference scripts and templates

- `templates/sync.sh` — bash sync script (the one that actually runs)
- `templates/sync.py` — pure-Python equivalent, more portable
- `templates/.gitignore` — block-everything-then-allowlist pattern
- `references/setup-walkthrough.md` — step-by-step PAT + repo creation + first push
- `references/windows-pitfalls.md` — every Windows-specific gotcha we hit (MSYS, CRLF, python3 stub, etc.)
- `references/mac-setup.md` — what to do on the other machine after Windows is up

## Quick setup flow

1. **Create a GitHub PAT** with `repo` scope (fine-grained or classic both work; classic is simpler for this use case).
2. **Create the private repo** via `curl -X POST .../user/repos` with `private: true` and `auto_init: true`.
3. **Set git identity** on the master machine:
   ```bash
   git config --global user.name "okya1"
   git config --global user.email "<user@email>"
   ```
4. **Build the staging dir** by copying the sync items from Hermes dir (see `references/setup-walkthrough.md`).
5. **Write `.gitignore`** that blocks everything then allowlists the items — see `templates/.gitignore`. **Critical: explicitly list `!sync.sh` and `!README.md` in the allowlist**, otherwise they're blocked by the catch-all `*` rule and the sync script never gets committed.
6. **First push:** init repo, add the (unrelated) remote, `git pull --allow-unrelated-histories`, then push.
7. **Wire up cron** (every 5 min) to call `bash sync.sh`.

## sync.sh essentials

The script **MUST auto-detect its OS** rather than hardcode `HERMES_DIR`. See pitfall #8 below for the cross-contamination failure mode this prevents.

```bash
#!/usr/bin/env bash
set -e
SYNC_DIR="$(cd "$(dirname "$0")" && pwd)"

# === Auto-detect platform at runtime ===
detect_platform() {
  case "$OSTYPE" in
    msys*|win32*|cygwin*)
      # Windows. $USER is often empty in Git Bash — fall back to USERPROFILE
      # and convert via cygpath (preferred) or sed.
      if [ -n "$USERPROFILE" ]; then
        if command -v cygpath >/dev/null 2>&1; then
          WIN_USER_PATH="$(cygpath -u "$USERPROFILE")"
        else
          WIN_USER_PATH="$(echo "$USERPROFILE" | sed -E 's|^([A-Za-z]):|/\L\1|; s|\\|/|g')"
        fi
      elif [ -n "$USERNAME" ]; then
        WIN_USER_PATH="/c/Users/$USERNAME"
      else
        WIN_USER_PATH="$(echo "$HOME" | sed -E 's|.*/Users/([^/]+).*|\1|' | xargs -I{} echo "/c/Users/{}")"
      fi
      HERMES_DIR="$WIN_USER_PATH/AppData/Local/hermes"
      if [ -x "$WIN_USER_PATH/AppData/Local/hermes/hermes-agent/venv/Scripts/python" ]; then
        PYTHON_BIN="$WIN_USER_PATH/AppData/Local/hermes/hermes-agent/venv/Scripts/python"
      else
        PYTHON_BIN="python"
      fi
      PLATFORM_LABEL="windows"
      ;;
    darwin*)
      HERMES_DIR="$HOME/Library/Application Support/hermes"
      PYTHON_BIN="/usr/bin/python3"
      PLATFORM_LABEL="macos"
      ;;
    linux*)
      HERMES_DIR="$HOME/.local/share/hermes"
      PYTHON_BIN="/usr/bin/python3"
      PLATFORM_LABEL="linux"
      ;;
    *)
      echo "ERROR: unknown OS type: $OSTYPE" >&2
      echo "  Set HERMES_DIR_OVERRIDE and PYTHON_BIN_OVERRIDE env vars, then re-run." >&2
      exit 1
      ;;
  esac
  # Allow override via env (e.g. for non-standard installs)
  HERMES_DIR="${HERMES_DIR_OVERRIDE:-$HERMES_DIR}"
  PYTHON_BIN="${PYTHON_BIN_OVERRIDE:-$PYTHON_BIN}"
}
detect_platform
export HERMES_DIR PYTHON_BIN
echo "[platform: $PLATFORM_LABEL] HERMES_DIR=$HERMES_DIR PYTHON_BIN=$PYTHON_BIN"

REMOTE="origin"
BRANCH="main"

# 1) Pull latest — DON'T let a pull failure kill the run
cd "$SYNC_DIR"
if ! git pull --rebase --autostash "$REMOTE" "$BRANCH" 2>&1; then
  echo "  pull failed (will continue with local state)"
fi

# 2) Refresh staging from local Hermes state
echo "[2/3] Refreshing staging from $HERMES_DIR..."
"$PYTHON_BIN" - <<'PYEOF'
import os, shutil
from pathlib import Path
HERMES = Path(os.environ["HERMES_DIR"])
SYNC   = Path(os.environ["SYNC_DIR"])
ITEMS = {
  "config": "config.yaml", "soul": "SOUL.md", "auth": "auth.json",
  "memories": "memories", "skills": "skills", "cron": "cron",
}
# Tolerate Windows file locks (Hermes desktop holds .lock/.db-wal/.db
# open). Single copytree with dirs_exist_ok + ignore callback merges
# into the existing staging without rmtree race. See pitfall #11.
def _ignore_readonly(src_dir, names):
    out = []
    for n in names:
        p = Path(src_dir) / n
        if p.is_file():
            try:
                with open(p, "rb"): pass
            except (PermissionError, OSError):
                out.append(n)
    return out
for name, rel in ITEMS.items():
    src = HERMES / rel
    dst = SYNC / name
    if not src.exists(): continue
    if src.is_dir():
        shutil.copytree(src, dst, dirs_exist_ok=True, ignore=_ignore_readonly)
    else:
        shutil.copy2(src, dst)
print("staging refreshed")
PYEOF

# 3) Stage and push if changed
git add -A
if git diff --staged --quiet; then
  echo "  no changes to commit"
else
  git commit -m "auto-sync $(date '+%Y-%m-%d %H:%M:%S') from $(hostname) [$PLATFORM_LABEL]"
  git push "$REMOTE" "$BRANCH"
  echo "  pushed"
fi
echo "Done."
```

## Critical pitfalls (the ones that actually bit)

1. **`python3` is broken on Windows** — resolves to the Microsoft Store stub which silently fails on heredocs with exit 49. **Use `python`**, which resolves to the real venv. (Both are in PATH on this machine.)
2. **`set -e` + `git pull`** — a transient network blip will kill the whole sync run. Wrap in `if ! ... ; then echo ... ; fi`.
3. **`.gitignore` catch-all `*` blocks your own script** — must explicitly allowlist `!sync.sh` and `!README.md`, otherwise `git add -A` silently skips them and the fix never reaches the remote.
4. **Don't `-f best` on Twitter videos** (related but different skill) — Twitter ships video and audio as separate HLS streams; let yt-dlp pick the best of each and merge. (This pitfall belongs in the video skill, not here, but it bites in the same Windows env.)
5. **MSYS bash on Windows mangles backslashes in absolute paths passed as args** — `bash -n 'C:\Users\foo\sync.sh'` becomes `C:Usersfoosync.sh`. Workaround: `cd` to the dir first and pass a relative path, or copy to `/tmp` and check there. **For subprocess in Python**: convert the path yourself with `str(p).replace("\\","/")` and prepend `/c/...` form, or pass `cwd=<dir> + "script.sh"` (relative).
6. **`subprocess.run(capture_output=True)` on Windows drops the first line of bash output** — use `Popen + communicate` with bytes mode and explicit UTF-8 decode (Windows bash often emits Korean codepage, not UTF-8). This bit us hard in the verification script. When `WINDIR` is set in MSYS, output encoding switches to **UTF-16LE** — always pass `text=False` and decode with `errors="replace"`, not `errors="strict"`.
7. **First push to a repo with auto-init README**: `git push` is rejected with "fetch first". Fix: `git pull --allow-unrelated-histories --no-edit`, then push.
8. **`sync.sh` must auto-detect OS — never hardcode `HERMES_DIR`**. If Mac edits `HERMES_DIR="$HOME/Library/Application Support/hermes"` and pushes, Windows's next pull silently overwrites its own Windows path with the Mac one. `bash sync.sh` then keeps "succeeding" while refreshing the wrong directory — silent corruption. The fix is to detect `$OSTYPE` at runtime and pick the right path. See `references/cross-platform-safety.md` for the full failure mode and the `detect_platform()` function.
9. **`$USER` is empty in Git Bash on Windows.** Use `$USERPROFILE` (always set by Windows), then convert with `cygpath -u` (preferred) or `sed` to MSYS form `/c/Users/...`. Falling back to `$USERNAME` only works if it matches the home dir name (e.g. WSL has them divergent).
10. **`write_file` saves with CRLF on Windows.** Bash then chokes on `$'\r'` syntax errors. After any script edit on Windows, strip CR before committing: `python -c "p=Path('sync.sh'); p.write_bytes(p.read_bytes().replace(b'\r',b''))"`. Verify with `xxd sync.sh | head -2` — should be `23 21 2f 62 69 6e 2f 62 61 73 68 0a` (LF, not CRLF).
11. **`shutil.copytree(src, dst, ignore_errors=True)` does NOT exist as a parameter in Python's stdlib.** Despite the name suggesting a flag, `copytree` only accepts `ignore` (callable) and `onerror` (legacy) for error handling. `ignore_errors` is a parameter on `shutil.rmtree` only. And the seemingly-safe pattern `rmtree(ignore_errors=True) → copytree(dirs_exist_ok=True)` has a race: if `rmtree` leaves locked files behind (e.g. SQLite WAL, .lock files held by the running Hermes process), `copytree` raises `FileExistsError` on the next iteration and the staging dir ends up empty/broken. The correct pattern is a single `shutil.copytree(src, dst, dirs_exist_ok=True, ignore=callback)` that tests each file for read access and skips locked ones, leaving the rest intact. The `ignore` callback is `(src_dir, names) -> list_of_names_to_skip`.
12. **The `.gitignore` allowlist must use `**` to recurse into whitelisted directories.** A line like `!skills/` only allows the directory entry to be tracked, not the files inside — git treats the dir as an empty placeholder. You need both: `!skills/` (so git sees the dir) AND `!skills/**` (so the files inside are tracked). Same for `!memories/**` and `!cron/**`. Without the `**`, only the top-level files (config, soul, auth) get tracked and the memory/skills/cron directories appear empty in `git ls-files`. Verify with `git ls-tree -r --name-only origin/main | grep -E '^(memories|skills|cron)/' | wc -l` — should be hundreds, not 0.
13. **Cross-machine pushes are dangerous: never auto-push from a sandbox / verification script that isn't the real host.** A `subprocess.Popen` bash on Windows spawned from Python reports `OSTYPE=linux-gnu` (not `msys`), so `sync.sh` runs with the Linux branch, copies a fake `HERMES_DIR` to staging, and pushes that to origin — silently polluting the canonical repo. The fix is two-pronged: (a) `sync.sh` should detect `[platform]` and refuse to run if it doesn't match the actual host, or at minimum log the mismatch prominently; (b) verification scripts that need to test `sync.sh` should run it from a real terminal, not from a Python sandbox. Symptoms: commit history shows `[linux]` tags from a Windows user, or `HERMES_DIR` in commit log doesn't match the host.
14. **Cron-spawned bash on Windows can flash a visible Git Bash console window every cycle.** When the user has `sync.sh` running every 5 min via Hermes cron, each spawn can briefly pop a terminal window in the foreground. This happens because Git Bash inherits a console handle when launched non-interactively on Windows. The fix is an **auto-detach guard at the very top of `sync.sh`**, BEFORE `set -e`:

    ```bash
    if [ -z "${INTERACTIVE:-}" ] && [ -n "$WINDIR" ]; then
      exec </dev/null >/dev/null 2>&1
    fi
    ```

    Important gotcha: **do NOT use `[ ! -t 1 ]` in the guard.** MSYS bash reports stdout as a TTY even when it's a pipe, especially in non-interactive subprocess contexts. Using `INTERACTIVE` (user-set opt-out) + `$WINDIR` (set on every Windows process) is reliable; TTY detection is not. The user can force interactive output by setting `INTERACTIVE=1` in the env before running. See `references/headless-mode.md` for the full guard logic and ad-hoc verification.

15. **The verification sandbox doesn't propagate `WINDIR` to spawned bash, even when set in `subprocess.run(env=...)`.** This is a known Python+MSYS limitation: `os.environ.copy()` → `subprocess.run(..., env=...)` may drop MSYS-specific variables. Symptom: a verification that sets `WINDIR="C:\\WINDOWS"` in subprocess env finds it empty inside the spawned bash. **Workaround in verification scripts**: instead of asserting on real env propagation, test the guard's *logic* with a minimal inline test script that uses the same `if` condition, OR accept the limitation and verify the file content (`bash -n` + regex match) plus a real-environment smoke test. Don't waste time trying to force env propagation in subprocess on Windows.

## Verification

After setup, run `bash sync.sh` once on the master machine. Expected output:
```
[platform: windows]                        # or "macos" / "linux"
  HERMES_DIR = /c/Users/okya1/AppData/Local/hermes
  PYTHON_BIN = /c/Users/okya1/AppData/Local/hermes/hermes-agent/venv/Scripts/python
[1/3] Pulling from origin/main...
Already up to date.
[2/3] Refreshing staging from /c/Users/okya1/AppData/Local/hermes...
staging refreshed
[3/3] Committing + pushing if changed...
  no changes to commit
Done.
```

The `[platform: ...]` line is the key indicator that auto-detection is working. If it shows the wrong platform, OS detection is broken (see pitfalls #8 and #9).

To verify a fresh setup of the platform auto-detection logic without going through GitHub: from a terminal, run `OSTYPE=darwin bash sync.sh` — output should switch to `HERMES_DIR = $HOME/Library/Application Support/hermes`.

## Cron registration

On Windows, register via the Hermes cron system with prompt:
```
Run `bash "C:/Users/<user>/hermes-sync/sync.sh"` every 5 min.
Ignore errors. Don't notify the user.
```
The cron will fire and the script will handle the rest. On macOS, use launchd or crontab:
```
*/5 * * * * /bin/bash /Users/<user>/hermes-sync/sync.sh >> /tmp/hermes-sync.log 2>&1
```

## See also

- `references/setup-walkthrough.md` — full step-by-step with commands
- `references/cross-platform-safety.md` — the two-machine cross-contamination problem and the `detect_platform()` pattern that fixes it
- `references/windows-pitfalls.md` — every Windows/MSYS quirk we hit
- `references/headless-mode.md` — how to suppress the visible console window that Git Bash spawns on each cron tick (the auto-detach guard at the top of sync.sh)
- `references/mac-setup.md` — what to do on the second machine
- `templates/sync.sh`, `templates/sync.py`, `templates/.gitignore`
- `scripts/verify-headless-guard.py` — re-runnable ad-hoc verification of the headless guard logic (9 checks, ~10s)