Repo Project Sync Workflow

KINTSUGI Repository-to-Project Sync Workflow

Experiment Overview

Context

KINTSUGI uses a shared codebase model where:

• Main repo: /blue/maigan/smith6jt/KINTSUGI/ contains the source code
• Project folders: /blue/maigan/smith6jt/KINTSUGI_Projects/.../notebooks/ contain working copies

Project folders sync FROM the main repo. If you edit a project folder directly, those changes will be overwritten when the user syncs from the main repo.

Verified Workflow

CORRECT: Edit Main Repo First

# 1. Make edits to the main repo
/blue/maigan/smith6jt/KINTSUGI/notebooks/Kdecon/deconvolution.py

# 2. Commit triggers auto-sync via post-commit hook
git commit -m "fix: update deconvolution"
# -> scripts/sync_to_projects.py auto-discovers and syncs ALL 33 projects

# 3. Manual sync (if needed outside of git commit)
python scripts/sync_to_projects.py          # sync changed files
python scripts/sync_to_projects.py --force  # force sync all files
python scripts/sync_to_projects.py --dry-run  # preview changes

Key Paths

Auto-Discovery Mechanism (Feb 2026)

Prior to Feb 18, 2026, sync_to_projects.py used a hardcoded DEFAULT_PROJECT_FOLDERS list containing only 2 projects. As batch processing scaled to 31+ datasets under KINTSUGI_Projects/, new projects were never synced. This caused CX_19-001_SP_CC2-A28 to run with 2-month-old Kreg code, crashing registration.

How It Works Now

• _BATCH_PROJECT_GLOB = "/blue/maigan/smith6jt/KINTSUGI_Projects/*/notebooks" discovers all batch project notebooks directories at import time
• _discover_default_project_folders() builds the full project list: 2 static entries + all glob matches
• Discovered 33 total projects (2 static + 31 batch) as of Feb 2026
• Post-commit hook calls sync_to_projects.py automatically on every commit
• First run after the change (commit 1f0bc2b) auto-synced 338 files to all 33 projects

Key Implementation Details

• Discovery happens at import time (module-level), not per-invocation
• Uses MD5 checksum comparison to skip unchanged files (efficient for 33 projects)
• New batch projects added to KINTSUGI_Projects/ are automatically included on next sync
• No manual DEFAULT_PROJECT_FOLDERS updates are ever needed again

Failed Attempts (Critical)

What Worked

• Auto-discovery via glob ensures new batch projects are automatically included without code changes
• Post-commit hook makes sync automatic -- no manual step to forget
• MD5 checksum comparison makes syncing 33 projects efficient (only copies changed files)
• Single source of truth pattern: main repo is always canonical, project folders are copies

Key Insights

• The main repo (/blue/maigan/smith6jt/KINTSUGI/) is the single source of truth
• Project folders are working copies that get synced FROM main repo
• After editing main repo, the post-commit hook auto-syncs to all discovered projects
• When user reports a fix "didn't work", check if they synced (overwriting your changes)
• Jupyter kernels cache imports but autoreload is enabled -- just re-run the cell, no kernel restart needed
• If a project has stale code, run python scripts/sync_to_projects.py --force to force-refresh all files

Trigger Conditions

This skill applies when:

• Editing any file in KINTSUGI/notebooks/ subdirectories (Kdecon, Kstitch, Kreg, Kview2, etc.)
• User mentions syncing or copying files between repos
• A fix "doesn't work" after being applied
• Working with KINTSUGI_Projects folders

References

• KINTSUGI CLAUDE.md development workspace section
• VS Code multi-root workspace: kintsugi-dev.code-workspace