# Upstream Sync

> Synchronize content from an upstream Git repository into the local repo with path mapping, conflict detection, and integrity verification. Use when the user asks to sync, merge, or pull changes from an upstream/fork repository, or mentions keywords like "upstream sync", "sync upstream", "上游同步", "同步上游". Accepts upstream repo URL and starting commit SHA.

- Skill: `cklwblove/upstream-sync` (Agent Skill, multi-file: 4 files)
- Install (CLI): `npx skillmds@latest add cklwblove/upstream-sync`
- Raw SKILL.md: https://api.skillmd.com/api/skills/cklwblove/upstream-sync/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: cklwblove (https://skillmd.com/u/cklwblove)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/cklwblove/upstream-sync

---


# Upstream Sync

Sync content from an upstream repository into the local repo, handling different directory structures via path mapping.

## Bundled Scripts

The skill includes sync scripts in `scripts/`:

- `scripts/sync-upstream.ts` — 5-phase sync orchestrator
- `scripts/sync-utils.ts` — path mapping, git helpers, integrity checks

## Workflow

### Step 0: Deploy Scripts to Project

Copy the bundled scripts to the project's `scripts/` directory (skip if already present and up-to-date):

```bash
mkdir -p <PROJECT_ROOT>/scripts
cp <SKILL_DIR>/scripts/sync-upstream.ts <PROJECT_ROOT>/scripts/
cp <SKILL_DIR>/scripts/sync-utils.ts <PROJECT_ROOT>/scripts/
```

The project needs `tsx` in devDependencies. If missing: `npm i -D tsx`.

### Step 1: Configure Upstream Remote

```bash
git remote get-url upstream 2>/dev/null || git remote add upstream <UPSTREAM_URL>
git fetch upstream
```

### Step 2: Generate Path Mappings

Analyze both repos and create `<PROJECT_ROOT>/.upstream-mapping.json`:

```bash
# List upstream root structure
git ls-tree --name-only upstream/main

# List local content directories
ls <PROJECT_ROOT>/docs/
```

Generate mapping config:

```json
{
  "_comment": "Upstream -> local path mapping",
  "exclude": ["README.md"],
  "mappings": [
    { "upstream": "01_intro/", "local": "docs/zh/01_intro/" },
    { "upstream": "02_fundamentals/", "local": "docs/zh/02_fundamentals/" },
    { "upstream": "_images/", "local": "docs/zh/_images/" },
    { "upstream": "SUMMARY.md", "local": "SUMMARY.md" }
  ]
}
```

**Mapping rules:**
- Directory: upstream path ends with `/`, local path ends with `/`
- File: exact paths, no trailing `/`
- `exclude`: upstream paths skipped entirely (never synced)
- First match wins

**Preserve local rules (`preserveLocal`):**

Use `preserveLocal` to specify line patterns that should be preserved from the local version during sync. This is useful when you've made local customizations (like replacing GitHub URLs with local paths) that should not be overwritten by upstream changes.

```json
{
  "preserveLocal": [
    {
      "description": "Keep local code file links instead of GitHub URLs",
      "pattern": "\\[code/[^\\]]+\\]\\(code/",
      "glob": "*.md"
    },
    {
      "description": "Keep local relative doc links instead of GitHub absolute paths",
      "pattern": "\\]\\(\\.\\./chapter",
      "glob": "*.md"
    }
  ]
}
```

**Preserve rule fields:**
- `description`: Human-readable description of what this rule preserves
- `pattern`: Regex pattern to match lines that should be preserved from local version
- `glob` (optional): File pattern to limit which files this rule applies to (e.g., `*.md`)

**How it works:**
- During auto-merge, if a local file exists and preserve rules are configured
- Each line in the upstream content is checked against the preserve patterns
- If a line matches a pattern AND the local file has a different version of that line
- The local version is preserved instead of being overwritten by upstream

### Step 3: Initialize State

Create `<PROJECT_ROOT>/sync-state.json` with the starting commit:

```json
{
  "upstream_remote": "upstream",
  "upstream_branch": "main",
  "last_synced_commit": "<STARTING_SHA>",
  "last_sync_time": null,
  "sync_history": []
}
```

### Step 4: Run Sync

```bash
cd <PROJECT_ROOT>
node --import tsx scripts/sync-upstream.ts --from <COMMIT_SHA>
```

| Flag | Description |
|------|-------------|
| `--from <sha>` | Start from this commit (overrides state file) |
| `--dry-run` | Preview only, no changes |
| `--no-fetch` | Skip `git fetch` (data already local) |
| `--skip-build-check` | Skip `npm run docs:build` verification |

### Step 5: Review & Merge

1. Check terminal output for auto-merged, manual-review, and **unmapped** files
2. Review `sync-conflict-report.md` for details (includes suggested mapping JSON for unmapped paths)
3. If unmapped new upstream dirs appear: add suggested entries to `.upstream-mapping.json`, then re-run sync
4. Handle manual-review files (large diffs >= 30 lines, deleted files, SUMMARY.md):
   ```bash
   git show upstream/main:<upstream-path>   # view upstream version
   # edit local file, then stage + commit
   ```
5. Merge into main:
   ```bash
   git checkout main
   git merge sync/<timestamp>
   git branch -d sync/<timestamp>
   ```

## Auto vs Manual Classification

| Condition | Handling |
|-----------|----------|
| New file (added) within mapped path | Auto-merge — write new local file + `git add` |
| Rename within mapped path | Auto-merge — write new path, remove old local path |
| < 30 lines, no local modifications | Auto-merge |
| >= 30 lines changed | Manual review — keep local, do not overwrite |
| Has local-only modifications | Manual review |
| File deleted upstream | Manual review — **keep local file** until reviewed |
| `SUMMARY.md` (any depth) | Always manual review |
| Path matches no mapping rule | **Not synced** — listed in report with suggested mapping |

## Unmapped New Paths

Upstream may add new top-level directories that are not in `.upstream-mapping.json`. These are **not** silently ignored:

1. Phase 2 prints each unmapped path with kind (`added` / `modified` / …)
2. Suggests mapping entries inferred from existing local prefix (e.g. `docs/zh/`)
3. Writes them into `sync-conflict-report.md` under **Unmapped Upstream Paths**
4. After you add the mappings, re-run sync to pull those files

## Integrity Verification

Post-merge checks:
- Directory targets exist and contain non-empty `.md` files
- File targets exist and are non-empty
- New files written from upstream are listed
- No files lost compared to pre-merge snapshot (deleted upstream files are not auto-removed)

## Sync Phases

1. **Prepare** — read state, fetch upstream, detect new commits
2. **Analyse** — diff files, apply path mapping + exclude, classify auto vs manual, warn on unmapped
3. **Backup** — create `backup/pre-sync-<ts>` and `sync/<ts>` branches
4. **Merge** — file-by-file via `git show <commit>:<path>`, write/add mapped local paths, stage dirs + files + applied paths
5. **Report** — update state, print summary (incl. unmapped + suggestions), save report