# Fix Jsonl Surrogates

> Diagnose and repair 'invalid high surrogate' API errors in Claude Code chat sessions. Use this skill when the user encounters API Error 400 with 'invalid high surrogate in string', 'not valid JSON', surrogate-related errors, or wants to fix/scan/repair a broken Claude Code JSONL chat file. Also use when a session refuses to resume due to JSON encoding errors, or when the user mentions request IDs (req_...) alongside JSON parse failures.

- Skill: `tebjan/fix-jsonl-surrogates` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add tebjan/fix-jsonl-surrogates`
- Raw SKILL.md: https://api.skillmd.com/api/skills/tebjan/fix-jsonl-surrogates/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Integrations & APIs
- License: CC-BY-SA-4.0
- Author: tebjan (https://skillmd.com/u/tebjan)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/tebjan/fix-jsonl-surrogates

---


# Fix JSONL Surrogates

Repairs Claude Code JSONL chat files that contain lone Unicode surrogates (U+D800-U+DFFF),
which cause the Anthropic API to reject requests with "invalid high surrogate in string".

## Background

Node.js internally uses UTF-16 strings. `JSON.stringify()` doesn't validate surrogate pairs,
so lone surrogates from file reads, terminal output, or web content can end up serialized
as `\uD8xx` without a matching low surrogate — producing invalid JSON that the API rejects.

Common triggers on Windows:
- Emoji in code comments or terminal output
- Box-drawing characters from build logs
- Web content with malformed UTF-16 encoding

Upstream issue: anthropics/claude-code#44230

## How to Use

### Quick diagnosis

If the user has a specific session or request ID, find the JSONL file:

```bash
grep -rl "req_XXXXX" ~/.claude/projects/*/
```

### Run the repair script

The bundled Python script handles scanning and fixing. It works on individual files,
session directories, or entire project trees.

```bash
# Scan a specific JSONL file + its session directory (subagents, tool-results)
python {SKILL_DIR}/scripts/fix_surrogates.py <path-to-file.jsonl>

# Dry run first to see what would change
python {SKILL_DIR}/scripts/fix_surrogates.py <path-to-file.jsonl> --dry-run --verbose

# Scan all sessions in a project directory
python {SKILL_DIR}/scripts/fix_surrogates.py ~/.claude/projects/<project-dir>/

# Scan everything recursively (includes subagent files)
python {SKILL_DIR}/scripts/fix_surrogates.py ~/.claude/projects/<project-dir>/ --recursive
```

The script:
1. Reads each file with `errors='surrogateescape'` to preserve bad characters for detection
2. Parses JSON and recursively checks all string values for surrogate code points
3. Replaces lone surrogates with U+FFFD (replacement character) via `str.encode('utf-8', 'surrogatepass').decode('utf-8', 'replace')`
4. Backs up originals as `.bak` before modifying
5. Reports a summary of what was found and fixed

### Important notes

- **Surrogates are often transient**: They may exist only in Node.js memory during request
  construction, not in the stored JSONL. If the script finds nothing, the error was transient
  and retrying the session should work.
- **Check all session artifacts**: The script automatically scans the main JSONL plus any
  subagent files and tool-result cache in the session directory.
- **Safe to re-run**: The script is idempotent. Running it on already-clean files is a no-op.

### Manual investigation

If the script finds nothing but the error persists, the surrogate may come from content
loaded at request time (CLAUDE.md, memory files, etc.). Scan those too:

```bash
python {SKILL_DIR}/scripts/fix_surrogates.py ~/.claude/ --recursive --dry-run
```

### What `{SKILL_DIR}` means

Replace `{SKILL_DIR}` with the actual path to this skill directory. When Claude invokes
this skill, it knows the skill's location and can substitute the correct path.