How do I install the Fix Jsonl Surrogates skill?

Run npx skillmds@latest add tebjan/fix-jsonl-surrogates in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Fix Jsonl Surrogates safe to use?

This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Fix Jsonl Surrogates?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Fix Jsonl Surrogates free to use?

Yes. Installing skills from SkillMD is free. This skill is licensed under CC-BY-SA-4.

Who published Fix Jsonl Surrogates?

tebjan (@tebjan) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Fix Jsonl Surrogates by tebjan — Agent Skill

Name: Fix Jsonl Surrogates
Author: tebjan

Fix Jsonl Surrogates

Diagnose and repair 'invalid high surrogate' API errors in Claude Code chat sessions. Use this skill when the user encounters API Error 400 with 'invalid high surrogate in string', 'not valid JSON', surrogate-related errors, or wants to fix/scan/repair a broken Claude Code JSONL chat file. Also use when a session refuses to resume due to JSON encoding errors, or when the user mentions request IDs (req_...) alongside JSON parse failures.

tebjan 1189f18 2 files · 11.7 KB Updated Sep 17, 2026

File contents

Fix JSONL Surrogates

Repairs Claude Code JSONL chat files that contain lone Unicode surrogates (U+D800-U+DFFF), which cause the Anthropic API to reject requests with "invalid high surrogate in string".

Background

Node.js internally uses UTF-16 strings. JSON.stringify() doesn't validate surrogate pairs, so lone surrogates from file reads, terminal output, or web content can end up serialized as \uD8xx without a matching low surrogate — producing invalid JSON that the API rejects.

Common triggers on Windows:

Emoji in code comments or terminal output
Box-drawing characters from build logs
Web content with malformed UTF-16 encoding

Upstream issue: anthropics/claude-code#44230

How to Use

Quick diagnosis

If the user has a specific session or request ID, find the JSONL file:

grep -rl "req_XXXXX" ~/.claude/projects/*/

Run the repair script

The bundled Python script handles scanning and fixing. It works on individual files, session directories, or entire project trees.

# Scan a specific JSONL file + its session directory (subagents, tool-results)
python {SKILL_DIR}/scripts/fix_surrogates.py <path-to-file.jsonl>

# Dry run first to see what would change
python {SKILL_DIR}/scripts/fix_surrogates.py <path-to-file.jsonl> --dry-run --verbose

# Scan all sessions in a project directory
python {SKILL_DIR}/scripts/fix_surrogates.py ~/.claude/projects/<project-dir>/

# Scan everything recursively (includes subagent files)
python {SKILL_DIR}/scripts/fix_surrogates.py ~/.claude/projects/<project-dir>/ --recursive

The script:

Reads each file with errors='surrogateescape' to preserve bad characters for detection
Parses JSON and recursively checks all string values for surrogate code points
Replaces lone surrogates with U+FFFD (replacement character) via str.encode('utf-8', 'surrogatepass').decode('utf-8', 'replace')
Backs up originals as .bak before modifying
Reports a summary of what was found and fixed

Important notes

Surrogates are often transient: They may exist only in Node.js memory during request construction, not in the stored JSONL. If the script finds nothing, the error was transient and retrying the session should work.
Check all session artifacts: The script automatically scans the main JSONL plus any subagent files and tool-result cache in the session directory.
Safe to re-run: The script is idempotent. Running it on already-clean files is a no-op.

Manual investigation

If the script finds nothing but the error persists, the surrogate may come from content loaded at request time (CLAUDE.md, memory files, etc.). Scan those too:

python {SKILL_DIR}/scripts/fix_surrogates.py ~/.claude/ --recursive --dry-run

What `{SKILL_DIR}` means

Replace {SKILL_DIR} with the actual path to this skill directory. When Claude invokes this skill, it knows the skill's location and can substitute the correct path.

---name: fix-jsonl-surrogatesdescription: Diagnose and repair 'invalid high surrogate' API errors in Claude Code chat sessions. Use this skill when the user encounters API Error 400 with 'invalid high surrogate in string', 'not valid JSON', surrogate-related errors, or wants to fix/scan/repair a broken Claude Code JSONL chat file. Also use when a session refuses to resume due to JSON encoding errors, or when the user mentions request IDs (req_...) alongside JSON parse failures.license: CC-BY-SA-4.0---# Fix JSONL SurrogatesRepairs Claude Code JSONL chat files that contain lone Unicode surrogates (U+D800-U+DFFF),which cause the Anthropic API to reject requests with "invalid high surrogate in string".## BackgroundNode.js internally uses UTF-16 strings. `JSON.stringify()` doesn't validate surrogate pairs,so lone surrogates from file reads, terminal output, or web content can end up serializedas `\uD8xx` without a matching low surrogate — producing invalid JSON that the API rejects.Common triggers on Windows:- Emoji in code comments or terminal output- Box-drawing characters from build logs- Web content with malformed UTF-16 encodingUpstream issue: anthropics/claude-code#44230## How to Use### Quick diagnosisIf the user has a specific session or request ID, find the JSONL file:```bashgrep -rl "req_XXXXX" ~/.claude/projects/*/```### Run the repair scriptThe bundled Python script handles scanning and fixing. It works on individual files,session directories, or entire project trees.```bash# Scan a specific JSONL file + its session directory (subagents, tool-results)python {SKILL_DIR}/scripts/fix_surrogates.py <path-to-file.jsonl># Dry run first to see what would changepython {SKILL_DIR}/scripts/fix_surrogates.py <path-to-file.jsonl> --dry-run --verbose# Scan all sessions in a project directorypython {SKILL_DIR}/scripts/fix_surrogates.py ~/.claude/projects/<project-dir>/# Scan everything recursively (includes subagent files)python {SKILL_DIR}/scripts/fix_surrogates.py ~/.claude/projects/<project-dir>/ --recursive```The script:1. Reads each file with `errors='surrogateescape'` to preserve bad characters for detection2. Parses JSON and recursively checks all string values for surrogate code points3. Replaces lone surrogates with U+FFFD (replacement character) via `str.encode('utf-8', 'surrogatepass').decode('utf-8', 'replace')`4. Backs up originals as `.bak` before modifying5. Reports a summary of what was found and fixed### Important notes- **Surrogates are often transient**: They may exist only in Node.js memory during request construction, not in the stored JSONL. If the script finds nothing, the error was transient and retrying the session should work.- **Check all session artifacts**: The script automatically scans the main JSONL plus any subagent files and tool-result cache in the session directory.- **Safe to re-run**: The script is idempotent. Running it on already-clean files is a no-op.### Manual investigationIf the script finds nothing but the error persists, the surrogate may come from contentloaded at request time (CLAUDE.md, memory files, etc.). Scan those too:```bashpython {SKILL_DIR}/scripts/fix_surrogates.py ~/.claude/ --recursive --dry-run```### What `{SKILL_DIR}` meansReplace `{SKILL_DIR}` with the actual path to this skill directory. When Claude invokesthis skill, it knows the skill's location and can substitute the correct path.

Frequently asked questions