Xiaohongshu Article to Markdown
Extracts authentication from a Chromium browser via CDP, fetches post data via httpx, downloads carousel images, and OCRs them into a clean Markdown file using macOS Vision Framework.
Quick Start
python3 <skill-path>/scripts/xhs_to_markdown.py <xiaohongshu_url> [--output-dir DIR] [--cdp-port PORT]
Arguments:
url(required): Xiaohongshu post URL (e.g.,https://www.xiaohongshu.com/explore/...)--output-dir: Output directory, defaults to current working directory--cdp-port: CDP debugging port, default 9222
Output file: {title}_{noteId}.md
Prerequisites
- macOS (uses Vision Framework for OCR — no Python OCR dependencies needed)
- Chromium-based browser (Chrome / Edge / Brave / Arc) installed and logged in to xiaohongshu.com
- Python dependencies:
pip install playwright httpx && playwright install chromium - Swift (pre-installed on macOS)
Launch browser with CDP:
# Arc
/Applications/Arc.app/Contents/MacOS/Arc --remote-debugging-port=9222
# Chrome
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --remote-debugging-port=9222
# Edge
/Applications/Microsoft\ Edge.app/Contents/MacOS/Microsoft\ Edge --remote-debugging-port=9222
# Brave
/Applications/Brave\ Browser.app/Contents/MacOS/Brave\ Browser --remote-debugging-port=9222
Architecture
URL → CDP(extract cookies) → httpx(fetch HTML) → parse __INITIAL_STATE__ JSON
→ extract HD image URLs → httpx download images → Swift Vision OCR → Markdown
| Step | Method | Notes |
|---|---|---|
| Cookie extraction | CDP ctx.cookies() |
Invisible, no tabs opened |
| Page fetch | httpx GET | With browser cookies for auth |
| Data parsing | __INITIAL_STATE__ JSON |
Embedded in page HTML |
| Image download | httpx concurrent | Strips quality compression suffixes |
| OCR | macOS Vision Framework (Swift) | .accurate level, zh-Hans + zh-Hant + en |
| Output | Markdown file | Title + meta + OCR text + description |
Output Format
# {title}
> Author: {author} | {time} | [Original]({url})
{OCR text from image 1}
{OCR text from image 2}
...
---
{post description, if present}
Lessons Learned
| Date | Lesson | Action |
|---|---|---|