# Mianshiya Scraper

> Export mianshiya.com (面试鸭) interview question pages to standard Markdown with local images. Use when Codex needs to fetch a 面试鸭 question/article URL, handle the site's login and anti-bot flow through an automated browser, preserve authenticated cookies/storage state, download embedded images, and generate an Obsidian-friendly or plain Markdown file from the full page content.

- Skill: `xicode-ai/mianshiya-scraper` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add xicode-ai/mianshiya-scraper`
- Raw SKILL.md: https://api.skillmd.com/api/skills/xicode-ai/mianshiya-scraper/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Docs & Writing
- Author: xicode-ai (https://skillmd.com/u/xicode-ai)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/xicode-ai/mianshiya-scraper

---


# Mianshiya Scraper

## Overview

Use this skill to turn a 面试鸭 question page such as `https://www.mianshiya.com/bank/.../question/...` into a local Markdown note plus local images. The bundled script uses Playwright so the user can complete QR-code login in a real browser window before content extraction.

## Quick Start

Run:

```bash
python /Users/louis/.agents/skills/mianshiya-scraper/scripts/export_mianshiya.py \
  "https://www.mianshiya.com/bank/1991433644421414913/question/1991796895479013377" \
  --output-dir "/path/to/output"
```

Default behavior:

- Launch Chromium with a persistent profile at `~/.cache/mianshiya-scraper/profile`.
- Let the user scan and finish login if no valid session exists.
- Save authenticated browser state under `~/.cache/mianshiya-scraper/storage-state.json`.
- Extract the article/question content from the rendered DOM.
- Wait for answer/body Markdown content to appear before exporting.
- Download images into `<output-dir>/assets/` by default.
- Write `<output-dir>/<question title>.md` with title, source URL, export timestamp, and Markdown content. Preserve the readable interview-question title, including Chinese text and numbering; sanitize only path-unsafe characters.

## Workflow

1. Ask for or infer the target URL and output directory. Default the output directory to the current workspace when the user does not specify one.
2. Run `scripts/export_mianshiya.py` with the URL. Use `--headed` when the user likely needs to login; this is the default.
3. If the browser opens to a login or partially locked page, tell the user to scan the QR code or complete the site's login flow in that browser window. Continue after the script detects authenticated answer/body content.
4. Inspect the generated `.md` file briefly. Confirm the filename is based on the interview-question title and image links are relative to the shared `assets/` folder.
5. If extraction is incomplete, rerun with `--debug-html` and inspect the saved HTML snapshot. Update the script's content selector heuristics rather than manually copying site content.

## Script Notes

- Prefer the bundled script over ad hoc scraping. 面试鸭 has login and anti-bot checks, so authenticated browser rendering is more reliable than direct HTTP requests.
- Do not ask the user for passwords or tokens. Let them authenticate in the browser and reuse only the local Playwright profile/storage state.
- Use `--force-login` when the saved session is stale.
- Use `--slug "custom-name"` only when the user explicitly wants to override the title-based output filename.
- Use `--assets-dir "custom-assets"` only when the user explicitly wants a different image directory. The default should remain `assets`.
- Use `--no-images` only when the user explicitly wants text-only output.
- Use `--allow-empty-answer` only when the user explicitly accepts exporting a question shell without unlocked answer/body content.
- If Playwright browsers are missing and installation requires network access, request approval before running the install command.

## Output Standard

Generate Markdown with:

- YAML frontmatter: `title`, `source`, `exported_at`, `site`.
- One `# Title` heading.
- Clean paragraphs, headings, lists, code blocks, blockquotes, tables where possible.
- Markdown filename should be the question title, e.g. `417. Java 中 Exception 和 Error 有什么区别？.md`, not `question-<id>.md`.
- Images as relative Markdown links under the shared assets directory, e.g. `![alt](assets/image-abc123.png)`.
- No navigation, ads, login prompts, duplicated buttons, or unrelated sidebar content.

## Resources

- `scripts/export_mianshiya.py`: browser login, extraction, image download, Markdown export.
- `references/extraction.md`: selector and troubleshooting notes for future maintenance.