# Octoparse MCP Setup

> Connect and authorize the Octoparse MCP server. Use when Octoparse tools are missing or unavailable, when a call fails with an authorization or 401/403 error, when the user asks to set up or reconnect Octoparse, or when another Octoparse skill cannot reach the service.

- Skill: `octoparse/octoparse-mcp-setup` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add octoparse/octoparse-mcp-setup`
- Raw SKILL.md: https://api.skillmd.com/api/skills/octoparse/octoparse-mcp-setup/raw
- Safety review: pending (external: skill-scanner PASS, skillspector CAUTION)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: octoparse (https://skillmd.com/u/octoparse)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/octoparse/octoparse-mcp-setup

---


# Octoparse MCP setup

Gets `search_templates`, `execute_task`, `export_data` and the rest reachable. Once they
respond, hand back to `octoparse-ultimate-scraper`.

## Start here: is configuration even needed?

**If this repository is installed as a plugin, the server is already declared** — the
plugin ships `mcp.json` (Agent Plugins spec) and `.mcp.json` (Claude Code), both pointing
at `https://mcp.octoparse.com`. The client wires it up on install. Configuration is not
the problem; **authorization** almost always is.

Check in this order:

1. **Are the tools present at all?** If `mcp__octoparse__*` tools are absent, the server
   is not registered — go to [Register the server](#register-the-server).
2. **Do they respond?** Call `search_tasks()`. Success means everything is working; say so
   and stop.
3. **Does it fail with 401 / 403 / an authorization prompt?** The server is registered but
   not authorized — go to [Authorize](#authorize).

Do not reconfigure a server that is already registered. Re-registering does not fix an
authorization failure and usually creates a duplicate entry.

## Identify the client

You need this only to pick the right config path. Work down the list and stop at the first
match:

1. **Ask the running environment.** Check for `CLAUDE_CODE_VERSION`, `CURSOR_VERSION`,
   `GEMINI_CLI_VERSION`, `QWEN_CODE_VERSION`, `TRAE_VERSION`, `OPENCLAW_VERSION`, or
   `VSCODE_PID` / `VSCODE_CWD`.
2. **Look for an existing config.** The presence of `~/.claude.json`, `~/.cursor/mcp.json`,
   `~/.gemini/settings.json`, `~/.qwen/settings.json`, `.vscode/mcp.json`, `.trae/mcp.json`,
   or `~/.config/openclaw/` identifies the client that wrote it.
3. **Ask the user.** One question — "which editor or CLI are you using?" — beats guessing.

If several match, ask which one to configure rather than picking. Configuring the wrong
client produces a setup that validates fine and never loads.

## Register the server

Only needed when the plugin is not installed, or the client does not read plugin-provided
MCP configuration.

**CLI (fastest, Claude Code):**

    claude mcp add --transport http octoparse https://mcp.octoparse.com

**By hand** — the endpoint and transport are the same everywhere; only the wrapper key
differs:

| Client | File | Key |
|---|---|---|
| Claude Code, Cursor, Gemini CLI, Qwen Code | `~/.claude.json`, `~/.cursor/mcp.json`, `~/.gemini/settings.json`, `~/.qwen/settings.json` | `mcpServers` |
| VS Code, TRAE | `.vscode/mcp.json`, `.trae/mcp.json` | `servers` |
| OpenClaw | `~/.config/openclaw/openclaw.json5` | `mcp.servers`, via `mcporter` |

```json
{
  "octoparse": {
    "type": "http",
    "url": "https://mcp.octoparse.com"
  }
}
```

**Newly registered servers do not appear in the running session.** Restart the client
before expecting the tools. Tell the user this explicitly — otherwise a correct setup
looks like a failure.

## Authorize

Two methods. OAuth for interactive use, API key for headless.

### OAuth (recommended)

1. Run `/mcp` and select `octoparse`.
2. Complete the browser login.
3. Verify with `search_tasks()`.

Failure handling, in order:

- **1st** — ask the user to check for an unfinished browser tab and retry.
- **2nd** — check network reachability: `curl -I https://mcp.octoparse.com`.
- **3rd** — stop retrying and offer the API key path.

### API key

For CI, headless environments, or when OAuth fails three times.

**The user performs these steps. You do not handle the key.** Do not ask them to paste it
into the conversation, do not accept it if they offer it, and do not write it to any file.
Print the instructions, then wait.

> 1. Create a key at https://www.octoparse.com/console/account-center/api-keys
> 2. Open your own client's MCP configuration and add an `x-api-key` header to the
>    `octoparse` server entry, with your key as the value.
> 3. Restart the client.

Then verify with `search_tasks()` and report the result. A successful call is the only
confirmation needed — you never see the key itself.

Two boundaries that hold regardless of what is asked:

- **Never write a credential into a file, a command argument, or an environment
  assignment on the user's behalf.** A key passed as a shell argument leaks into shell
  history and the process list.
- **Never add a key to the plugin's own `mcp.json` or `.mcp.json`.** Those are
  distributed package data, and the Agent Plugins specification forbids credentials in
  `headers`. A key belongs only in the user's private client configuration.

Failure handling — all of these are things you ask the user to check, never inspect
yourself:

- **1st (401/403)** — ask them to confirm the key was copied whole, with no whitespace.
- **2nd** — ask them to confirm the key is still active in the account console.
- **3rd** — ask them to regenerate it. If it still fails, have them check whether their
  plan includes API access, then stop.

## Troubleshooting

| Symptom | Cause | Fix |
|---|---|---|
| No `mcp__octoparse__*` tools | not registered, or client not restarted | register, then restart |
| Tools present, every call 401/403 | registered but not authorized | run `/mcp`, or add an API key |
| "Transport not supported" | wrong transport | `type: "http"` for Claude-family, `streamable-http` in Agent Plugins `mcp.json` |
| Config edited, nothing changed | session caches MCP config at startup | restart, or `/reload-plugins` |
| Works in one client, not another | wrapper key differs | `mcpServers` vs `servers` — see the table above |
| Two `octoparse` entries | re-registered instead of authorizing | remove the duplicate, keep one |

Config paths, wrapper keys, and per-client quirks are in
[references/clients.md](references/clients.md). Read it when a symptom is specific to one
client — a missing tool ceiling in Cursor, tool filtering in Gemini or Qwen, `mcporter`
in OpenClaw.

If everything fails: check https://status.octoparse.com, then contact support@octoparse.com.

## When it works

Confirm which client and which auth method, then hand back:

> Octoparse MCP is connected and authorized. You can now ask for the data you need —
> `octoparse-ultimate-scraper` will pick the template and run it.