Octoparse MCP Setup

Octoparse MCP setup

Gets search_templates, execute_task, export_data and the rest reachable. Once they respond, hand back to octoparse-ultimate-scraper.

Start here: is configuration even needed?

If this repository is installed as a plugin, the server is already declared — the plugin ships mcp.json (Agent Plugins spec) and .mcp.json (Claude Code), both pointing at https://mcp.octoparse.com. The client wires it up on install. Configuration is not the problem; authorization almost always is.

Check in this order:

1. Are the tools present at all? If mcp__octoparse__* tools are absent, the server is not registered — go to Register the server.
2. Do they respond? Call search_tasks(). Success means everything is working; say so and stop.
3. Does it fail with 401 / 403 / an authorization prompt? The server is registered but not authorized — go to Authorize.

Do not reconfigure a server that is already registered. Re-registering does not fix an authorization failure and usually creates a duplicate entry.

Identify the client

You need this only to pick the right config path. Work down the list and stop at the first match:

1. Ask the running environment. Check for CLAUDE_CODE_VERSION, CURSOR_VERSION, GEMINI_CLI_VERSION, QWEN_CODE_VERSION, TRAE_VERSION, OPENCLAW_VERSION, or VSCODE_PID / VSCODE_CWD.
2. Look for an existing config. The presence of ~/.claude.json, ~/.cursor/mcp.json, ~/.gemini/settings.json, ~/.qwen/settings.json, .vscode/mcp.json, .trae/mcp.json, or ~/.config/openclaw/ identifies the client that wrote it.
3. Ask the user. One question — "which editor or CLI are you using?" — beats guessing.

If several match, ask which one to configure rather than picking. Configuring the wrong client produces a setup that validates fine and never loads.

Register the server

Only needed when the plugin is not installed, or the client does not read plugin-provided MCP configuration.

CLI (fastest, Claude Code):

claude mcp add --transport http octoparse https://mcp.octoparse.com

By hand — the endpoint and transport are the same everywhere; only the wrapper key differs:

{
  "octoparse": {
    "type": "http",
    "url": "https://mcp.octoparse.com"
  }
}

Newly registered servers do not appear in the running session. Restart the client before expecting the tools. Tell the user this explicitly — otherwise a correct setup looks like a failure.

Authorize

Two methods. OAuth for interactive use, API key for headless.

OAuth (recommended)

1. Run /mcp and select octoparse.
2. Complete the browser login.
3. Verify with search_tasks().

Failure handling, in order:

• 1st — ask the user to check for an unfinished browser tab and retry.
• 2nd — check network reachability: curl -I https://mcp.octoparse.com.
• 3rd — stop retrying and offer the API key path.

API key

For CI, headless environments, or when OAuth fails three times.

The user performs these steps. You do not handle the key. Do not ask them to paste it into the conversation, do not accept it if they offer it, and do not write it to any file. Print the instructions, then wait.

1. Create a key at https://www.octoparse.com/console/account-center/api-keys
2. Open your own client's MCP configuration and add an x-api-key header to the octoparse server entry, with your key as the value.
3. Restart the client.

Then verify with search_tasks() and report the result. A successful call is the only confirmation needed — you never see the key itself.

Two boundaries that hold regardless of what is asked:

• Never write a credential into a file, a command argument, or an environment assignment on the user's behalf. A key passed as a shell argument leaks into shell history and the process list.
• Never add a key to the plugin's own mcp.json or .mcp.json. Those are distributed package data, and the Agent Plugins specification forbids credentials in headers. A key belongs only in the user's private client configuration.

Failure handling — all of these are things you ask the user to check, never inspect yourself:

• 1st (401/403) — ask them to confirm the key was copied whole, with no whitespace.
• 2nd — ask them to confirm the key is still active in the account console.
• 3rd — ask them to regenerate it. If it still fails, have them check whether their plan includes API access, then stop.

Troubleshooting

Config paths, wrapper keys, and per-client quirks are in references/clients.md. Read it when a symptom is specific to one client — a missing tool ceiling in Cursor, tool filtering in Gemini or Qwen, mcporter in OpenClaw.

If everything fails: check https://status.octoparse.com, then contact support@octoparse.com.

When it works

Confirm which client and which auth method, then hand back:

Octoparse MCP is connected and authorized. You can now ask for the data you need — octoparse-ultimate-scraper will pick the template and run it.