# Undermind Search > Stage 1 of the lit review pipeline: run an Undermind deep search from the natural-language brief produced by Stage 0 through the Undermind MCP server (agent-driven; no browser, no stored credentials), then parse + enrich the returned papers into the pipeline schema. Only use this skill when explicitly requested. Do NOT auto-trigger on general literature review or paper search requests. - Skill: `kennethkhoocy/undermind-search` (Agent Skill, multi-file: 3 files) - Install (CLI): `npx skillmds@latest add kennethkhoocy/undermind-search` - Raw SKILL.md: https://api.skillmd.com/api/skills/kennethkhoocy/undermind-search/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: DevOps & Infra - Author: kennethkhoocy (https://skillmd.com/u/kennethkhoocy) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/kennethkhoocy/undermind-search --- # Undermind Search (Stage 1) Takes the **Undermind brief** that Stage 0 extracts and returns the papers Undermind's deep search finds, enriched and saved as `.json` + `.bib` for dedup and screening. The stage is agent-driven end to end. There is no subprocess driver: the host agent calls the Undermind MCP tools, saves what they return, and one script normalizes it. - **The agent** — `launch_deep_search` → poll `inspect_deep_searches` → page the results. - **`scripts/undermind_harvest.py`** — copies the paged tool results out of the session transcript byte-for-byte into `OUT/undermind_page_.txt` (Claude Code `~/.claude/projects//.jsonl`, following its `` pointers; Codex `~/.codex/sessions/**/rollout-*.jsonl`). - **`scripts/undermind_ingest.py`** — parses those files (or any RIS/BibTeX export), fills missing DOIs (Crossref) and abstracts/journals (OpenAlex), and writes the pipeline JSON. Importable, and runnable standalone. ## Prerequisite: the Undermind MCP server Undermind's official MCP endpoint is `https://mcp.undermind.ai/mcp` (OAuth, your own Undermind account). - **Claude Code:** `claude mcp add --transport http undermind https://mcp.undermind.ai/mcp` then `claude mcp login undermind` in an interactive terminal. A session that was already open when you logged in only exposes `mcp__undermind__authenticate`; call it once, approve in the browser, and the full tool set swaps in. - **Codex:** in `~/.codex/config.toml`: ```toml [mcp_servers.undermind] url = "https://mcp.undermind.ai/mcp" [mcp_servers.undermind.oauth] client_id = "codex" ``` Codex completes the OAuth flow on first use. Call `get_orientation` once per conversation before the other tools. ## The flow ``` brief ─► create_workspace (or reuse) ─► launch_deep_search(goal=brief) ─► poll inspect_deep_searches(status_only) every ~60 s (2–5 min) ─► inspect_deep_searches(papers_only, detail_level="full", limit=50, offset=0/50/…) ─► undermind_harvest.py --latest --match --out-dir OUT (pages → OUT/undermind_page_.txt) ─► undermind_ingest.py --input OUT/undermind_page_*.txt -o OUT/stage1_undermind.json ``` Step by step, with `OUT` the run's output directory: 1. `list_workspaces` → reuse a workspace for this manuscript, or `create_workspace(title=)`. Keep the `workspace_id`. 2. `launch_deep_search(workspace_id, goal=, name=)`. The tool returns immediately. Launch it **first**, then start the other channels while it runs. The brief goes in verbatim: the `goal` is meant to be a self-contained paragraph, which is exactly what Stage 0 writes (no keyword lists, no journal names). The MCP search asks no clarifying questions; the plan's `undermind_clarifications` field is informational only (a holdover from the browser flow). 3. Poll `inspect_deep_searches(names=[], status_only=true)` from a background Monitor, not a foreground wait. A `rate_limited` result means wait and retry; Undermind runs only a few deep searches concurrently. 4. On completion, page the papers: `inspect_deep_searches(names=[], papers_only=true, detail_level="full", limit=50, offset=0)`, then `offset=50`, `100`, … until a page comes back short. Do not retype or summarize anything. 5. Harvest the pages from the transcript, then ingest: ```bash python undermind-search/scripts/undermind_harvest.py --latest --match "" --out-dir OUT python undermind-search/scripts/undermind_ingest.py --input OUT/undermind_page_*.txt \ -o OUT/stage1_undermind.json ``` The harvester scans the newest transcript under the Claude Code projects dir (`CLAUDE_CONFIG_DIR` or `~/.claude`) and `~/.codex/sessions`, keeps every distinct tool result that contains `[Key] Title (Year)` blocks and whose header (the search-name line above the paper list) contains the `--match` text, and writes them as `undermind_page_01.txt`, `_02`, …; without `--match`, when the transcript holds several searches, only the latest search's pages are kept and the others are named. Pages whose `showing a-b of n` ranges leave a gap raise a WARNING naming the missing offsets. (`--transcript PATH` to name the file; `--min-records N` to drop short fragments). Falling back to saving a page by hand as `.txt` or as the `{"result": "..."}` JSON also works: the ingest reads both. Optional add-ons before ingest, for the same workspace: `search_papers` with `search_type="citations"` / `"references"` seeded on the top cite keys widens coverage; save its output as another page file and ingest it together. ## Input formats the ingest accepts Auto-detected from content: | Input | Notes | |-------|-------| | Raw MCP tool text | `[Key] Title (Year)` blocks from `inspect_deep_searches`, `get_paper_info`, or `search_papers` at any detail level. Journal, `Date:`, `By …`, `DOI:`, `Link:`, `PDF`, and the abstract paragraph are read; trailer legends are ignored. | | `{"result": "…"}` JSON | The tool result saved as JSON. | | JSON array | `[{"title", "authors" (string or list), "year", "journal", "doi", "url", "abstract"}]`; only `title` is required. | | `.ris` / `.bib` | The web app's Export, or any reference manager. | Several `--input` files are concatenated and deduplicated by cite key, DOI, or normalized title, so paginated dumps just go in together. ## CLI `undermind_harvest.py`: | Flag | Default | Description | |------|---------|-------------| | `--latest` | (default) | Scan the most recently modified transcript under the Claude Code / Codex session dirs | | `--transcript PATH` | — | Scan this transcript instead | | `--match TEXT` | — | Keep only pages whose header (search-name line) contains this text; omitted → the latest search in the transcript | | `--min-records N` | `1` | Skip pages with fewer paper blocks | | `--out-dir DIR` | `.` | Where `undermind_page_.txt` is written | | `--selftest` | off | Run the harvester self-check | `undermind_ingest.py`: | Flag | Default | Description | |------|---------|-------------| | `--input, -i PATH…` | — | One or more input files (globs expanded internally; PowerShell-safe) | | `-o, --output PATH` | `stage1_undermind.json` | Output JSON (a `.bib` sibling is written) | | `--sibling {bibtex,ris,none}` | `bibtex` | Reference-manager copy beside the JSON | | `--source TAG` | `undermind` | `source` value stamped on each record | | `--no-enrich` | off | Skip Crossref/OpenAlex enrichment | | `--selftest` | off | Run the parser self-check | ## Graceful degradation If the MCP tools are unavailable (server not added, not authenticated, or a Codex/Claude session without MCP), the agent prints `UNDERMIND_DEFERRED`, writes `[]` to `stage1_undermind.json`, and continues; the ingest does the same when no input file exists or nothing parses. The brief is always available in `undermind_brief.txt` for a manual run in the web app (Export → RIS, then ingest the file). The autonomous `orchestrator.py` never runs this stage: it has no MCP client. ## Output schema Same as the other stages, with `source: "undermind"`: ```json {"title": "...", "authors": "A, B", "year": "2024", "doi": "https://doi.org/10.x/y", "abstract": "...", "journal": "...", "url": "...", "source": "undermind", "verified": true, "citations": 0, "open_access": false} ``` ## Troubleshooting - **Only `mcp__undermind__authenticate` is available** — the session predates the login. Call it, approve in the browser, and the tools appear. - **`claude mcp list` says "Needs authentication" although login worked** — a stale `mcp-needs-auth-cache.json` in the Claude config dir; harmless to delete. - **`rate_limited`** — Undermind caps concurrent deep searches; wait and retry. - **The harvester finds no pages** — the tool results have not been flushed to the transcript yet, or the newest transcript belongs to another session; wait a moment and retry, or pass `--transcript` with the session file. `--match` is case-insensitive and is tested against the page header (the search-name line), never against paper text. - **A page parses to zero records** — the tool output format drifted. Run `--selftest`, compare the saved page with the header pattern in `parse_mcp_text`, and adjust; the JSON-array input is the format-independent fallback.