# Author Research > Deep research for book authors — searches Google Scholar, Internet Archive, JSTOR, Google Books/Ngram, Library of Congress, HathiTrust, OECD/World Bank data, historic newspapers, and news archives. Produces an organized research dossier with primary sources, data, historical context, and full citations. - Skill: `joshuashepherd/author-research` (Agent Skill) - Install (CLI): `npx skillmds@latest add joshuashepherd/author-research` - Raw SKILL.md: https://api.skillmd.com/api/skills/joshuashepherd/author-research/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Research & Search - Author: JoshuaShepherd (https://skillmd.com/u/joshuashepherd) - Updated: 2026-09-08 - Page: https://skillmd.com/skills/joshuashepherd/author-research --- Conduct deep research for a book or long-form writing project on: $ARGUMENTS $ARGUMENTS can include: - A research topic or question (e.g. `the history of communal movements in America`) - A source-type focus (e.g. `--type primary-sources`, `--type statistics`, `--type historical`, `--type news`) - A time period (e.g. `--era 1960-1980`, `--era victorian`, `--era modern`) - A geographic focus (e.g. `--region europe`, `--region global`) - An output path override (default: `_research/general/`) - A depth flag (e.g. `--depth quick` for survey, `--depth deep` for comprehensive) - Empty — ask the user what they're writing about and what they need --- ## Purpose This skill is built for **authors researching a book or long-form writing project**. It searches across the richest freely available sources for the kind of material that makes great non-fiction: primary documents, historical records, statistical data, expert scholarship, cultural artifacts, and news archives. It produces an organized research dossier — the kind of file an author keeps on their desk throughout the writing process. **What it produces:** - `_research/general/[topic-slug]/README.md` — research overview and key discoveries - `_research/general/[topic-slug]/primary-sources.md` — historical documents, archives, original texts - `_research/general/[topic-slug]/scholarship.md` — key books, papers, and expert perspectives - `_research/general/[topic-slug]/data-and-statistics.md` — quantitative data, tables, trends - `_research/general/[topic-slug]/timeline.md` — chronological development of the topic - `_research/general/[topic-slug]/quotable.md` — compelling quotes, passages, and anecdotes - `_research/general/[topic-slug]/further-leads.md` — promising leads for deeper research - `_research/general/[topic-slug]/source-index.md` — all searches, URLs, retrieval status --- ## Source Library ### 1. Google Scholar (Cross-Disciplinary Academic Search) **What it is:** The broadest academic search engine. Indexes papers, books, theses, court opinions, patents, and technical reports across all disciplines. Often finds material that specialized databases miss. **Web Endpoint:** `https://scholar.google.com/scholar?q={query}` **Search strategy:** ``` # Basic search https://scholar.google.com/scholar?q={query} # With date range https://scholar.google.com/scholar?q={query}&as_ylo=2015&as_yhi=2025 # Exact phrase https://scholar.google.com/scholar?q="{exact phrase}" # Author search https://scholar.google.com/scholar?q=author:"{name}"+{topic} # Exclude patents and citations https://scholar.google.com/scholar?q={query}&as_vis=1 ``` **What to extract:** - Paper/book titles and authors - Citation counts (indicates influence) - "Cited by" links (for finding related work) - "Related articles" links - Snippet text showing relevant passages - Links to free full-text versions (look for [PDF] links) **Usage notes:** - No API — use via web search/fetch - Google Scholar often finds free versions of paywalled papers (preprints, author copies) - Citation count is the best quick proxy for a work's influence - The "Cited by" link for a foundational work is a goldmine — it shows every paper that built on it - Also use web search: `"scholar.google.com" {topic} {additional terms}` ### 2. Internet Archive / Open Library (Digitized Books & Media) **What it is:** Massive digital library of books, periodicals, audio, video, software, and web pages. Open Library provides controlled digital lending of full books. The Wayback Machine archives web pages. **Web Endpoints:** ``` # Search the full archive https://archive.org/search?query={query} # Search books specifically https://archive.org/search?query={query}&and[]=mediatype:texts # Open Library book search https://openlibrary.org/search?q={query} # Open Library search API https://openlibrary.org/search.json?q={query}&limit=20 # Wayback Machine (for archived web pages) https://web.archive.org/web/{url} ``` **What to extract:** - Full-text books in the public domain - Digitized periodicals and magazines - Historical documents and pamphlets - Audio recordings (lectures, interviews, radio) - Book metadata (publication info, subjects, editions) **Usage notes:** - Public domain works are freely downloadable in multiple formats - In-copyright books available via "controlled digital lending" — 1-hour borrow - The Internet Archive's collection of scanned periodicals is extraordinary for historical research - Search within texts is supported for many digitized works - Open Library API returns structured JSON with ISBNs, covers, subjects, and edition data ### 3. JSTOR Open Access **What it is:** Digital library of academic journals, books, and primary sources. Thousands of journals are now in JSTOR's open-access collection, freely available without institutional access. **Web Endpoints:** ``` # Main search https://www.jstor.org/action/doBasicSearch?Query={query} # Open access filter https://www.jstor.org/action/doBasicSearch?Query={query}&acc=oa # Search within a discipline https://www.jstor.org/action/doBasicSearch?Query={query}&disc={discipline_code} ``` **Search strategy:** ``` site:jstor.org "{topic}" open access site:jstor.org "{topic}" {key author} "jstor" "{topic}" free full text ``` **What to extract:** - Journal articles (abstracts always free; full text for open-access items) - Book chapters and reviews - Primary source documents in JSTOR's special collections - Citation metadata **Usage notes:** - JSTOR open access includes thousands of journal titles with full-text access - Even for non-open-access articles, abstracts and first pages are viewable - JSTOR's humanities and social science coverage is particularly deep - Primary Source collections include letters, pamphlets, and historical documents - Some content becomes open access after an embargo period (moving wall) ### 4. Google Books & Google Ngram Viewer (Text Search & Word Trends) **What it is:** Google Books searches inside millions of digitized books with snippet view. Ngram Viewer tracks the frequency of words and phrases across centuries of published text — invaluable for understanding how ideas have evolved. **Web Endpoints:** ``` # Google Books search https://www.google.com/search?tbm=bks&q={query} # Google Books API https://www.googleapis.com/books/v1/volumes?q={query}&maxResults=20 # With subject filter https://www.googleapis.com/books/v1/volumes?q={query}+subject:{subject}&maxResults=20 # Ngram Viewer https://books.google.com/ngrams/json?content={word1,word2}&year_start=1800&year_end=2019&corpus=en-2019&smoothing=3 ``` **What to extract from Google Books:** - Book titles, authors, publication dates, publishers - Snippet views showing relevant passages (search within books) - Subject classifications - Related books - Preview availability **What to extract from Ngram Viewer:** - Word/phrase frequency over time (when did a concept enter the discourse?) - Comparative frequency (which of two related terms dominates?) - Inflection points (when did usage spike or drop?) - Historical context for the evolution of ideas and terminology **Usage notes:** - Google Books snippet view often reveals the most relevant passages in a book — great for deciding what to acquire and read - Ngram data covers 1500-2019 across multiple language corpora - Ngram is phenomenal for book introductions — "The term X first appeared in the 1840s and peaked in usage around..." - The API returns structured JSON with ISBNs, descriptions, categories, and preview links - Use Ngram case-insensitive smoothing=3 for cleaner trend lines ### 5. Library of Congress Digital Collections (Primary Sources) **What it is:** The world's largest library, with massive free digital collections: manuscripts, maps, photographs, newspapers, audio recordings, legislation, and more. **Web Endpoints:** ``` # Main digital collections search https://www.loc.gov/search/?q={query}&fa=online-format:online+text # Collections by format https://www.loc.gov/collections/ # Chronicling America (historic newspapers 1777-1963) https://chroniclingamerica.loc.gov/search/pages/results/?andtext={query}&dateFilterType=yearRange&date1=1800&date2=1963 # Prints & Photographs https://www.loc.gov/pictures/search/?q={query} # Maps https://www.loc.gov/maps/?q={query} # Manuscripts https://www.loc.gov/manuscripts/?q={query} # Congress.gov (legislation) https://www.congress.gov/search?q={query} ``` **What to extract:** - Primary source documents (letters, manuscripts, speeches) - Historical photographs with captions and dates - Historic newspaper articles and editorials - Maps and cartographic materials - Legislative records and congressional documents - Audio recordings (speeches, oral histories, music) **Usage notes:** - Everything in the digital collections is **free and public** - Chronicling America is extraordinary for historical research — full-text searchable newspapers from 1777-1963 - The Prints & Photographs division has millions of images with detailed cataloging - Manuscript collections include papers of presidents, scientists, and cultural figures - LOC subject headings are standardized — learning them helps with precise searches - Digital collections are continuously growing as more material is digitized ### 6. HathiTrust Digital Library (Research Library Collections) **What it is:** Partnership of major research libraries. 17M+ digitized volumes. Full-text search across the entire corpus. Public domain works are fully accessible; in-copyright works are searchable with snippet results. **Web Endpoints:** ``` # Full-text search https://babel.hathitrust.org/cgi/ls?q1={query}&field1=ocr&a=srchls # Catalog search https://catalog.hathitrust.org/Search/Home?lookfor={query}&type=all # Advanced search with date https://babel.hathitrust.org/cgi/ls?q1={query}&field1=ocr&yop=after&year=1800&yop2=before&year2=1950 ``` **What to extract:** - Full text of public domain books (pre-1929 US publications) - Snippet results from in-copyright books (showing relevant passages) - Rare and out-of-print books not available elsewhere - Serial publications and periodicals - Government documents and reports **Usage notes:** - Public domain works are fully readable and downloadable as PDF - For in-copyright works, full-text search returns snippet results (like Google Books but different corpus) - HathiTrust often has books that Google Books doesn't and vice versa — search both - Particularly strong for 19th and early 20th century materials - "Emergency Temporary Access" during certain periods allows broader borrowing - The research center tools allow computational analysis of the full corpus ### 7. OECD Data & World Bank Open Data (Global Statistics) **What it is:** The two most comprehensive free sources for international economic, social, and development statistics. OECD covers developed economies; World Bank covers all countries with emphasis on development indicators. **Web Endpoints:** ``` # OECD Data https://data.oecd.org/searchresults/?q={query} https://stats.oecd.org/ # World Bank Open Data https://data.worldbank.org/indicator?tab=all https://api.worldbank.org/v2/country/all/indicator/{indicator_code}?format=json&per_page=100 # World Bank search https://datacatalog.worldbank.org/search?q={query} # Common World Bank indicators: # NY.GDP.PCAP.CD - GDP per capita # SP.POP.TOTL - Population # SE.ADT.LITR.ZS - Literacy rate # IT.NET.USER.ZS - Internet users (% of population) # SL.UEM.TOTL.ZS - Unemployment rate ``` **What to extract:** - Country-level statistics across hundreds of indicators - Time series data (many indicators tracked annually since 1960+) - Cross-country comparison tables - Development indicators (poverty, health, education, infrastructure) - Economic data (GDP, trade, employment, inflation) - Technology adoption metrics (internet access, mobile subscriptions) **Usage notes:** - Both are **completely free** with downloadable datasets - World Bank API returns structured JSON — easy to extract specific data - OECD data is particularly strong on education, labor, health, and economic policy - World Bank data is particularly strong on development, poverty, and low/middle-income countries - Both offer visualization tools on their websites - Data can be cited as: "World Bank, World Development Indicators, [year]" ### 8. Chronicling America + News Archives (Historical & Modern News) **What it is:** Chronicling America (Library of Congress) provides digitized US newspapers from 1777-1963. For modern news, web search and news aggregators provide coverage. **Web Endpoints:** ``` # Chronicling America — historic newspapers https://chroniclingamerica.loc.gov/search/pages/results/?andtext={query}&dateFilterType=yearRange&date1={start_year}&date2={end_year} # Chronicling America — browse by state/title https://chroniclingamerica.loc.gov/newspapers/ # Modern news via web search Search: "{topic}" site:nytimes.com OR site:washingtonpost.com OR site:reuters.com OR site:apnews.com ``` **What to extract:** - Historical newspaper articles (full text + page images) - Contemporary accounts of historical events - Editorials and opinion pieces showing period attitudes - Advertisements (social history gold) - Modern news coverage of your topic **Usage notes:** - Chronicling America is **completely free** — full page images and OCR text - OCR quality varies — try multiple search term spellings for older papers - Coverage is not uniform — some states/periods have much better coverage than others - For modern news, respect paywalls — note when an article is behind a paywall and cite it without copying full text - News archives are crucial for establishing "what people were saying at the time" ### 9. Google News & News Aggregation (Current Coverage) **What it is:** For understanding the current media landscape around a topic — how it's being covered, by whom, and from what angles. **Search strategy:** ``` # Google News search "{topic}" site:reuters.com OR site:apnews.com (for wire services — factual baseline) "{topic}" site:theatlantic.com OR site:newyorker.com (for long-form analysis) "{topic}" site:ft.com OR site:economist.com (for economic/business angle) "{topic}" 2024 OR 2025 feature OR investigation OR analysis (for deep reporting) ``` **What to extract:** - Key narratives and frames being used - Expert sources quoted (potential interviewees or citation sources) - Data points and statistics cited - Contrasting perspectives and debates - Timeline of how coverage has evolved **Usage notes:** - Use news as a **pointer to deeper sources** — journalists cite reports, studies, and experts - Track which experts are repeatedly quoted on a topic — they're the authorities to cite - News coverage reveals the public framing of issues — useful for understanding audience context - Be aware of publication perspective — note editorial leanings when relevant --- ## Research Process ### Phase 1 — Research Brief Before searching, establish: 1. **What is being written?** — Book, chapter, article, essay? What's the thesis or narrative arc? 2. **What kind of material is needed?** - Primary sources (original documents, speeches, data) - Secondary sources (scholarship, analysis, criticism) - Statistical data (numbers, trends, comparisons) - Historical context (what was happening at the time?) - Contemporary voices (news, opinion, cultural artifacts) - Quotable material (passages, anecdotes, vivid details) 3. **Time period** — What era(s) are relevant? 4. **Geographic scope** — What regions matter? 5. **Search terms** — Generate 5+ query variations across different framings State the research brief before executing. ### Phase 2 — Multi-Source Search Search strategically across sources based on what's needed: **For historical topics:** 1. Library of Congress (primary sources, newspapers, photographs) 2. HathiTrust (older books, out-of-print material) 3. Internet Archive (digitized books, periodicals) 4. Chronicling America (newspaper accounts) 5. Google Books Ngram (how terminology evolved) 6. Google Scholar (modern scholarship about the historical topic) **For contemporary topics:** 1. Google Scholar (academic analysis) 2. OECD / World Bank (statistical data) 3. Google News (current coverage and expert sources) 4. JSTOR (deeper scholarly analysis) 5. Google Books (recent books on the topic) **For cross-cultural or global topics:** 1. World Bank / OECD (comparative data) 2. Google Scholar (cross-cultural studies) 3. Internet Archive (international publications) 4. News search across international outlets For each source: - Run multiple query variations - Prioritize by relevance and credibility - Extract key data, quotes, and findings - Log everything in the source index ### Phase 3 — Material Classification Organize all found material into categories: **Primary Sources** — Original documents, data, firsthand accounts - What: Letters, speeches, legislation, data sets, photographs, recordings - Value: Irrefutable evidence; lets the reader encounter the past/topic directly - Citation standard: Full archival citation with collection, box/folder if applicable **Scholarly Sources** — Expert analysis and interpretation - What: Peer-reviewed papers, academic books, dissertations - Value: Establishes credibility; shows you've engaged with expert thinking - Citation standard: Author, title, journal/publisher, year, DOI if available **Statistical Sources** — Quantitative evidence - What: Data tables, trend charts, survey results, economic indicators - Value: Makes abstract claims concrete; enables comparison - Citation standard: Organization, dataset name, indicator, year(s), URL **Narrative Sources** — Stories, anecdotes, vivid details - What: News articles, personal accounts, cultural artifacts - Value: Makes the writing come alive; connects abstract ideas to human experience - Citation standard: Author, title, publication, date, URL ### Phase 4 — Timeline Construction Build a chronological timeline of the topic: ```markdown ## Timeline: [Topic] | Year | Event | Significance | Source | |------|-------|-------------|--------| | [year] | [what happened] | [why it matters] | [source] | ``` Include: - Key events and milestones - Publication dates of influential works - Policy/legislative changes - Cultural moments (when the topic entered public consciousness) - Data inflection points (when trends shifted) ### Phase 5 — Quotable Material Extract the most compelling quotes and passages: ```markdown ## Quotable Passages ### On [sub-theme] > "[Exact quote]" > — [Author], *[Source]*, [Year], p. [page if known] **Context:** [Why this quote matters; how it could be used in the book] ``` Look for: - Pithy definitions or framings - Surprising or counterintuitive statements from experts - Vivid descriptions or analogies - Historical figures' own words on the topic - Data points that tell a story in one sentence ### Phase 6 — Further Leads Document promising avenues for deeper research: ```markdown ## Further Leads ### Archives to Explore - [Archive name] at [Institution] — [what's there and why it matters] ### People to Interview / Read - [Name] — [who they are, why they matter, where to find their work] ### Books to Acquire and Read - [Title] by [Author] ([Year]) — [why it's relevant, citation count or reputation] ### Datasets to Download - [Dataset] from [Source] — [what it contains, URL] ### Conferences / Proceedings - [Conference name] — [relevant proceedings or keynotes] ``` ### Phase 7 — Output Generation Write all output files with clean markdown formatting. **README.md structure:** ```markdown # Research Dossier: [Topic] **Date:** [YYYY-MM-DD] **Research context:** [What's being written and why] **Scope:** [time period, geography, disciplines] **Sources searched:** [N databases, N individual sources found] ## Top Discoveries [5-7 most valuable finds — the material that will most directly serve the writing] ## Research Landscape [Brief overview: who studies this, what's the state of knowledge, where are the debates?] ## Source Quality Assessment [Which sources were richest? Where are the gaps? What might require library/archive visits?] ## Files in This Dossier - [primary-sources.md](primary-sources.md) — Historical documents and original materials - [scholarship.md](scholarship.md) — Academic papers, books, and expert analysis - [data-and-statistics.md](data-and-statistics.md) — Quantitative data and trends - [timeline.md](timeline.md) — Chronological development - [quotable.md](quotable.md) — Compelling quotes and passages - [further-leads.md](further-leads.md) — Promising avenues for deeper research - [source-index.md](source-index.md) — All searches and sources ``` --- ## Key Rules - **No fabricated sources** — Never invent book titles, article titles, authors, dates, or quotes. Every item must trace to a real search result. "I couldn't find X" is always better than a hallucinated citation. - **Primary sources are gold** — Always prioritize original documents over secondary accounts. An author's own words, the original data, the actual legislation — these are what make great non-fiction. - **Source everything** — Every fact, quote, and data point gets a citation. When writing for an author, the citation trail is as important as the content. - **Note access levels** — Clearly distinguish between freely available material, snippet-only results, and paywalled content. The author needs to know what they can access immediately vs. what requires library visits or purchases. - **Ngram data tells a story** — When relevant, include Google Ngram data showing how key terms have risen and fallen in usage. This is remarkable material for book introductions and historical framing. - **The "further leads" file is crucial** — An author doesn't expect AI to do ALL the research. The highest-value output is often the leads that point toward rich material the author can pursue: specific archives, specific people to talk to, specific books to read deeply. - **Quotable material must be exact** — Only include quotes you've actually found in a fetched source. Never reconstruct or paraphrase a quote and present it in quotation marks. - **Source index is mandatory** — Every search, API call, and URL fetch must be logged in `source-index.md` with result counts and retrieval status. - **Think like a research assistant** — The author needs material that is *usable*: properly cited, well-organized, and contextualized. A pile of links is not research. Organized, annotated material with clear citations is research.