Database Lookup

Database Lookup

You have access to 78 public databases through their REST APIs. Your job is to figure out which database(s) are relevant to the user's question, query them, and return the raw JSON results along with which databases you used.

Core Workflow

1. Understand the query — What is the user looking for? A compound? A gene? A pathway? A patent? Expression data? An economic indicator? This determines which database(s) to hit.

2. Select database(s) — Use the database selection guide below. When in doubt, search multiple databases — it's better to cast a wide net than to miss relevant data.

3. Read the reference file — Each database has a reference file in references/ with endpoint details, query formats, and example calls. Read the relevant file(s) before making API calls.

4. Make the API call(s) — See the Making API Calls section below for which HTTP fetch tool to use on your platform.

5. Return results — Always return:

   • The raw JSON response from each database
   • A list of databases queried with the specific endpoints used
   • If a query returned no results, say so explicitly rather than omitting it

Database Selection Guide

Match the user's intent to the right database(s). Many queries benefit from hitting multiple databases.

Physics & Astronomy

Earth & Environmental Sciences

Chemistry & Drugs

Materials Science & Crystallography

Biology & Genomics

Organism/species matters. Most biology databases cover multiple organisms. If the user's query is about a specific organism, pass it explicitly — don't assume human. Common patterns: Ensembl uses {species} in the URL path (e.g. homo_sapiens), STRING/BioGRID/QuickGO use NCBI taxon IDs (species=9606 for human, 10090 for mouse), UniProt uses organism_id:9606 in search queries, KEGG uses organism codes (hsa, mmu). GTEx and Human Protein Atlas are human-only. Check the reference file for each database's specific parameter.

Disease & Clinical

Patents & Regulatory

Economics & Finance

Social Sciences & Demographics

Cross-domain queries

When the user's query spans multiple domains (e.g. "what do we know about aspirin" or "find everything about BRCA1"), query all relevant databases in parallel.

Common Identifier Formats

Different databases use different identifier systems. If a query fails, the identifier format may be wrong. Here's a quick reference:

Identifier Resolution

When a database doesn't recognize an identifier, convert it using these workflows:

Genes: Symbol (e.g. "TP53") → look up in NCBI Gene (esearch by symbol) → get NCBI Gene ID → convert to Ensembl ID via Ensembl /xrefs/symbol/homo_sapiens/{symbol}, or to UniProt accession via UniProt search (gene_exact:{symbol} AND organism_id:9606).

Compounds: Name → PubChem /compound/name/{name}/cids/JSON → get CID → convert to ChEMBL ID via UniChem or ChEMBL molecule search. If name lookup fails, try SMILES, InChIKey, or CAS number.

Variants: rsID (e.g. "rs334") works directly in dbSNP, ClinVar, GWAS Catalog, gnomAD. For genomic coordinates, use Ensembl VEP to get consequence annotations and linked rsIDs.

Diseases: Name → Open Targets or Monarch search → get EFO or MONDO ID → use in downstream queries.

POST-Only APIs

These databases require HTTP POST and will not work with WebFetch (GET-only). Use curl via your platform's shell tool instead:

API Keys and Access Restrictions

Some databases require API keys or have access restrictions. When an API key is needed:

1. Check the current environment first — the key may already be exported as a shell environment variable (e.g. $FRED_API_KEY). Read it directly from the environment.
2. Fall back to .env — if the variable isn't in the environment, check the .env file in the current working directory.
3. If neither has it — proceed without the key (most APIs still work at lower rate limits) and tell the user which key is missing and how to get one.

Databases requiring API keys (free registration)

These are all free to obtain. The APIs work without keys but have lower rate limits. Always try with a key first — if the env variable isn't set, proceed without the key and note in your response that rate limits may be lower.

Databases with paid or restricted access

When a database requires paid access or registration the user hasn't set up:

1. Fall back to a free alternative that can answer the same question
2. Tell the user which database you couldn't access, why, and what you used instead
3. If the user specifically requests a restricted database, explain the access requirements so they can set it up

Loading API keys

Step 1 — Check the current environment. The key may already be exported as a shell variable. For example, in Claude Code you can check with Bash: echo $FRED_API_KEY. If the variable is set and non-empty, use it.

Step 2 — Check .env file. If the environment variable isn't set, read .env from the current working directory. Format:

FRED_API_KEY=your_key_here
BEA_API_KEY=your_key_here

Step 3 — Proceed without. If neither source has the key, proceed without it (most APIs still work at lower rate limits) and mention this to the user.

Making API Calls

Use your environment's HTTP fetch tool to call REST endpoints. The tool name varies by platform:

If you don't recognize your platform or the fetch tool fails, fall back to curl via whatever shell/terminal tool is available. Example:

curl -s -H "Accept: application/json" "https://api.example.com/endpoint"

Request guidelines

• Set Accept: application/json header where supported
• URL-encode special characters in query parameters — SMILES strings (/, #, =, @), compound names with parentheses, and ontology terms with colons (HP:0001250 → HP%3A0001250) are common sources of failures. With curl, use --data-urlencode for safety.
• Parallel OK: When querying different databases (e.g., PubChem + ChEMBL + Reactome), run them in parallel — most APIs have generous rate limits.
• Serialize requests to rate-limited APIs: NCBI APIs (Gene, GEO, Protein, Taxonomy, dbSNP, SRA) at 3 req/sec without key, 10 with key. Also watch: Ensembl (15 req/sec), BLS v1 (25 req/day without key), SEC EDGAR (10 req/sec), NOAA (5 req/sec with token).
• If you get a rate-limit error (HTTP 429 or 503), wait briefly and retry once

Error recovery

If an API returns an error or empty results:

1. Check the identifier format — use the Common Identifier Formats table above. A gene symbol may need to be converted to NCBI Gene ID or Ensembl ID first.
2. Try alternative identifiers — if a compound name fails in PubChem, try SMILES, InChIKey, or CID. If a gene symbol fails, try the NCBI Gene ID.
3. Try a different database — if one database is down or returns nothing, check the "Also consider" column in the selection guide for alternatives.
4. Report the failure — tell the user which database failed, the error, and what you tried instead.

Pagination

Many APIs return paginated results — if you only read the first page, you may miss data. Common patterns:

• Offset/Limit: offset=0&limit=100 → increment offset by limit for the next page (ChEMBL, FRED, NOAA, USGS, NCBI E-utilities, ENA, GDC, FDA)
• Cursor-based: Response includes a nextPageToken or cursor value — pass it in the next request (ClinicalTrials.gov, UniProt)
• Page number: page=1&per_page=50 → increment page (World Bank, cBioPortal, ZINC)

Check the reference file for each database's specific pagination parameters. If a response includes total, totalCount, or next and the number of returned results is less than the total, there are more pages.

For targeted lookups (single gene, single compound), the first page is usually sufficient. Paginate when the user needs comprehensive results (e.g., "all clinical trials for X" or "all known variants in gene Y").

Output Format

Structure your response like this:

## Databases Queried
- **PubChem** — /compound/name/aspirin/property/...
- **Reactome** — /search/query?query=aspirin

## Results

### PubChem
[raw JSON response]

### Reactome
[raw JSON response]

If results are very large, present the most relevant portion and note that additional data is available. But default to showing the full raw JSON — the user asked for it.

Adding New Databases

This skill is designed to grow. Each database is a self-contained reference file in references/. To add a new database:

1. Create references/<database-name>.md following the same format as existing files
2. Add an entry to the database selection guide above
3. The reference file should include: base URL, key endpoints, query parameter formats, example calls, rate limits, and response structure

Available Databases

Read the relevant reference file before making any API call.

Physics & Astronomy

Earth & Environmental Sciences

Chemistry & Drugs

Materials Science

Biology & Genomics

Disease & Clinical

Patents & Regulatory

Economics & Finance

Social Sciences & Demographics