Bright Data JavaScript SDK
Access web data through a unified Node.js/TypeScript SDK (@brightdata/sdk). One
client, several services: web unlocking (scrapeUrl), platform scraping
(scrape.<platform>), SERP search (search.google/bing/yandex), AI discovery
(discover), datasets, browser automation, and Scraper Studio.
Requires Node.js ≥ 20. Ships ESM + CommonJS with full TypeScript types. The
client is exported as bdclient (lowercase — not a typo).
Setup gate (do first)
npm install @brightdata/sdk # or: pnpm add / yarn add
The client reads the BRIGHTDATA_API_TOKEN env var, or you pass { apiKey }.
Get a token at https://brightdata.com/cp/setting/users.
// ESM (package.json "type": "module", or .mjs)
import { bdclient } from '@brightdata/sdk';
// CommonJS (.cjs / "type": "commonjs")
const { bdclient } = require('@brightdata/sdk');
const client = new bdclient(); // reads BRIGHTDATA_API_TOKEN
// const client = new bdclient({ apiKey: '...' });
try {
const html = await client.scrapeUrl('https://example.com');
} finally {
await client.close(); // always close (or `await using`)
}
await using client = new bdclient() (TS 5.2+ / Node ≥20) auto-closes at scope end.
Service Selection (decide first, then look up the method)
Pick the service BEFORE reaching for a specific method. Most routing mistakes
come from skipping this step and pattern-matching on keywords.
Have a URL?
├── On a supported platform (Amazon, LinkedIn, Facebook, Instagram, YouTube,
│ TikTok, Reddit, Pinterest, ChatGPT, Perplexity, DigiKey)?
│ → Platform scraping: client.scrape.<platform>.<method>(urls, opts?)
│
├── Generic page (no dedicated platform scraper)?
│ → Web unlocker: client.scrapeUrl(url, { dataFormat: 'markdown' })
│
└── Need login / JS / click-scroll-fill / CAPTCHA / multi-step nav?
→ Browser API: client.browser.getConnectUrl() (connect via Playwright)
No URL?
├── Want entities matching natural-language criteria
│ ("find AI startups in Berlin", "competitors of Acme")?
│ → Discover: client.discover(query, { intent })
│
├── Want web pages / search-result links ("search Google for X")?
│ → SERP: client.search.google(query) [or .bing / .yandex]
│
├── Want to search WITHIN a platform ("Amazon products by keyword",
│ "LinkedIn jobs", "Instagram reels by profile")?
│ → Platform discovery: client.scrape.<platform>.discover*(filters)
│ or amazon.productSearch(...) (NOTE: client.search is SERP-only here)
│
└── Want bulk/historical data at scale?
→ Datasets: client.datasets.<name>.query(filter) → .download(snapshotId)
Edge cases:
- Supported-platform URL BUT user mentions login/click/scroll/JS → Browser API (the interaction trumps the platform).
- Supported-platform scrape returns 403/blocked → fall back to
client.scrapeUrl() (web unlocker).
- "Find/research who are X" with a URL alongside ("competitors of acme.com") → still Discover; the URL is context, not the scrape target.
⚠️ Key differences from the Python SDK
If you know the Python SDK (brightdata-sdk), the JS surface differs — do not
port names blindly:
| Concept |
Python |
JavaScript |
| Client |
SyncBrightDataClient / BrightDataClient |
bdclient (single, all async) |
| Web unlocker |
client.scrape_url(url=...) |
client.scrapeUrl(url, opts) |
| Platform search |
client.search.amazon.products(...) |
none — use scrape.amazon.productSearch / discover* |
client.search |
SERP and platform search |
SERP only (google/bing/yandex) |
| Batch |
*_trigger methods + job.wait() |
pass a string[] to one call, or *Trigger + job.wait() |
| Naming |
snake_case |
camelCase |
| Datasets |
client.datasets.amazon_products |
client.datasets.amazonProducts |
Method Names: Verify Before Asserting
Before claiming a platform method exists/doesn't exist, consult
references/scrapers.md — it lists every platform's verified methods. The SDK
ships TypeScript types, so in a typed project you can also let the compiler/editor
confirm a method exists.
Each platform exposes up to three method styles (see references/scrapers.md):
collect<Thing>(input, opts?) — returns the rows directly (object[]).
<thing>(input, opts?) — orchestrated (trigger → poll → download); returns { data, status, rowCount }. Default to this.
discover<Thing>By<X>(filters, opts?) — find items by keyword/category/URL filter instead of by direct URL.
Likely hallucinations (do NOT write these — verify in references/scrapers.md):
| Wrong |
Right |
client.search.amazon.products(...) |
client.scrape.amazon.productSearch(...) (no platform search router in JS) |
client.scrape.linkedin.people(...) |
client.scrape.linkedin.profiles(urls) |
client.scrape.chatgpt... |
client.scrape.chatGPT... (camelCase G+T) |
client.datasets.amazon_products |
client.datasets.amazonProducts |
BrightDataClient / new BdClient() |
bdclient (all lowercase) |
Useful standalone methods
| Method |
What it does |
client.scrapeUrl(url | url[], opts?) |
Web unlocker — any URL → html / markdown / json / screenshot. Pass an array for parallel batch. |
client.search.google(q | q[], opts?) |
SERP results (also .bing, .yandex). Array = batch. |
client.discover(query, { intent }) |
AI-ranked entity/page discovery. |
client.discoverTrigger(query, opts?) |
Non-blocking discover → Job with .wait() / .fetch(). |
client.datasets.list() |
List all available datasets at runtime. |
client.scraperStudio.run(collectorId, { input }) |
Run a custom Scraper Studio collector. |
client.browser.getConnectUrl({ country }) |
CDP WebSocket URL for Playwright/Puppeteer/Selenium. |
client.listZones() |
List active Bright Data zones. |
client.saveResults(data, { filename, format }) |
Write results to a file. |
client.close() |
Close HTTP connections. Always call when done. |
Gotchas
- Always
await client.close() (or await using). The client holds a keep-alive transport; leaking it hangs the process.
client.search is SERP-only (google/bing/yandex). To search within a platform, use that platform's discover* / productSearch scraper methods — there is no client.search.amazon.
chatGPT is camelCase on client.scrape — client.scrape.chatGPT.search(...), not .chatgpt.
- Batch: pass an array, don't loop.
scrapeUrl([...urls]) and search.google([...queries]) run in parallel internally. For platform scrapers with many inputs, prefer the orchestrated method with an array, or the *Trigger + job.wait() pattern (see references/advanced.md). Don't wrap blocking calls in Promise.all of single-item calls — you'll fight the rate limiter.
- Orchestrated methods block for minutes.
products/profiles/etc. trigger a job and poll until ready (often 2–10 min). Don't set tiny pollTimeouts; tune via { pollInterval, pollTimeout }. Defaults are calibrated.
- Datasets are historical/bulk, not live. Need current data → use platform scrapers or
scrapeUrl. query() returns a snapshotId; download() blocks until the snapshot is ready.
- Web unlocker fallback on 403. If a platform scraper is blocked, retry the URL through
client.scrapeUrl().
- Don't add your own retry loop. The transport already retries network/timeout errors with backoff; double-retrying wastes credits. Tune
timeout / rateLimit on the constructor instead.
- Cost hierarchy (cheapest first): datasets → SERP → platform scrapers → web unlocker → discover → Scraper Studio → Browser API. Prefer the cheapest service that satisfies the request.
Error handling
All errors extend BRDError. Import the specific classes to branch:
import { bdclient, ValidationError, AuthenticationError, BRDError } from '@brightdata/sdk';
try {
const data = await client.scrape.amazon.products(['https://www.amazon.com/dp/B0D77BX8Y4']);
} catch (err) {
if (err instanceof AuthenticationError) { /* bad/expired token */ }
else if (err instanceof ValidationError) { /* bad input */ }
else if (err instanceof BRDError) { /* any SDK error */ }
else throw err;
}
Classes: BRDError (base), ValidationError, AuthenticationError, ZoneError,
NetworkError, NetworkTimeoutError, TimeoutError, APIError,
DataNotReadyError, FSError.
Examples
Scrape a generic page as markdown
const md = await client.scrapeUrl('https://example.com/article', { dataFormat: 'markdown' });
Get an Amazon product (orchestrated — default)
const res = await client.scrape.amazon.products(['https://www.amazon.com/dp/B0D77BX8Y4']);
console.log(res.status, res.rowCount, res.data);
SERP, batched
const results = await client.search.google(['best running shoes', 'best trail shoes'], { country: 'us' });
Find entities (Discover)
const startups = await client.discover('AI startups in Berlin', {
intent: 'early-stage machine-learning companies',
numResults: 10,
});
Bulk/historical via datasets
const ds = client.datasets;
const snapshotId = await ds.instagramProfiles.query({ url: 'https://www.instagram.com/natgeo/' }, { records_limit: 50 });
// download() blocks until the snapshot is ready
const rows = await ds.instagramProfiles.download(snapshotId);
Troubleshooting
AuthenticationError / 401 — token missing or invalid. Check BRIGHTDATA_API_TOKEN or the apiKey option.
- 403 / blocked — site blocked the scraper. Retry the URL via
client.scrapeUrl() (web unlocker).
- Timeout — increase
timeout (constructor, 1000–300000 ms) or the orchestrated pollTimeout; do not lower it.
- "Dataset not found" — call
client.datasets.list(); dataset names are camelCase (amazonProducts, linkedinProfiles).
- Process hangs after work finishes — you forgot
await client.close().
- SSL/proxy errors in sandboxes — see
references/advanced.md for zone/SSL options.
- Rate-limit errors — set
{ rateLimit, ratePeriod } on the constructor, or batch via arrays / the trigger pattern instead of Promise.all.
When to load references
references/scrapers.md — when the user names Amazon, LinkedIn, Facebook, Instagram, YouTube, TikTok, Reddit, Pinterest, ChatGPT, Perplexity, DigiKey — verified per-platform method tables (collect / orchestrated / discover) and the scrapeUrl web-unlocker options.
references/search.md — SERP engines (search.google/bing/yandex) and the Discover API (discover / discoverTrigger), with all options.
references/datasets-overview.md — dataset names, the query → getStatus → download lifecycle, and filters.
references/advanced.md — constructor options & env vars, batch/trigger orchestration, Browser API (Playwright/Puppeteer), Scraper Studio, error classes, and zones/SSL.
1---
2name: brightdata-sdk-js
3description: |
4 Web data extraction and discovery using the Bright Data JavaScript/TypeScript
5 SDK (`@brightdata/sdk`). Use when the user is working in Node.js/TypeScript and
6 asks to "scrape", "get data from", "extract", "search for", or "find"
7 information from websites. Also use when the user mentions specific platforms
8 like Amazon, LinkedIn, Instagram, Facebook, TikTok, YouTube, Reddit, Pinterest,
9 ChatGPT, Perplexity, or DigiKey, or asks for "bulk data", "historical data", or
10 "dataset" from JS. Covers scraping, SERP search, AI discovery, datasets,
11 browser automation, and Scraper Studio. For Python, use brightdata-sdk; for the
12 terminal CLI, use brightdata-cli.
13metadata:
14 source: https://skillmd.com/skills/brightdata/brightdata-sdk-js
15 author: brightdata
16 version: "1.0"
17 package: "@brightdata/sdk"
18 repository: https://github.com/brightdata/sdk-js
19---
20
21# Bright Data JavaScript SDK
22
23Access web data through a unified Node.js/TypeScript SDK (`@brightdata/sdk`). One
24client, several services: web unlocking (`scrapeUrl`), platform scraping
25(`scrape.<platform>`), SERP search (`search.google/bing/yandex`), AI discovery
26(`discover`), datasets, browser automation, and Scraper Studio.
27
28**Requires Node.js ≥ 20.** Ships ESM + CommonJS with full TypeScript types. The
29client is exported as `bdclient` (lowercase — not a typo).
30
31## Setup gate (do first)
32
33```bash
34npm install @brightdata/sdk # or: pnpm add / yarn add
35```
36
37The client reads the `BRIGHTDATA_API_TOKEN` env var, or you pass `{ apiKey }`.
38Get a token at https://brightdata.com/cp/setting/users.
39
40```javascript
41// ESM (package.json "type": "module", or .mjs)
42import { bdclient } from '@brightdata/sdk';
43
44// CommonJS (.cjs / "type": "commonjs")
45const { bdclient } = require('@brightdata/sdk');
46
47const client = new bdclient(); // reads BRIGHTDATA_API_TOKEN
48// const client = new bdclient({ apiKey: '...' });
49try {
50 const html = await client.scrapeUrl('https://example.com');
51} finally {
52 await client.close(); // always close (or `await using`)
53}
54```
55
56`await using client = new bdclient()` (TS 5.2+ / Node ≥20) auto-closes at scope end.
57
58## Service Selection (decide first, then look up the method)
59
60Pick the service BEFORE reaching for a specific method. Most routing mistakes
61come from skipping this step and pattern-matching on keywords.
62
63```
64Have a URL?
65 ├── On a supported platform (Amazon, LinkedIn, Facebook, Instagram, YouTube,
66 │ TikTok, Reddit, Pinterest, ChatGPT, Perplexity, DigiKey)?
67 │ → Platform scraping: client.scrape.<platform>.<method>(urls, opts?)
68 │
69 ├── Generic page (no dedicated platform scraper)?
70 │ → Web unlocker: client.scrapeUrl(url, { dataFormat: 'markdown' })
71 │
72 └── Need login / JS / click-scroll-fill / CAPTCHA / multi-step nav?
73 → Browser API: client.browser.getConnectUrl() (connect via Playwright)
74
75No URL?
76 ├── Want entities matching natural-language criteria
77 │ ("find AI startups in Berlin", "competitors of Acme")?
78 │ → Discover: client.discover(query, { intent })
79 │
80 ├── Want web pages / search-result links ("search Google for X")?
81 │ → SERP: client.search.google(query) [or .bing / .yandex]
82 │
83 ├── Want to search WITHIN a platform ("Amazon products by keyword",
84 │ "LinkedIn jobs", "Instagram reels by profile")?
85 │ → Platform discovery: client.scrape.<platform>.discover*(filters)
86 │ or amazon.productSearch(...) (NOTE: client.search is SERP-only here)
87 │
88 └── Want bulk/historical data at scale?
89 → Datasets: client.datasets.<name>.query(filter) → .download(snapshotId)
90```
91
92Edge cases:
93- Supported-platform URL BUT user mentions login/click/scroll/JS → Browser API (the interaction trumps the platform).
94- Supported-platform scrape returns 403/blocked → fall back to `client.scrapeUrl()` (web unlocker).
95- "Find/research who are X" with a URL alongside ("competitors of acme.com") → still Discover; the URL is context, not the scrape target.
96
97## ⚠️ Key differences from the Python SDK
98
99If you know the Python SDK (`brightdata-sdk`), the JS surface differs — do not
100port names blindly:
101
102| Concept | Python | JavaScript |
103|---|---|---|
104| Client | `SyncBrightDataClient` / `BrightDataClient` | `bdclient` (single, all async) |
105| Web unlocker | `client.scrape_url(url=...)` | `client.scrapeUrl(url, opts)` |
106| Platform search | `client.search.amazon.products(...)` | **none** — use `scrape.amazon.productSearch` / `discover*` |
107| `client.search` | SERP **and** platform search | **SERP only** (google/bing/yandex) |
108| Batch | `*_trigger` methods + `job.wait()` | pass a `string[]` to one call, or `*Trigger` + `job.wait()` |
109| Naming | `snake_case` | `camelCase` |
110| Datasets | `client.datasets.amazon_products` | `client.datasets.amazonProducts` |
111
112## Method Names: Verify Before Asserting
113
114Before claiming a platform method exists/doesn't exist, **consult
115`references/scrapers.md`** — it lists every platform's verified methods. The SDK
116ships TypeScript types, so in a typed project you can also let the compiler/editor
117confirm a method exists.
118
119Each platform exposes up to three method styles (see `references/scrapers.md`):
120- **`collect<Thing>(input, opts?)`** — returns the rows directly (`object[]`).
121- **`<thing>(input, opts?)`** — orchestrated (trigger → poll → download); returns `{ data, status, rowCount }`. **Default to this.**
122- **`discover<Thing>By<X>(filters, opts?)`** — find items by keyword/category/URL filter instead of by direct URL.
123
124**Likely hallucinations** (do NOT write these — verify in `references/scrapers.md`):
125
126| Wrong | Right |
127|---|---|
128| `client.search.amazon.products(...)` | `client.scrape.amazon.productSearch(...)` (no platform search router in JS) |
129| `client.scrape.linkedin.people(...)` | `client.scrape.linkedin.profiles(urls)` |
130| `client.scrape.chatgpt...` | `client.scrape.chatGPT...` (camelCase G+T) |
131| `client.datasets.amazon_products` | `client.datasets.amazonProducts` |
132| `BrightDataClient` / `new BdClient()` | `bdclient` (all lowercase) |
133
134## Useful standalone methods
135
136| Method | What it does |
137|---|---|
138| `client.scrapeUrl(url \| url[], opts?)` | Web unlocker — any URL → html / markdown / json / screenshot. Pass an array for parallel batch. |
139| `client.search.google(q \| q[], opts?)` | SERP results (also `.bing`, `.yandex`). Array = batch. |
140| `client.discover(query, { intent })` | AI-ranked entity/page discovery. |
141| `client.discoverTrigger(query, opts?)` | Non-blocking discover → `Job` with `.wait()` / `.fetch()`. |
142| `client.datasets.list()` | List all available datasets at runtime. |
143| `client.scraperStudio.run(collectorId, { input })` | Run a custom Scraper Studio collector. |
144| `client.browser.getConnectUrl({ country })` | CDP WebSocket URL for Playwright/Puppeteer/Selenium. |
145| `client.listZones()` | List active Bright Data zones. |
146| `client.saveResults(data, { filename, format })` | Write results to a file. |
147| `client.close()` | Close HTTP connections. Always call when done. |
148
149## Gotchas
150
151- **Always `await client.close()`** (or `await using`). The client holds a keep-alive transport; leaking it hangs the process.
152- **`client.search` is SERP-only** (google/bing/yandex). To search *within* a platform, use that platform's `discover*` / `productSearch` scraper methods — there is no `client.search.amazon`.
153- **`chatGPT` is camelCase** on `client.scrape` — `client.scrape.chatGPT.search(...)`, not `.chatgpt`.
154- **Batch: pass an array, don't loop.** `scrapeUrl([...urls])` and `search.google([...queries])` run in parallel internally. For platform scrapers with many inputs, prefer the orchestrated method with an array, or the `*Trigger` + `job.wait()` pattern (see `references/advanced.md`). Don't wrap blocking calls in `Promise.all` of single-item calls — you'll fight the rate limiter.
155- **Orchestrated methods block for minutes.** `products`/`profiles`/etc. trigger a job and poll until ready (often 2–10 min). Don't set tiny `pollTimeout`s; tune via `{ pollInterval, pollTimeout }`. Defaults are calibrated.
156- **Datasets are historical/bulk, not live.** Need current data → use platform scrapers or `scrapeUrl`. `query()` returns a `snapshotId`; `download()` blocks until the snapshot is ready.
157- **Web unlocker fallback on 403.** If a platform scraper is blocked, retry the URL through `client.scrapeUrl()`.
158- **Don't add your own retry loop.** The transport already retries network/timeout errors with backoff; double-retrying wastes credits. Tune `timeout` / `rateLimit` on the constructor instead.
159- **Cost hierarchy (cheapest first):** datasets → SERP → platform scrapers → web unlocker → discover → Scraper Studio → Browser API. Prefer the cheapest service that satisfies the request.
160
161## Error handling
162
163All errors extend `BRDError`. Import the specific classes to branch:
164
165```javascript
166import { bdclient, ValidationError, AuthenticationError, BRDError } from '@brightdata/sdk';
167
168try {
169 const data = await client.scrape.amazon.products(['https://www.amazon.com/dp/B0D77BX8Y4']);
170} catch (err) {
171 if (err instanceof AuthenticationError) { /* bad/expired token */ }
172 else if (err instanceof ValidationError) { /* bad input */ }
173 else if (err instanceof BRDError) { /* any SDK error */ }
174 else throw err;
175}
176```
177
178Classes: `BRDError` (base), `ValidationError`, `AuthenticationError`, `ZoneError`,
179`NetworkError`, `NetworkTimeoutError`, `TimeoutError`, `APIError`,
180`DataNotReadyError`, `FSError`.
181
182## Examples
183
184### Scrape a generic page as markdown
185```javascript
186const md = await client.scrapeUrl('https://example.com/article', { dataFormat: 'markdown' });
187```
188
189### Get an Amazon product (orchestrated — default)
190```javascript
191const res = await client.scrape.amazon.products(['https://www.amazon.com/dp/B0D77BX8Y4']);
192console.log(res.status, res.rowCount, res.data);
193```
194
195### SERP, batched
196```javascript
197const results = await client.search.google(['best running shoes', 'best trail shoes'], { country: 'us' });
198```
199
200### Find entities (Discover)
201```javascript
202const startups = await client.discover('AI startups in Berlin', {
203 intent: 'early-stage machine-learning companies',
204 numResults: 10,
205});
206```
207
208### Bulk/historical via datasets
209```javascript
210const ds = client.datasets;
211const snapshotId = await ds.instagramProfiles.query({ url: 'https://www.instagram.com/natgeo/' }, { records_limit: 50 });
212// download() blocks until the snapshot is ready
213const rows = await ds.instagramProfiles.download(snapshotId);
214```
215
216## Troubleshooting
217
218- **`AuthenticationError` / 401** — token missing or invalid. Check `BRIGHTDATA_API_TOKEN` or the `apiKey` option.
219- **403 / blocked** — site blocked the scraper. Retry the URL via `client.scrapeUrl()` (web unlocker).
220- **Timeout** — increase `timeout` (constructor, 1000–300000 ms) or the orchestrated `pollTimeout`; do not lower it.
221- **"Dataset not found"** — call `client.datasets.list()`; dataset names are camelCase (`amazonProducts`, `linkedinProfiles`).
222- **Process hangs after work finishes** — you forgot `await client.close()`.
223- **SSL/proxy errors in sandboxes** — see `references/advanced.md` for zone/SSL options.
224- **Rate-limit errors** — set `{ rateLimit, ratePeriod }` on the constructor, or batch via arrays / the trigger pattern instead of `Promise.all`.
225
226## When to load references
227
228- `references/scrapers.md` — when the user names **Amazon, LinkedIn, Facebook, Instagram, YouTube, TikTok, Reddit, Pinterest, ChatGPT, Perplexity, DigiKey** — verified per-platform method tables (collect / orchestrated / discover) and the `scrapeUrl` web-unlocker options.
229- `references/search.md` — SERP engines (`search.google/bing/yandex`) and the Discover API (`discover` / `discoverTrigger`), with all options.
230- `references/datasets-overview.md` — dataset names, the `query → getStatus → download` lifecycle, and filters.
231- `references/advanced.md` — constructor options & env vars, batch/trigger orchestration, Browser API (Playwright/Puppeteer), Scraper Studio, error classes, and zones/SSL.