Brightdata Linkedin

BrightData LinkedIn Scraper

LinkedIn data collection using BrightData's Datasets API (Web Scraper IDE). All operations save results to disk and return summaries to context.

When to Use This Skill

Use brightdata-linkedin (this skill) when:

• Need LinkedIn job search by keyword/location (not just by URL)
• Need to discover posts from company pages or profiles
• Need people search functionality
• Want direct control over the Datasets API parameters
• Need both sync and async modes

Use brightdata (SDK skill) when:

• Scraping general websites, SERP, or Amazon data
• Using simple URL-based LinkedIn scraping
• Prefer the SDK abstraction layer

Overview

This skill provides access to BrightData's LinkedIn scrapers for collecting:

• People Profiles: Full profile data from LinkedIn profile URLs
• Company Information: Company page data from LinkedIn company URLs
• Job Listings: Job details by URL or discover jobs by keyword/location
• Posts: Posts by URL or discover posts by company/profile
• People Search: Search results from LinkedIn people search URLs

Quick Start

Single Profile Scrape:

python skills/brightdata-linkedin/scripts/fetch.py \
  --endpoint profiles \
  --input "https://www.linkedin.com/in/username"

Bulk Company Scrape:

python skills/brightdata-linkedin/scripts/fetch.py \
  --endpoint companies \
  --input companies.json \
  --batch

Job Search by Keyword:

python skills/brightdata-linkedin/scripts/fetch.py \
  --endpoint jobs-search \
  --location "Berlin" \
  --keyword "software engineer" \
  --country DE \
  --job-type "Full-time"

Available Endpoints

Parameters

Common Parameters

Job Search Parameters (jobs-search endpoint)

Posts Discovery Parameters (posts-company, posts-profile)

Batch Operations

For scraping multiple URLs, use batch mode with a file input.

JSON Format

{
  "urls": [
    "https://www.linkedin.com/in/person1",
    "https://www.linkedin.com/in/person2",
    "https://www.linkedin.com/in/person3"
  ]
}

Also supports: inputs, data, items keys, or a plain array.

CSV Format

url
https://www.linkedin.com/in/person1
https://www.linkedin.com/in/person2
https://www.linkedin.com/in/person3

TXT Format

https://www.linkedin.com/in/person1
https://www.linkedin.com/in/person2
https://www.linkedin.com/in/person3

Batch Example

python skills/brightdata-linkedin/scripts/fetch.py \
  --endpoint profiles \
  --input profiles.json \
  --batch

Job Search Batch

For job searches with multiple locations/keywords, create a JSON file:

{
  "searches": [
    {"location": "Berlin", "keyword": "software engineer", "country": "DE"},
    {"location": "Munich", "keyword": "data scientist", "country": "DE"},
    {"location": "Hamburg", "keyword": "product manager", "country": "DE"}
  ]
}

python skills/brightdata-linkedin/scripts/fetch.py \
  --endpoint jobs-search \
  --input job_searches.json \
  --batch

Sync vs Async Mode

Synchronous Mode (Default)

• Results returned immediately in response
• Best for single requests or small batches
• Timeout: configurable (default 120s)

python skills/brightdata-linkedin/scripts/fetch.py \
  --endpoint profiles \
  --input "https://www.linkedin.com/in/username" \
  --mode sync

Asynchronous Mode

• Request queued, returns snapshot_id
• Use for large batches or long-running jobs
• Retrieve results later with snapshot_id

# Submit async request
python skills/brightdata-linkedin/scripts/fetch.py \
  --endpoint profiles \
  --input profiles.json \
  --batch \
  --mode async

# Check status and retrieve results
python skills/brightdata-linkedin/scripts/status.py \
  --snapshot-id <snapshot_id>

Output Format

All operations return a summary to stdout:

{
  "success": true,
  "endpoint": "profiles",
  "mode": "sync",
  "record_count": 5,
  "output_file": "output/brightdata_linkedin_profiles_5_20250125T123456.json",
  "summary": "Successfully scraped 5 LinkedIn profiles",
  "stats": {
    "total_inputs": 5,
    "successful": 5,
    "failed": 0
  }
}

Raw Output File Structure

Saved to output/brightdata_linkedin_{endpoint}_{timestamp}.json:

{
  "endpoint": "profiles",
  "dataset_id": "gd_l1viktl72bvl7bjuj0",
  "mode": "sync",
  "timestamp": "2025-01-25T12:34:56+00:00",
  "inputs_count": 5,
  "stats": {
    "total_inputs": 5,
    "successful": 5,
    "failed": 0
  },
  "results": [
    {
      "input": {"url": "https://www.linkedin.com/in/username"},
      "success": true,
      "data": { /* BrightData response */ }
    }
  ]
}

API Configuration

Requires BRIGHTDATA_API_TOKEN in .env:

BRIGHTDATA_API_TOKEN=your_api_token_here

Get your API token from the BrightData dashboard: https://brightdata.com/cp/

Usage Examples

Profile Scraping

Single profile:

python skills/brightdata-linkedin/scripts/fetch.py \
  --endpoint profiles \
  --input "https://www.linkedin.com/in/williamhgates"

Multiple profiles (comma-separated):

python skills/brightdata-linkedin/scripts/fetch.py \
  --endpoint profiles \
  --input "https://www.linkedin.com/in/person1,https://www.linkedin.com/in/person2"

Batch from file:

python skills/brightdata-linkedin/scripts/fetch.py \
  --endpoint profiles \
  --input profiles.txt \
  --batch

Company Information

Single company:

python skills/brightdata-linkedin/scripts/fetch.py \
  --endpoint companies \
  --input "https://www.linkedin.com/company/google"

Job Search

Basic search:

python skills/brightdata-linkedin/scripts/fetch.py \
  --endpoint jobs-search \
  --location "San Francisco" \
  --keyword "machine learning engineer"

Advanced search with filters:

python skills/brightdata-linkedin/scripts/fetch.py \
  --endpoint jobs-search \
  --location "Berlin" \
  --keyword "senior software engineer" \
  --country DE \
  --job-type "Full-time" \
  --experience-level "Mid-Senior" \
  --remote "Remote" \
  --time-range "Past week"

Posts Discovery

Posts from a company:

python skills/brightdata-linkedin/scripts/fetch.py \
  --endpoint posts-company \
  --input "https://www.linkedin.com/company/google" \
  --start-date 2025-01-01 \
  --end-date 2025-01-25

Posts from a profile:

python skills/brightdata-linkedin/scripts/fetch.py \
  --endpoint posts-profile \
  --input "https://www.linkedin.com/in/username"

People Search

Search results page:

python skills/brightdata-linkedin/scripts/fetch.py \
  --endpoint people-search \
  --input "https://www.linkedin.com/search/results/people/?keywords=cto&origin=GLOBAL_SEARCH_HEADER"

Error Handling

The script handles errors gracefully:

• Partial success: If some inputs fail, successful ones are still saved
• Stats tracking: Failed requests counted in stats.failed
• Per-result errors: Each failed result includes error message

Check the output:

{
  "success": true,
  "stats": {
    "total_inputs": 10,
    "successful": 8,
    "failed": 2
  }
}

Rate Limits and Pricing

• BrightData charges per successful data record
• Rate limits depend on your plan
• Use async mode for large batches to avoid timeouts
• Check your BrightData dashboard for usage and limits

Limitations

• Platform ToS: Ensure compliance with LinkedIn's Terms of Service
• Data freshness: Some data may be cached
• URL formats: Requires valid LinkedIn URLs
• Rate limits: Subject to BrightData API limits

Dependencies

No additional packages required beyond standard library. Uses requests for HTTP calls.