# Scraping Twitter Profiles

> Scrapes Twitter/X profile data for any list of usernames using apidojo's Twitter User scraper on Apify. Triggers when the user asks to: get profile data for Twitter accounts, fetch follower counts for a list of usernames, scrape Twitter bio and stats for multiple users, export Twitter profile metadata, check account details for a set of handles, or bulk-fetch Twitter user information. Returns username, display name, bio, follower count, following count, tweet count, verified status, and profile URL. Ideal for data analysts, list enrichment pipelines, and market researchers.

- Skill: `apidojo-io/scraping-twitter-profiles` (Agent Skill)
- Install (CLI): `npx skillmds@latest add apidojo-io/scraping-twitter-profiles`
- Raw SKILL.md: https://api.skillmd.com/api/skills/apidojo-io/scraping-twitter-profiles/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- License: Apache-2.0
- Author: apidojo-io (https://skillmd.com/u/apidojo-io)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/apidojo-io/scraping-twitter-profiles

---


# Scraping Twitter Profiles

Bulk profile data export for any set of Twitter/X usernames. Returns account-level metadata: follower counts, bios, join dates, verification status.

## Prerequisites

- `APIFY_TOKEN` environment variable set
- Optional: Apify MCP server installed

## Inputs

| Parameter | Type | Required | Default | Notes |
|-----------|------|----------|---------|-------|
| `startUrls` | array | Optional | `[]` | Twitter profile or tweet URLs |
| `twitterHandles` | array | Optional | `[]` | Twitter usernames (without @) |
| `twitterUserIds` | array | Optional | `[]` | Twitter user IDs |
| `getFollowers` | boolean | Optional | `false` | Extract follower lists |
| `getFollowing` | boolean | Optional | `false` | Extract following lists |
| `getRetweeters` | boolean | Optional | `false` | Extract retweeters of a tweet URL |
| `includeUnavailableUsers` | boolean | Optional | `false` | Include unavailable/suspended users |
| `maxItems` | number | Optional | Unlimited | Maximum users to return |
| `customMapFunction` | string | Optional | — | JavaScript function to transform each output object |

## Workflow

```
Progress:
- [ ] Step 1: Normalize username list
- [ ] Step 2: Run twitter-user-scraper
- [ ] Step 3: Poll for SUCCEEDED
- [ ] Step 4: Deliver profile dataset
```

### Step 1: Normalize

Remove `@` symbols, deduplicate, validate no empty strings.

### Step 2: Run the Actor


**Recommended — run_actor.js (handles waiting, output, and file saving automatically):**
```bash
# Quick answer (prints table to chat)
node scripts/run_actor.js \
  --actor "apidojo~twitter-user-scraper" \
  --input '{"param": "value"}'

# Save as CSV
node scripts/run_actor.js \
  --actor "apidojo~twitter-user-scraper" \
  --input '{"param": "value"}' \
  --output YYYY-MM-DD_results.csv --format csv

# Save as JSON
node scripts/run_actor.js \
  --actor "apidojo~twitter-user-scraper" \
  --input '{"param": "value"}' \
  --output YYYY-MM-DD_results.json --format json
```
> `APIFY_TOKEN` must be set in environment or `.env` file.

**If Apify MCP is available:**
```
Tool: apify:run-actor
Actor: "apidojo~twitter-user-scraper"
Input:
{
  "usernames": ["handle1", "handle2", "handle3"]
}
```

**REST API fallback:**
```bash
curl -X POST \
  "https://api.apify.com/v2/acts/apidojo~twitter-user-scraper/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"usernames": ["handle1", "handle2"]}'
```

Save `id` as `RUN_ID`. Poll until `status = SUCCEEDED`:
```bash
curl "https://api.apify.com/v2/actor-runs/$RUN_ID?token=$APIFY_TOKEN" | grep '"status"'
```

Fetch results:
```bash
curl "https://api.apify.com/v2/actor-runs/$RUN_ID/dataset/items?token=$APIFY_TOKEN&format=json"
```

### Step 3: Handle Edge Cases

- **Missing accounts in results**: Account suspended, deleted, or handle changed. Note missing handles in output.
- **Private accounts**: Returns profile metadata but no tweet content — flag `isPrivate: true` rows.
- **Deactivated accounts**: Returns null or empty object — exclude from final dataset.

## Output Format

```
# Twitter Profile Dataset
Accounts requested: N | Accounts returned: N | Missing: N

| Username | Display Name | Followers | Following | Tweets | Verified | Bio (truncated) | Joined |
|----------|-------------|-----------|-----------|--------|----------|-----------------|--------|
| ...      | ...         | ...       | ...       | ...    | ...      | ...             | ...    |

Available fields: username, displayName, followersCount, followingCount, tweetCount,
isVerified, isBlueVerified, description, location, url, profileImageUrl, createdAt, isPrivate
```

## Troubleshooting

**Some handles missing from results:** Accounts may be suspended or renamed — cross-reference manually.
**Large lists (> 1000):** Break into batches of 500 and run sequentially.