# Dfs Onpage API

> Comprehensive on-page SEO analysis using DataForSEO OnPage API. Supports instant single-page analysis and full website crawls with detailed metrics including Core Web Vitals, technical issues, duplicate content, broken links, and redirect chains. Use when needing on-page SEO audits, technical SEO analysis, site health checks, or detailed page-level performance metrics for individual URLs or entire websites.

- Skill: `buzzmatic/dfs-onpage-api` (Agent Skill, multi-file: 13 files)
- Install (CLI): `npx skillmds@latest add buzzmatic/dfs-onpage-api`
- Raw SKILL.md: https://api.skillmd.com/api/skills/buzzmatic/dfs-onpage-api/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Marketing & Growth
- Author: Buzzmatic (https://skillmd.com/u/buzzmatic)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/buzzmatic/dfs-onpage-api

---


# DataForSEO OnPage API

Comprehensive on-page SEO analysis with support for instant single-page checks and full website crawls.

## Overview

This skill provides access to the DataForSEO OnPage API for technical SEO analysis. It supports two primary workflows:

1. **Instant Analysis** - Quick single-page analysis without task creation (ideal for spot checks)
2. **Full Site Crawls** - Comprehensive website audits with task-based crawling (ideal for complete technical audits)

## Quick Decision Guide

**Use instant_pages.py when:**
- Analyzing a single page quickly
- Need immediate results without waiting for crawl
- Checking page performance, Core Web Vitals, or specific technical issues
- No task management needed

**Use task_post.py + retrieval scripts when:**
- Auditing an entire website
- Need bulk data for multiple pages
- Want to identify site-wide issues (duplicates, broken links, redirect chains)
- Require comprehensive technical SEO audit

## Instant Single-Page Analysis

### instant_pages.py

Performs real-time analysis of individual web pages without task creation. Returns immediate results.

**Basic usage:**
```bash
python scripts/instant_pages.py --url https://example.com/page
```

**With Core Web Vitals (browser rendering):**
```bash
python scripts/instant_pages.py \
  --url https://example.com/page \
  --enable-browser-rendering \
  --load-resources
```

**With custom output path:**
```bash
python scripts/instant_pages.py \
  --url https://example.com/page \
  --enable-browser-rendering \
  --output output/onpage_instant_example.json
```

**Key Parameters:**
- `--url` (required): Target page URL
- `--enable-browser-rendering`: Measure Core Web Vitals (CLS, LCP, FID)
- `--enable-javascript`: Execute page JavaScript
- `--load-resources`: Load images, scripts, stylesheets
- `--store-raw-html`: Save HTML for later analysis
- `--check-spell`: Enable spelling verification
- `--browser-preset`: Device type (desktop, mobile, tablet)
- `--output`: Custom output path

**Returns:**
- Page metrics (status code, load time, size)
- OnPage score (0-100)
- Core Web Vitals (if browser rendering enabled)
- SEO elements (title, meta, headings)
- Technical issues (missing tags, broken links, etc.)

## Full Website Crawl Workflow

### Step 1: Create Crawl Task (task_post.py)

Initiates a website crawl. Returns a task ID for retrieving results.

**Basic crawl:**
```bash
python scripts/task_post.py \
  --target example.com \
  --max-crawl-pages 500
```

**Advanced crawl with Core Web Vitals:**
```bash
python scripts/task_post.py \
  --target example.com \
  --max-crawl-pages 1000 \
  --enable-browser-rendering \
  --load-resources \
  --calculate-keyword-density \
  --check-spell
```

**Sitemap-based crawl:**
```bash
python scripts/task_post.py \
  --target example.com \
  --max-crawl-pages 500 \
  --respect-sitemap \
  --crawl-sitemap-only
```

**With priority URLs:**
```bash
python scripts/task_post.py \
  --target example.com \
  --max-crawl-pages 500 \
  --priority-urls "https://example.com/important,https://example.com/page2"
```

**Key Parameters:**
- `--target` (required): Domain name (without https:// or www.)
- `--max-crawl-pages` (required): Maximum pages to crawl
- `--start-url`: Initial URL to start crawling
- `--max-crawl-depth`: Maximum link depth
- `--crawl-delay`: Delay between requests (ms)
- `--enable-browser-rendering`: Enable Core Web Vitals
- `--enable-javascript`: Execute page JavaScript
- `--load-resources`: Load images, scripts, stylesheets
- `--store-raw-html`: Store HTML for later retrieval
- `--respect-sitemap`: Follow sitemap page order
- `--custom-sitemap`: Alternative sitemap URL
- `--crawl-sitemap-only`: Limit to sitemap pages only
- `--calculate-keyword-density`: Compute keyword frequency
- `--check-spell`: Enable spell-checking
- `--priority-urls`: Comma-separated priority URLs (max 20)
- `--output`: Custom output path

**Returns:**
- Task ID (save this for retrieving results)
- Cost information

**Example output:**
```
Task created successfully!
Task ID: abc123-def456-ghi789
```

### Step 2: Check Task Status (tasks_ready.py)

Check which tasks are completed and ready for data retrieval.

```bash
python scripts/tasks_ready.py
```

**Output:**
- List of completed tasks with IDs
- Pages crawled for each task
- Retrieval command examples

### Step 3: Retrieve Results

Once the task is completed, use the task ID to retrieve different types of data.

#### Get Summary (summary.py)

Website-level summary with overall metrics and issue counts.

```bash
python scripts/summary.py --id abc123-def456-ghi789
```

**With custom output path:**
```bash
python scripts/summary.py \
  --id abc123-def456-ghi789 \
  --output output/onpage_summary.json
```

**Returns:**
- Crawl status (pages crawled, pages in queue)
- Domain info (CMS, server, SSL status)
- Page metrics (OnPage score, links, duplicates, broken links)
- Issue summary (counts by type)

#### Get Pages (pages.py)

Detailed page-level data with filtering and pagination support.

**Basic usage:**
```bash
python scripts/pages.py --id abc123-def456-ghi789
```

**Filter pages with issues:**
```bash
python scripts/pages.py \
  --id abc123-def456-ghi789 \
  --filter "checks.no_h1_tag,=,true"
```

**Filter by status code:**
```bash
python scripts/pages.py \
  --id abc123-def456-ghi789 \
  --filter "status_code,=,404"
```

**Filter by OnPage score:**
```bash
python scripts/pages.py \
  --id abc123-def456-ghi789 \
  --filter "onpage_score,<,50" \
  --order-by "onpage_score,asc"
```

**Multiple filters (AND logic):**
```bash
python scripts/pages.py \
  --id abc123-def456-ghi789 \
  --filter "status_code,=,200" \
  --filter "onpage_score,<,70" \
  --limit 1000
```

**Pagination:**
```bash
python scripts/pages.py \
  --id abc123-def456-ghi789 \
  --limit 1000 \
  --offset 1000
```

**Key Parameters:**
- `--id` (required): Task ID
- `--limit`: Max pages to return (default 100, max 1000)
- `--offset`: Skip initial results for pagination
- `--filter`: Filter condition (field,operator,value) - can specify multiple
- `--order-by`: Sort criteria (field,direction) - can specify multiple
- `--output`: Custom output path

**Common filters:**
- `status_code,=,404` - Find 404 pages
- `onpage_score,<,50` - Low-quality pages
- `checks.no_h1_tag,=,true` - Missing H1
- `checks.duplicate_title,=,true` - Duplicate titles
- `checks.broken_links,=,true` - Pages with broken links

#### Get Links (links.py)

Internal and external link data.

```bash
python scripts/links.py --id abc123-def456-ghi789
```

**Filter broken links:**
```bash
python scripts/links.py \
  --id abc123-def456-ghi789 \
  --filter "broken,=,true"
```

#### Get Resources (resources.py)

Website resources (images, scripts, stylesheets).

```bash
python scripts/resources.py --id abc123-def456-ghi789
```

**Filter by resource type:**
```bash
python scripts/resources.py \
  --id abc123-def456-ghi789 \
  --filter "resource_type,=,image"
```

#### Get Duplicate Tags (duplicate_tags.py)

Pages with duplicate title or meta description tags.

```bash
python scripts/duplicate_tags.py --id abc123-def456-ghi789
```

**Returns:**
- Groups of pages sharing the same title or meta description
- Useful for identifying duplicate meta tag issues

#### Get Duplicate Content (duplicate_content.py)

Pages with similar content.

```bash
python scripts/duplicate_content.py --id abc123-def456-ghi789
```

**Returns:**
- Groups of pages with similar content
- Useful for identifying thin or duplicate content issues

#### Get Non-Indexable Pages (non_indexable.py)

Pages blocked from search engine indexing.

```bash
python scripts/non_indexable.py --id abc123-def456-ghi789
```

**Returns:**
- Pages blocked by robots.txt, meta noindex, or other mechanisms
- Reason for each non-indexable page

#### Get Redirect Chains (redirect_chains.py)

Multiple redirect issues.

```bash
python scripts/redirect_chains.py --id abc123-def456-ghi789
```

**Returns:**
- Redirect chains with start and end URLs
- Chain length for each redirect sequence
- Useful for identifying redirect inefficiencies

## Common Workflows

### 1. Quick Page Audit

Analyze a single page for technical issues:

```bash
python scripts/instant_pages.py \
  --url https://example.com/page \
  --enable-browser-rendering \
  --load-resources \
  --check-spell
```

### 2. Full Site Technical Audit

Comprehensive website audit:

```bash
# Step 1: Create crawl task
python scripts/task_post.py \
  --target example.com \
  --max-crawl-pages 1000 \
  --enable-browser-rendering \
  --calculate-keyword-density

# Step 2: Wait for completion, then check status
python scripts/tasks_ready.py

# Step 3: Get summary
python scripts/summary.py --id <task-id>

# Step 4: Get detailed pages
python scripts/pages.py --id <task-id> --limit 1000

# Step 5: Get specific issues
python scripts/duplicate_tags.py --id <task-id>
python scripts/links.py --id <task-id> --filter "broken,=,true"
python scripts/redirect_chains.py --id <task-id>
```

### 3. Find Low-Quality Pages

Identify pages with low OnPage scores:

```bash
python scripts/pages.py \
  --id <task-id> \
  --filter "onpage_score,<,60" \
  --order-by "onpage_score,asc" \
  --limit 1000
```

### 4. Audit Site-Wide Issues

Get all pages with specific technical issues:

```bash
# Missing H1 tags
python scripts/pages.py \
  --id <task-id> \
  --filter "checks.no_h1_tag,=,true"

# Duplicate titles
python scripts/pages.py \
  --id <task-id> \
  --filter "checks.duplicate_title,=,true"

# Large page size
python scripts/pages.py \
  --id <task-id> \
  --filter "size,>,1000000" \
  --order-by "size,desc"
```

## Output Structure

All scripts save data to:
- `output/` under the current working directory by default
- A custom path when using `--output`

**File naming:**
- `onpage_instant_<domain>.json` - Instant analysis
- `onpage_task_<domain>.json` - Task creation response
- `onpage_summary_<task-id>.json` - Summary data
- `onpage_pages_<task-id>.json` - Pages data
- `onpage_links_<task-id>.json` - Links data
- `onpage_resources_<task-id>.json` - Resources data
- `onpage_duplicate_tags_<task-id>.json` - Duplicate tags
- `onpage_duplicate_content_<task-id>.json` - Duplicate content
- `onpage_non_indexable_<task-id>.json` - Non-indexable pages
- `onpage_redirect_chains_<task-id>.json` - Redirect chains

## API Limits

| Limit | Value |
|-------|-------|
| Max crawl pages per task | Configurable (charged per page) |
| Max concurrent requests | 30 |
| Rate limit | 2000 requests/minute |
| Max priority URLs | 20 |
| Task retention | Results retained for collection |

## Pricing Notes

- Charged per crawl task (varies by page count and features)
- Browser rendering adds cost per page
- JavaScript execution adds cost per page
- Instant pages charged per request
- Cost displayed after each operation

## Environment Variables

Requires in `.env`:
```
DATAFORSEO_LOGIN=your_login
DATAFORSEO_PASSWORD=your_password
```

The scripts automatically load credentials from:
1. Repo root `.env` (4 directories up from script)
2. Current working directory `.env`

## Troubleshooting

**"Task not found" error:**
- Task may still be processing - wait and retry
- Use `tasks_ready.py` to check task status
- Verify task ID is correct

**"HTTP Error 401":**
- Check DATAFORSEO_LOGIN and DATAFORSEO_PASSWORD in .env
- Verify credentials are correct

**"No results returned":**
- Task may have failed during crawl
- Check summary for crawl status
- Verify target domain is accessible

**"Rate limit exceeded":**
- API enforces 2000 requests/minute
- Scripts handle rate limiting automatically
- Contact DataForSEO if persistent

## Key Metrics Explained

**OnPage Score (0-100):**
- Composite score reflecting overall page optimization
- Higher = better optimized
- < 60 typically indicates issues requiring attention

**Core Web Vitals (requires browser rendering):**
- **CLS (Cumulative Layout Shift)**: Visual stability (lower is better, < 0.1 good)
- **LCP (Largest Contentful Paint)**: Load performance (lower is better, < 2500ms good)
- **FID (First Input Delay)**: Interactivity (lower is better, < 100ms good)

**Common Checks:**
- `no_h1_tag`: Missing H1 heading
- `duplicate_title`: Title tag appears on multiple pages
- `duplicate_description`: Meta description appears on multiple pages
- `broken_links`: Page contains broken links
- `large_page_size`: Page size exceeds threshold
- `slow_page_load`: Page load time exceeds threshold
- `no_image_alt`: Images missing alt attributes
- `deprecated_html`: Uses deprecated HTML tags