Apify Ecommerce

E-commerce Data Extraction

Extract product data, prices, reviews, and seller information from any e-commerce platform using Apify's E-commerce Scraping Tool.

Prerequisites

• .env file with APIFY_TOKEN (at ~/.claude/.env)
• Node.js 20.6+ (for native --env-file support)

Workflow Selection

Progress Tracking

Task Progress:
- [ ] Step 1: Select workflow and determine data source
- [ ] Step 2: Configure Actor input
- [ ] Step 3: Ask user preferences (format, filename)
- [ ] Step 4: Run the extraction script
- [ ] Step 5: Summarize results

Workflow 1: Products & Pricing

Use case: Extract product data, prices, and stock status. Track competitor prices, detect MAP violations, benchmark products, or research markets.

Best for: Pricing analysts, product managers, market researchers.

Input Options

Example - Product URLs

{
  "detailsUrls": [
    {"url": "https://www.amazon.com/dp/B09V3KXJPB"},
    {"url": "https://www.walmart.com/ip/123456789"}
  ],
  "additionalProperties": true
}

Example - Keyword Search

{
  "keyword": "Samsung Galaxy S24",
  "marketplaces": ["www.amazon.com", "www.walmart.com"],
  "additionalProperties": true,
  "maxProductResults": 50
}

Optional: AI Summary

Add these fields to get AI-generated insights:

Example with AI summary:

{
  "keyword": "robot vacuum",
  "marketplaces": ["www.amazon.com"],
  "maxProductResults": 50,
  "additionalProperties": true,
  "fieldsToAnalyze": ["name", "offers", "brand"],
  "customPrompt": "Summarize price range and identify top brands"
}

Output Fields

• name - Product name
• url - Product URL
• offers.price - Current price
• offers.priceCurrency - Currency code (may vary by seller region)
• brand.slogan - Brand name (nested in object)
• image - Product image URL
• Additional seller/stock info when additionalProperties: true

Note: Currency may vary in results even for US searches, as prices reflect different seller regions.

Workflow 2: Customer Reviews

Use case: Extract reviews for sentiment analysis, brand perception monitoring, or quality issue detection.

Best for: Brand managers, customer experience teams, QA teams, product managers.

Input Options

Example - Extract Reviews from Product

{
  "reviewListingUrls": [
    {"url": "https://www.amazon.com/dp/B09V3KXJPB"}
  ],
  "sortReview": "Most recent",
  "additionalReviewProperties": true,
  "maxReviewResults": 500
}

Example - Keyword Search

{
  "keywordReviews": "wireless earbuds",
  "marketplacesReviews": ["www.amazon.com"],
  "sortReview": "Most recent",
  "additionalReviewProperties": true,
  "maxReviewResults": 200
}

Sort Options

• Most recent - Latest reviews first (recommended)
• Most relevant - Platform default relevance
• Most helpful - Highest voted reviews
• Highest rated - 5-star reviews first
• Lowest rated - 1-star reviews first

Note: The sortReview: "Lowest rated" option may not work consistently across all marketplaces. For quality analysis, collect a large sample and filter by rating in post-processing.

Quality Analysis Tips

• Set high maxReviewResults for statistical significance
• Look for recurring keywords: "broke", "defect", "quality", "returned"
• Filter results by rating if sorting doesn't work as expected
• Cross-reference with competitor products for benchmarking

Workflow 3: Seller Intelligence

Use case: Find sellers across stores, discover unauthorized resellers, evaluate vendor options.

Best for: Brand protection teams, procurement, supply chain managers.

Note: This workflow uses Google Shopping to find sellers across stores. Direct seller profile URLs are not reliably supported.

Input Configuration

{
  "googleShoppingSearchKeyword": "Nike Air Max 90",
  "scrapeSellersFromGoogleShopping": true,
  "countryCode": "us",
  "maxGoogleShoppingSellersPerProduct": 20,
  "maxGoogleShoppingResults": 100
}

Options

Supported Marketplaces

Amazon (20+ regions)

www.amazon.com, www.amazon.co.uk, www.amazon.de, www.amazon.fr, www.amazon.it, www.amazon.es, www.amazon.ca, www.amazon.com.au, www.amazon.co.jp, www.amazon.in, www.amazon.com.br, www.amazon.com.mx, www.amazon.nl, www.amazon.pl, www.amazon.se, www.amazon.ae, www.amazon.sa, www.amazon.sg, www.amazon.com.tr, www.amazon.eg

Major US Retailers

www.walmart.com, www.costco.com, www.costco.ca, www.homedepot.com

European Retailers

allegro.pl, allegro.cz, allegro.sk, www.alza.cz, www.alza.sk, www.alza.de, www.alza.at, www.alza.hu, www.kaufland.de, www.kaufland.pl, www.kaufland.cz, www.kaufland.sk, www.kaufland.at, www.kaufland.fr, www.kaufland.it, www.cdiscount.com

IKEA (40+ country/language combinations)

Supports all major IKEA regional sites with multiple language options.

Google Shopping

Use for seller discovery across multiple stores.

Running the Extraction

Step 1: Set Skill Path

SKILL_PATH=~/.claude/skills/apify-ecommerce

Step 2: Run Script

Quick answer (display in chat):

node --env-file=~/.claude/.env $SKILL_PATH/reference/scripts/run_actor.js \
  --actor "apify/e-commerce-scraping-tool" \
  --input 'JSON_INPUT'

CSV export:

node --env-file=~/.claude/.env $SKILL_PATH/reference/scripts/run_actor.js \
  --actor "apify/e-commerce-scraping-tool" \
  --input 'JSON_INPUT' \
  --output YYYY-MM-DD_filename.csv \
  --format csv

JSON export:

node --env-file=~/.claude/.env $SKILL_PATH/reference/scripts/run_actor.js \
  --actor "apify/e-commerce-scraping-tool" \
  --input 'JSON_INPUT' \
  --output YYYY-MM-DD_filename.json \
  --format json

Step 3: Summarize Results

Report:

• Number of items extracted
• File location (if exported)
• Key insights based on workflow:
  • Products: Price range, outliers, MAP violations
  • Reviews: Average rating, sentiment trends, quality issues
  • Sellers: Seller count, unauthorized sellers found

Error Handling

AGI Framework Integration

Adapted for @techwavedev/agi-agent-kit Original source: antigravity-awesome-skills

Memory-First Protocol

Retrieve prior API design decisions, database schema choices, and error handling patterns. Cache API response templates for consistent error formatting.

# Check for prior backend/API context before starting
python3 execution/memory_manager.py auto --query "API design patterns and architecture decisions for Apify Ecommerce"

Storing Results

After completing work, store backend/API decisions for future sessions:

python3 execution/memory_manager.py store \
  --content "API architecture: REST with HATEOAS, JWT auth, rate limiting at 100 req/min per tenant" \
  --type decision --project <project> \
  --tags apify-ecommerce backend

Multi-Agent Collaboration

Share API contract changes with frontend agents so they update their client code, and with QA agents for test coverage.

python3 execution/cross_agent_context.py store \
  --agent "<your-agent>" \
  --action "Implemented API endpoints — 5 new routes with OpenAPI spec and integration tests" \
  --project <project>

Agent Team: Code Review

After implementation, dispatch code_review_team for two-stage review (spec compliance + code quality) before merging.