Apify Market Research

Market Research

Conduct market research using Apify Actors to extract data from multiple platforms.

Prerequisites

(No need to check it upfront)

• .env file with APIFY_TOKEN
• Node.js 20.6+ (for native --env-file support)
• mcpc CLI tool: npm install -g @apify/mcpc

Workflow

Copy this checklist and track progress:

Task Progress:
- [ ] Step 1: Identify market research type (select Actor)
- [ ] Step 2: Fetch Actor schema via mcpc
- [ ] Step 3: Ask user preferences (format, filename)
- [ ] Step 4: Run the analysis script
- [ ] Step 5: Summarize findings

Step 1: Identify Market Research Type

Select the appropriate Actor based on research needs:

Step 2: Fetch Actor Schema

Fetch the Actor's input schema and details dynamically using mcpc:

export $(grep APIFY_TOKEN .env | xargs) && mcpc --json mcp.apify.com --header "Authorization: Bearer $APIFY_TOKEN" tools-call fetch-actor-details actor:="ACTOR_ID" | jq -r ".content"

Replace ACTOR_ID with the selected Actor (e.g., compass/crawler-google-places).

This returns:

• Actor description and README
• Required and optional input parameters
• Output fields (if available)

Step 3: Ask User Preferences

Before running, ask:

1. Output format:
   • Quick answer - Display top few results in chat (no file saved)
   • CSV - Full export with all fields
   • JSON - Full export in JSON format
2. Number of results: Based on character of use case

Step 4: Run the Script

Quick answer (display in chat, no file):

node --env-file=.env ${CLAUDE_PLUGIN_ROOT}/reference/scripts/run_actor.js \
  --actor "ACTOR_ID" \
  --input 'JSON_INPUT'

CSV:

node --env-file=.env ${CLAUDE_PLUGIN_ROOT}/reference/scripts/run_actor.js \
  --actor "ACTOR_ID" \
  --input 'JSON_INPUT' \
  --output YYYY-MM-DD_OUTPUT_FILE.csv \
  --format csv

JSON:

node --env-file=.env ${CLAUDE_PLUGIN_ROOT}/reference/scripts/run_actor.js \
  --actor "ACTOR_ID" \
  --input 'JSON_INPUT' \
  --output YYYY-MM-DD_OUTPUT_FILE.json \
  --format json

Step 5: Summarize Findings

After completion, report:

• Number of results found
• File location and name
• Key market insights
• Suggested next steps (deeper analysis, validation)

Error Handling

APIFY_TOKEN not found - Ask user to create .env with APIFY_TOKEN=your_token mcpc not found - Ask user to install npm install -g @apify/mcpc Actor not found - Check Actor ID spelling Run FAILED - Ask user to check Apify console link in error output Timeout - Reduce input size or increase --timeout

AGI Framework Integration

Adapted for @techwavedev/agi-agent-kit Original source: antigravity-awesome-skills

Memory-First Protocol

Retrieve prior API design decisions, database schema choices, and error handling patterns. Cache API response templates for consistent error formatting.

# Check for prior backend/API context before starting
python3 execution/memory_manager.py auto --query "API design patterns and architecture decisions for Apify Market Research"

Storing Results

After completing work, store backend/API decisions for future sessions:

python3 execution/memory_manager.py store \
  --content "API architecture: REST with HATEOAS, JWT auth, rate limiting at 100 req/min per tenant" \
  --type decision --project <project> \
  --tags apify-market-research backend

Multi-Agent Collaboration

Share API contract changes with frontend agents so they update their client code, and with QA agents for test coverage.

python3 execution/cross_agent_context.py store \
  --agent "<your-agent>" \
  --action "Implemented API endpoints — 5 new routes with OpenAPI spec and integration tests" \
  --project <project>

Agent Team: Code Review

After implementation, dispatch code_review_team for two-stage review (spec compliance + code quality) before merging.