# Context Scrapers

> Web scraping spiders (Scrapy), parsers, and source management.

- Skill: `majiayu000/context-scrapers` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds add majiayu000/context-scrapers`
- Raw SKILL.md: https://api.skillmd.com/api/skills/majiayu000/context-scrapers/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: majiayu000 (https://skillmd.com/u/majiayu000)
- Updated: 2026-09-09
- Page: https://skillmd.com/skills/majiayu000/context-scrapers

---


# Scrapers Context

## Overview
Web scraping infrastructure using Scrapy. Handles data acquisition from city councils (Legistar) and municipal codes (Municode).

## Active Files

### Spiders
- `backend/affordabot_scraper/affordabot_scraper/spiders/sanjose_meetings.py` - Legistar meeting scraper
- `backend/affordabot_scraper/affordabot_scraper/spiders/sanjose_municode.py` - Municode scraper

### Configuration
- `backend/affordabot_scraper/affordabot_scraper/pipelines.py` - Item pipelines (DB storage)
- `backend/affordabot_scraper/affordabot_scraper/settings.py` - Scrapy settings

### Verification Scripts
- `scripts/verify_raw_scrapes.py` - Check raw_scrapes table
- `scripts/verify_municode_discovery.py` - Verify discovery logic

## Usage
Use this skill when modifying scrapers, adding new jurisdictions, or debugging ingestion.