agentskillexchange Scrapy Spider Architect
Generates Scrapy spider classes with CSS/XPath selectors, item pipelines, and middleware configurations for structured web scraping. Includes Scrapy-Splash integration for JavaScript-rendered content.
Installation
Use the upstream install or setup path that matches your environment:
- pip install scrapy
Requirements and caveats from upstream:
- :alt: Supported Python Versions
- It is cross-platform, and requires Python 3.10+. It is maintained by Zyte_
Basic usage or getting-started notes:
.. code:: bash
And follow the documentation_ to learn how to use it.
.. _documentation: https://docs.scrapy.org/en/latest/
Source: https://github.com/scrapy/scrapy
Extracted from upstream docs: https://raw.githubusercontent.com/scrapy/scrapy/HEAD/README.rst