Spark Python Data Source

Build custom Python data sources for Apache Spark using the PySpark DataSource API — batch and streaming readers/writers for external systems. Use this skill whenever someone wants to connect Spark to an external system (database, API, message queue, custom protocol), build a Spark connector or plugin in Python, implement a DataSourceReader or DataSourceWriter, pull data from or push data to a system via Spark, or work with the PySpark DataSource API in any way. Even if they just say "read from X in Spark" or "write DataFrame to Y" and there's no native connector, this skill applies.

databrickslabs Updated 73 repo stars
CODA demo — splash screen, multi-tab terminals, keyboard shortcuts

File contents

databrickslabs/coding-agents-databricks-apps commit 4f90c60c21

Frequently asked questions

npx get-skill add databrickslabs/spark-python-data-source