Apache Spark is a distributed computing system providing APIs for large-scale data processing. It offers in-memory computing, lazy evaluation, and unified API for batch and streaming.
Key Concepts
RDDs and DataFrames
Transformations and actions
Spark SQL
Streaming (structured)
MLlib for machine learning
Common Use Cases
Big data processing
ETL pipelines
Real-time analytics
Machine learning at scale
Data lake processing
Best Practices
Use DataFrames over RDDs
Implement proper partitioning
Cache judiciously
Use broadcast variables
Monitor with Spark UI
Resources
Spark.apache.org: spark.apache.org/docs
Related Skills: data-engineering, scala, python, hadoop
1---2name: spark3description: Apache Spark4---5# Apache Spark67Apache Spark is a distributed computing system providing APIs for large-scale data processing. It offers in-memory computing, lazy evaluation, and unified API for batch and streaming.89## Key Concepts1011- RDDs and DataFrames12- Transformations and actions13- Spark SQL14- Streaming (structured)15- MLlib for machine learning1617## Common Use Cases1819- Big data processing20- ETL pipelines21- Real-time analytics22- Machine learning at scale23- Data lake processing2425## Best Practices2627- Use DataFrames over RDDs28- Implement proper partitioning29- Cache judiciously30- Use broadcast variables31- Monitor with Spark UI3233## Resources3435- Spark.apache.org: spark.apache.org/docs36- Related Skills: data-engineering, scala, python, hadoop
Run npx skillmds@latest add neuralblitz/spark in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.
Apache Spark It is listed under AI & ML on SkillMD.
This skill has not completed SkillMD's automated safety review yet. Independent scanners report: SkillSpector: PASS, Skill Scanner: PASS. Capability flags: docs only. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.
This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.
Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.
NeuralBlitz (@neuralblitz) published this skill. Their other Agent Skills are listed on their SkillMD profile.