Apache Spark is a distributed computing system providing APIs for large-scale data processing. It offers in-memory computing, lazy evaluation, and unified API for batch and streaming.
Key Concepts
RDDs and DataFrames
Transformations and actions
Spark SQL
Streaming (structured)
MLlib for machine learning
Common Use Cases
Big data processing
ETL pipelines
Real-time analytics
Machine learning at scale
Data lake processing
Best Practices
Use DataFrames over RDDs
Implement proper partitioning
Cache judiciously
Use broadcast variables
Monitor with Spark UI
Resources
Spark.apache.org: spark.apache.org/docs
Related Skills: data-engineering, scala, python, hadoop
1---2name: spark3description: Apache Spark4metadata:5 source: https://skillmd.com/skills/neuralblitz/spark6---7# Apache Spark89Apache Spark is a distributed computing system providing APIs for large-scale data processing. It offers in-memory computing, lazy evaluation, and unified API for batch and streaming.1011## Key Concepts1213- RDDs and DataFrames14- Transformations and actions15- Spark SQL16- Streaming (structured)17- MLlib for machine learning1819## Common Use Cases2021- Big data processing22- ETL pipelines23- Real-time analytics24- Machine learning at scale25- Data lake processing2627## Best Practices2829- Use DataFrames over RDDs30- Implement proper partitioning31- Cache judiciously32- Use broadcast variables33- Monitor with Spark UI3435## Resources3637- Spark.apache.org: spark.apache.org/docs38- Related Skills: data-engineering, scala, python, hadoop
Run npx get-skill add neuralblitz/spark in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.
Apache Spark It is listed under AI & ML on SkillMD.
This skill has not completed SkillMD's automated safety review yet. Independent scanners report: SkillSpector: PASS, Skill Scanner: PASS. Capability flags: docs only. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.
This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.
Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.
NeuralBlitz (@neuralblitz) published this skill. Their other Agent Skills are listed on their SkillMD profile.