# Data Pipeline Engineering > ETL workflow design, automated data fetching, version tracking, and pipeline orchestration expertise Use when this capability is needed. - Skill: `tomevault-io/data-pipeline-engineering` (Agent Skill, multi-file: 2 files) - Install (CLI): `npx skillmds@latest add tomevault-io/data-pipeline-engineering` - Raw SKILL.md: https://api.skillmd.com/api/skills/tomevault-io/data-pipeline-engineering/raw - Safety review: pending (external: skill-scanner PASS, skillspector PASS) - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: DevOps & Infra - Author: tomevault-io (https://skillmd.com/u/tomevault-io) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/tomevault-io/data-pipeline-engineering --- # Data Pipeline Engineering Skill ## Purpose Expert knowledge in designing robust ETL (Extract, Transform, Load) pipelines for automated data processing, focusing on reliability, monitoring, and maintainability. ## Core Principles 1. **Idempotency** - Pipeline runs produce same results 2. **Observability** - Full visibility into pipeline health 3. **Error Recovery** - Graceful handling of failures 4. **Version Tracking** - Track all data changes 5. **Monitoring** - Real-time pipeline health checks ## Enforces - ETL workflow patterns (Extract → Transform → Load) - Automated scheduling (cron, GitHub Actions) - Data versioning and archival - Pipeline health monitoring - Error recovery strategies - Audit logging ## When to Use - Building automated data pipelines - Scheduling data fetching workflows - Implementing data versioning - Monitoring pipeline health - Designing error recovery ## References - [GitHub Actions](https://docs.github.com/en/actions) - [ETL Best Practices](https://en.wikipedia.org/wiki/Extract,_transform,_load) --- **Version**: 1.0 | **Last Updated**: 2026-02-06 | **Category**: Development & Operations --- > Converted and distributed by [TomeVault](https://tomevault.io/claim/hack23) — claim your Tome and manage your conversions.