# Airflow

> Apache Airflow workflow orchestration. Use for data pipelines.

- Skill: `g1joshi/airflow` (Agent Skill)
- Install (CLI): `npx skillmds@latest add g1joshi/airflow`
- Raw SKILL.md: https://api.skillmd.com/api/skills/g1joshi/airflow/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Productivity
- Author: G1Joshi (https://skillmd.com/u/g1joshi)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/g1joshi/airflow

---


# Airflow

Apache Airflow is the standard for data engineering pipelines. v3.0 (2025) introduces **Event-driven Triggers** and a modern React UI.

## When to Use

- **ETL/ELT**: Scheduling nightly data warehouse loads.
- **ML Ops**: Retraining models when new data arrives.
- **Dependency Management**: "Run Task B only if Task A succeeds".

## Core Concepts

### DAGs (Directed Acyclic Graphs)

Defined in Python.

### Task SDK

New in v3.0. Allows writing tasks in any language, not just Python.

### Edge Executor

Run tasks on remote edge devices.

## Best Practices (2025)

**Do**:

- **Use the TaskFlow API**: `@task` decorators are cleaner than `PythonOperator`.
- **Use Datasets**: Define data-aware scheduling (`schedule=[Dataset("s3://bucket/file")]`).

**Don't**:

- **Don't put top-level code in DAG files**: It runs every scheduler heartbeat.

## References

- [Airflow Documentation](https://airflow.apache.org/)