Trino Airflow Lakehouse Pipelines

Airflow-orchestrated Iceberg lakehouse ETL pipelines — Bronze/Silver/Gold layer DAGs, Iceberg maintenance jobs (optimize/expire_snapshots/remove_orphan_files) as Airflow tasks, snapshot expiration scheduling, compaction DAG patterns, partition-by-partition backfill with dynamic task mapping, late-arriving data handling, watermark tracking table, post-load data quality gates (row count/freshness/null rate checks via TrinoHook), full medallion pipeline DAG example

ivanshamaev Updated 24 repo stars

File contents

ivanshamaev/de-agent-skills commit 28a78b54e8

Frequently asked questions

npx get-skill add ivanshamaev/trino-airflow-lakehouse-pipelines