Writing Idempotent Transformations

Write data transformations and loads that produce the same result no matter how many times they run — using MERGE/upsert, deterministic partition overwrites, deduplication, and stable keys. Use when a retry could duplicate data, a job is not safe to re-run, a pipeline needs exactly-once effects, or loads must be backfill-safe.

Unknown-333 Updated 26 repo stars
Awesome Data Engineering Skills — data-engineering skills for your AI coding agent

File contents

Unknown-333/awesome-data-engineering-skills commit 9b3b111cfe

Frequently asked questions

npx get-skill add unknown-333/writing-idempotent-transformations