Sdp Meta

Use this skill for metadata-driven Databricks pipelines built with SDP-META (the framework in this repo; formerly DLT-META, package databricks-labs-sdp-meta, CLI `databricks labs sdp-meta`). Triggers: (1) 'sdp-meta', 'sdp meta', 'dlt-meta', 'dlt meta', or 'metadata-driven pipeline'; (2) onboarding a dataflowspec / writing an onboarding JSON/YAML that describes bronze/silver tables as metadata; (3) generating many bronze/silver Lakeflow SDP pipelines from config instead of hand-writing pipeline code; (4) `sdp-meta onboard`, `sdp-meta deploy`, or `sdp-meta bundle-init/bundle-add-flow/bundle-validate`; (5) scaffolding an sdp-meta Asset Bundle (DAB). This is the config-driven LAYER ON TOP OF Lakeflow Spark Declarative Pipelines — for hand-authored streaming tables / materialized views, that is plain SDP, not this framework.

databrickslabs Updated 73 repo stars
SDP-META — Metadata-driven Lakeflow Spark Declarative Pipelines

File contents

databrickslabs/sdp-meta commit 2af67196a9

Frequently asked questions

npx get-skill add databrickslabs/sdp-meta