Loyalty Segmentation Adaptation

Harvestly Loyalty Segmentation Adaptation

What this project is

The Harvestly Co. loyalty segmentation demo: a Databricks DAB that segments an 800K-member loyalty base into Champions, New Loyalists, Cooling Off, and Win-Back tiers via RFM (Recency-Frequency-Monetary) signals, and pairs each tier with playbook tactics from the Customer Marketing Playbook PDF.

Key files

• databricks.yml — bundle config; toggle run_with_synthetic_data and customer-data variables here
• src/data_generation/ — synthetic data generator (Faker-based)
• src/pipeline/ — SDP pipeline (bronze → silver → gold)
• src/deploy/ — Genie Space, KA, MAS deployers (if applicable)
• dashboard/ — Lakeview dashboard JSON (if applicable)

How to adapt

Edit targets.client.variables in databricks.yml: keep run_with_synthetic_data: "yes" for first-run (the demo runs end-to-end on synthetic data so the client experiences it immediately); set client_catalog, client_schema, and warehouse_id to values that exist in their workspace. Then deploy: databricks bundle deploy && databricks bundle run loyalty_segmentation_job. Once the client has the demo running, swap to real data by flipping run_with_synthetic_data: "no" and pointing at their own customer table.

Auto-detecting and writing workspace config — "run in my workspace"

When the user asks for help configuring databricks.yml (e.g., "run in my workspace", "set this up for my workspace", "configure for my catalog"), follow this pattern. Always default run_with_synthetic_data: "yes" so the first run works end-to-end without real data.

Step 1 — Auto-detect what you can

Run this discovery snippet in a notebook in the client's workspace:

workspace_url = spark.conf.get("spark.databricks.workspaceUrl")
current_user = spark.sql("SELECT current_user()").collect()[0][0]
current_catalog = spark.sql("SELECT current_catalog()").collect()[0][0]
current_schema = spark.sql("SELECT current_database()").collect()[0][0]
print({"workspace_url": workspace_url, "user": current_user, "current_catalog": current_catalog, "current_schema": current_schema})

For the warehouse, run SHOW WAREHOUSES (Python: spark.sql("SHOW WAREHOUSES").collect()) and pick the first running serverless warehouse — or any running warehouse if no serverless one is running.

Step 2 — Decide which values to use vs ask about

Classify each discovered value:

Always confirm before writing. Present the proposed values in a table and explicitly ask "Apply these?" rather than auto-writing.

Step 3 — Write the edit

Once the user confirms the values, use your file-editing capability to actually apply the edit to databricks.yml — don't just print a diff. Update targets.client.variables:

• client_catalog: <confirmed catalog>
• client_schema: <confirmed schema>
• warehouse_id: <confirmed warehouse ID>
• run_with_synthetic_data: "yes" ← keep this; do NOT change it on first-run

Present as an Accept/Reject UI so the client can decline.

Step 4 — Deploy

Run:

databricks bundle validate
databricks bundle deploy
databricks bundle run loyalty_segmentation_job

Confirm: validate succeeds, deploy creates the resources in the chosen catalog/schema, the synth-data job populates the bronze/silver/gold tables (customers, transactions, loyalty_redemptions, segments, segment_summary, campaign_performance), and the dashboard/Genie/agents come online. Provide the user with workspace links to inspect each created resource.

Step 5 — Re-runs and idempotency

If databricks.yml already has values that match the current workspace + the user's chosen catalog/schema (i.e., from a prior run of this skill), verify rather than blindly re-writing — say "no edits needed, ready to redeploy" and point at the deploy command.

Common gotchas

• The RFM thresholds (Champions = top 10% by spend; New Loyalists = joined <6 months & ≥3 orders; Cooling Off = last order 30–90 days; Win-Back = dormant >90 days) live as SQL constants in src/pipeline/silver/segments.sql — if the user's loyalty program has different cutoffs, propose edits there too.
• The dashboard's dataset_catalog / dataset_schema resolve from the same DAB variables — no separate dashboard edit needed.
• The Knowledge Assistant is loaded with the Customer Marketing Playbook PDF; for real-data use, the client can swap it for their own playbook in raw_data/pdf/.