Rehashing Datavault4dbt Entities

Rehashing datavault4dbt entities

When you change anything that affects hash output — the algorithm (MD5→SHA2), hash_datatype, hashdiff_use_trim, or case-sensitivity vars — every hashkey and hashdiff in the Raw Data Vault must change too, or referential integrity breaks. You have two options:

1. Full refresh the RDV (only if the full source history is still available to reload), or
2. Rehash existing entities in place with the package's rehashing macros (when history is not reloadable).

This skill is about option 2.

v1 → v2.0.0 upgrade. v2.0.0 unifies/corrects hash standardization: with default settings, hashkeys are now UPPER-normalized and hashdiffs are no longer UPPER-normalized, plus adapter-specific changes (Oracle/Redshift concat-string alignment, BigQuery MD5 fix, Fabric/Exasol standardization). Hash values differ from v1, so crossing this boundary on an existing vault requires a full refresh or a rehash.

The safe workflow (always)

1. Start small — rehash one test entity first and verify the configuration and logic.
2. Overwrite, don't drop — set overwrite_hash_values: true and keep drop_old_values: false. The macros rename the old columns with a _deprecated suffix instead of deleting them.
3. Validate — compare the new hash columns against the _deprecated columns.
4. Clean up — only after validation, drop the deprecated columns. The dbt log prints the dict of columns to drop; use it to build a cleanup model or call datavault4dbt.custom_alter_relation_add_remove_columns.

Single-entity rehash (surgical)

Run a one-off operation against a single hub/link/satellite:

dbt run-operation rehash_single_hub --args '{
    hub: customer_h,
    hashkey: HK_CUSTOMER_H,
    business_keys: C_CUSTKEY,
    overwrite_hash_values: true
}'

Bulk rehash by entity type (YAML-driven)

Create a dedicated model that calls the bulk macro with YAML metadata:

-- models/rehash/rehash_hubs.sql
{{ config(materialized='view') }}
{% set hub_yaml %}
config:
    overwrite_hash_values: true
hubs:
    - name: customer_h
      hashkey: hk_customer_h
      business_keys: [c_custkey]
    - name: order_h
      hashkey: hk_order_h
      business_keys: [order_id]
{% endset %}

{{ datavault4dbt.rehash_hubs(hub_yaml=hub_yaml, drop_old_values=false) }}

SELECT 'success' as status

dbt run -s rehash_hubs

The hub then holds both the new hashkey and the _deprecated old one, so you can validate before cleanup.

Full RDV rehash (ordered)

rehash_all_rdv_entities processes the whole vault in the correct order — hubs first, then links (aligned to the new hub hashkeys), then satellites (standard, multi-active, non-historized; recalculating hashkeys and hashdiffs):

-- models/rehash/rehash_entire_rdv.sql
{{ config(materialized='view') }}
{% set entity_yaml %}
config:
    overwrite_hash_values: true
hubs:
  - name: customer_h
    hashkey: hk_customer_h
    business_keys: [c_custkey]
links:
  - name: customer_order_l
    link_hashkey: hk_customer_order_l
    hub_config:
        - hub_name: customer_h
          hub_hashkey: hk_customer_h
          business_keys: [c_custkey]
satellites:
  - name: customer_s
    hashkey: hk_customer_h
    hashdiff: hd_customer_s
    parent_entity: customer_h
    payload: [c_name, c_address]
{% endset %}

{{ datavault4dbt.rehash_all_rdv_entities(entity_yaml=entity_yaml, drop_old_values=false) }}

SELECT 'success' as status

dbt run -s rehash_entire_rdv

Common mistakes

Authoritative reference: dbt_packages/datavault4dbt/docs/26_general-usage-notes/41_rehashing.