Data Vault on Lakeflow

When you model a Data Vault in DeltaVault, the Databricks generator turns each object into a load notebook that builds it, so the model you design on the canvas becomes deployable code. Each notebook runs one idempotent statement against the object’s target table. This guide explains what each object becomes. To build the model itself, see Data Vault modeling.

Hubs and links: insert-only loads

A hub holds the unique list of business keys for a concept, and a link holds the relationships between hubs. DeltaVault generates a load notebook for each, both insert-only: the notebook merges on the object’s hash key and inserts only the keys that are not there yet, so a rerun adds nothing twice.

A hub projects its hash key, its business keys, the load date, and the record source.
A link projects its own hash key, the hash keys of the hubs it connects, the load date, and the record source.

Because hubs and links only ever gain new rows, they need no change tracking.

How link hash keys are named

A link needs its own hash key plus the hash key of every hub it connects, and those columns have to exist in the staging table that feeds the link before Databricks can build it.

For a source table that feeds a link, staging generates one hash key column per hub the link connects to, named after that hub (for example customer_hk, order_hk), plus the link’s own hash key column, named after the link itself.

Each per-hub column has to carry the same value the hub was keyed on, so staging re-instantiates the parent hub’s business-key definition on the link’s own columns: it rebuilds the hub’s designated business-key expression, in the hub’s key order and with its separator and transforms, over the link table’s aligned foreign-key columns. Because the definition is re-instantiated rather than re-keyed by hand, the link’s hash matches the hub’s exactly. If the link’s foreign-key columns cannot be aligned to the hub’s key, the build fails loudly rather than emitting a mismatched key; see Business key column for how that alignment is checked and repaired. The link’s own column hashes across every hub’s business keys the link connects, in a fixed order set when the link is created. The link table selects these staging columns directly: they are the hash keys referred to in “its own hash key, the hash keys of the hubs it connects” above, not values the link recomputes itself.

Hub and satellite tables are unaffected by this naming: they keep the constant hash_key column name they have always used, so an already-deployed hub or satellite never needs to rename anything. Only the extra staging columns that feed a link, and the link’s own selected column, use the pattern-based names. The naming pattern itself is a setting in Settings > Data Vault, the Hash key column pattern field, {{this}}_HK by default.

Satellites: keeping history by appending versions

A satellite holds the changing attributes that hang off a hub or a link. DeltaVault generates each satellite as a versioned insert load: the notebook matches on the parent’s hash key together with the load date, and inserts a row only when that combination is absent, so a genuine change appends a new version and an unchanged row adds nothing. Because no row is ever updated or deleted, the satellite keeps the full history of every key, which is what a type 2 dimension gives you when you query it.

The business vault

The business vault holds objects you derive from raw vault data with business rules. DeltaVault generates business hubs, business links, and business satellites with the same shapes as their raw counterparts: business hubs and business links as insert-only loads, and business satellites as versioned inserts that append a new version per change.

Reference tables

Reference tables hold shared code lists and lookups. DeltaVault generates each as an upsert load keyed on the code: the notebook updates the row when the code is already there and inserts it when it is not, so the table always holds the current value of each code and no history.

Point-in-time and bridge objects

Point-in-time and bridge objects make a Data Vault easier to query:

A point-in-time object gathers the current rows of a hub and its satellites into one place, so you do not have to join across every satellite at query time.
A bridge object pre-joins a hub through its links to related hubs, so common traversals are ready to read.

DeltaVault generates both as full rebuild loads: each notebook recomputes the whole object from the current rows of what it summarizes and overwrites the target table with the result. They are tables that are rebuilt on every run, not incremental views, so a rerun is safe and always leaves the object consistent with the vault it was built from.

Where each object lands

Every Data Vault object is written under src/data_vault/loads in the generated bundle, one file per object. Alongside them, src/data_vault/control holds one control notebook per staging source, which runs that source’s hubs and links first and then its satellites and link satellites. The generated Data Vault workflow chains those control notebooks, then the point-in-time, bridge, and business vault objects. See Orchestration workflows for how that ordering is built.