Unified layers
Overview
Unified Layers provide an abstraction layer for pipelines. They standardize the details of transformed pipelines, providing a set of common functional outputs that are reusable across data models and reporting tools.
A Unified Layer stores the selected column names and types of the source pipeline. Relationships, hierarchies, and indexes on the source pipeline are not included in the Unified Layer.
When a Unified Layer is applied to a pipeline, Data Hub matches the Unified Layer columns to the pipeline columns based on their name and type. Where automatic matching is unsuccessful, Data Hub shows a warning to indicate the expected content needed to match the Unified Layers requirements.
Pipeline columns which are not mapped to Unified Layer columns are still available for reporting and all pipeline steps are part of the resulting Unified Layer.
Although Unified Layer does allow for the addition of custom columns to a pipeline for reporting, the pre-defined columns remain part of the contract.
Unified Layers provide lists of columns (and types) that a pipeline must output. A Unified Layer also adds column descriptions to the output. Each pipeline can only have a single Unified Layer applied.
A collection of Unified Layers is a Unified Data Model.
ZAP solution models use the ZAP Unified Data Model to standardize pipeline outputs making it possible for the same analytics to work across multiple models.
Unified layer definitions
Each layer contains these categories:
Layer description
Table with the following defining columns:
Column Name - The name of the column that needs to be mapped to in the unified layer.
Data Type - Data type of the unified layer column. This could be Numeric, Text, Integer or Date.
Description - A description of the column. This helps to identify which columns need to be mapped.
Keys - An indication of whether the column is a Primary key, Additional key, or neither.
Relationship table - List of relationships that the unified layer has with other layers. These relationships are defined by mappings of key columns between layers. Key columns are usually made up of the concatenation of key columns with the company column of a dimension pipeline. If the dimension is not a per company dimension, a concatenation with a blank value may be used.
General notes about the design of the unified layer.
Note
Refer to this knowledge base article for more detailed information: https://zendesk.zapbi.com/hc/en-us/articles/360001205696-Unified-Layer-definitions
Apply a Unified Layer to a Pipeline
When applying a Unified Layer to a data model pipeline, consider the following:
Only one Unified Layer can be applied to a pipeline.
A Unified Layer can be reused by creating a copy of the Unified Layer resource in RESOURCE EXPLORER.
Database details in the pipeline panel are constrained by Unified Layer.
Example: Constrained Database table name (Physical Name) based on the name of the Unified Layer.
Example: Constrained Database column name (Physical Name) based on column name stored in the Unified Layer.
Changing the original pipeline will not affect the Unified Layer or the pipelines using it.
Column name translations from the source pipeline will become available for the target pipeline.
Note
For information on using translations in Data Hub, refer to: Translating Folder Contents.
Modify Unified Layers
Unified Layers can be modified after they are created, however, editing a ZAP provided Unified Layer may negatively affect reports or other pipelines that use it.
Important
Although Unified Layers can be edited, Unified Layers act as a contract between ZAP and a data modeler.
Add columns to a Unified Layer
A Unified Layer allows for the addition of custom columns to a pipeline for reporting; however, the pre-defined columns are part of the "contract".
Remove columns from a Unified Layer
When Unified Layer columns are removed from pipeline, mapped columns and warehouse tables are released from the Unified Layer restrictions (e.g. output column name).
Modify modules on a Unified Layer
Modules may be added and removed using standard Data Hub methods.
Delete a Unified Layer
After Unified Layers have been deleted, errors will appear in the data model where the Unified Layer was still in use. Remove these errors to release the mapped columns and warehouse table from the Unified Layer restrictions (e.g. output column name).