Refresh Warehouse data or structure
Once a model has been configured, it needs to be processed. This creates the corresponding data warehouse and optionally the semantic layer. You can process the entire Model or specific Model Pipelines.
Process an entire model
On the design panel, click Process Model.
The Process screen appears.
If any publishable changes are present, use the Publishchanges check box to determine whether or not you want to include them in the process.
Note
The Publish changes check box and the list of publishable changes, only appear on the Process pop-up if these types of changes are detected in your model. Otherwise, this information is hidden.
Specify the Processwarehouse and Process cube options.
Notice that All appears in the Pipelines and Database Scripts text boxes, since all of your pipelines and scripts are automatically processed when you process the entire model.
Specify the incremental option for the model using the Incremental Data Load setting.
Default Incremental - Allows you to load both new and updated data to the staging database. All data sources and pipelines in the model automatically map to the Add & Update incremental load option.
Quick Incremental - Allows you to only load new data to the staging database, and process the model with only this newly added data. All data sources and pipelines in the model automatically map to the Add Only incremental load option
Full Refresh - Performs a full (non-incremental) data refresh. This slower option may need to be performed periodically to ensure integrity.
Note
If desired, edit the automatic incremental mapping, at either the data source or pipeline level.
Click Process.
On the model Properties panel, examine the Status area to observe the overall process steps as they are completed.
Note
If desired, edit the automatic incremental mapping, at either the data source or pipeline level.
Verify that all tasks are showing check marks to indicate that they have been completed successfully.
Note
If the final task (Update Published Model) is checked, the overall process was also successful.
If errors occur you may use a previously published version to resolve them.
Process specific pipelines
Process individual pipelines or groups of pipelines, instead of processing all pipelines (e.g. when an entire model is processed) by either of the following methods:
Select pipelines from the Pipelines list on the model tab.
Specify pipelines on the Process screen.
Select pipelines from pipelines list
Select pipelines for processing from the Pipelines list on a model's tab.
Select the pipeline (or pipelines) from the Pipelines list.
Click Process Pipelines.
Note
This button is not accessible until you select at least one pipeline from the Pipelines list.
The Process screen appears. Notice that the selected pipelines automatically appear in the Pipelines text box.
Note
You can also type pipelines directly into the Pipelines text box if you forgot to select from the Pipelines list.
If any publishable changes are present, use the Publish changes check box to determine whether or not you want to include them in the process.
Note
The Publish changes check box and the list of publishable changes will only appear on the Process dialog box if these types of changes are detected in your model.
Specify the Processwarehouse, Process cube, Pipelines, Database Scripts, and Incremental Data Load options.
Click Process.
On the Properties panel, examine the Status area to observe the process steps as they are completed. Expand listed tasks to view related pipelines. All processed pipelines are listed first, followed by pipelines that are currently processing (marked by the spinning gear icon), and lastly those pipelines whose processing has not yet started.
Verify that all tasks are completed successfully.
Note
If the final task (Update Published Model) shows a checkmark, the overall process has been successful. If errors occur, you may be able to use a previously published version to resolve them.
Remove individual pipelines
If you have added a pipeline to the Process screen or Properties panel, but you decide that you do not want to process it, remove it with the "x" button that appears when you hover over the pipeline entry in the corresponding text box.
Add a process configuration
Process configurations allow you to configure how a model is processed. They can also be used to set up scheduled background tasks to periodically refresh the model.
From the model's tab, expand the Processing Configurations list.
Any previously defined model process configurations are listed.
Click the +Process Configuration button. A new model process configuration is added, and its default properties appear in the Properties panel.
(Optional) In the Properties panel, specify a name for the model process configuration, replacing the default New Configuration name.
In the Properties panel Options area, specify what will be processed via the configuration.
Specify the Processwarehouse and Process cube settings.
To process specific pipelines, use one of the following methods:
Type a pipeline directly in the Pipeline text box. As you type, pipelines with matching names appear for selection.
Drag pipelines from the Resource Explorer and drop them into the Pipelines text box.
Note
Remove pipelines from inclusion in the process, using the small "x" button.
To process specific database scripts:
Repeat the process used for pipelines in the Database Scripts text box.
Specify the Incremental DataLoad option.
In the Properties panel Advanced area, specify if you want to override the model's parallel settings. To override these settings, specify the Max source reading operations and Max result writing operations settings. This will allow you to control parallel pipeline processing.
Scheduling options:
In the Properties panel Schedule area, specify when a model will be processed.
Note
This defined schedule is executed as an internal Data Hub process.
Click the Schedule check box to activate all of the schedule settings. If the process configuration will only be executed manually, leave the Schedule check box cleared.
Note
Clearing this check box is a way to temporarily prevent process configuration from running according to schedule.
Specify Daily or Interval execution as individual settings for these two execution options.
Note
Any specified days/times are local to the webserver hosting Data Hub.
Email options:
In the Properties panel Email area, specify if email notifications will be sent based on the results of the related processes using the Email On drop-down list.
Never. Emails are never sent for the processes executed via the process configuration.
Failure. An email is only sent when processes executed via the process configuration are not successful.
All processes. An email is sent whenever a process is executed via the process configuration, regardless of the outcome of the process (success or failure).
The generated email message is automatically sent to the defined model owner. The current model owner can be seen immediately below the Email On drop-down.
Note
Change model owner by using Model Owners setting in the Processing area of model Properties panel.
Examine the process configuration in the list on the model tab. Notice that the properties you selected are listed in the Options column, including whether or not the warehouse and cube will be processed, any specified warehouse and cube options, and if any pipelines are included. In addition, the Schedule column lists the defined processing schedule.
Execute the model process configuration.
In the Processing list, right-click the process configuration.
Select Process from the menu.
Delete any existing model process configuration by right-clicking it, and selecting Delete from the menu that appears.