June 2023

Partitioned processing: During model processing, migrating only periods of active source information based on date partitions, assists in better managing large data sets.

When the Active partition period is set to Month, and Active partitions of 4, all data for the four months leading up to the current date will be migrated. If current date is 15 June, all data from 1 March to 15 June will be migrated. Detailed information about Partitioned processing and configuration

Historical Partitions: Once the active periods of data have been migrated, the remainder of historic source information will be migrated using the same Partition period (year as seen below), until an empty partition is found. The rest of the data is then loaded as a bulk historic partition.

Configure Empty Bulk Historical threshold: Sparsely distributed data may preemptively trigger the bulk historical partition, particularly when partitions are configured to be a day in size. This setting raises the threshold for a bulk load from the default single empty partition and should be used if the bulk partition is failing to migrate or is abnormally large compared to other partitions.