Cloud release notes November 2020

Changes to using a REST API as a data source

Additional Retry-After policy

The Retry-After response HTTP header indicates how long Data Hub should wait before making a follow-up request. Data Hub will retry on the following cases where this header is used:

When sent with a 503 (Service Unavailable) response, this indicates how long the service is expected to be unavailable.
When sent with a 429 (Too Many Requests) response, this indicates how long to wait before making a new request.

Tip

Data Hub has a built-in retry policy; however, you can now manually set an additional policy using one status code and a fixed retry delay to circumvent this issue.

How to

When creating a new model from a REST data source, if you receive an error on the Select Data step, return to the Connect step and complete the following:

Enter the status error code into the Retry Status Code text box.
Enter a digit into the Retry Delay (seconds) text box.
Return to Select Data, and the table will not show the previous error.

Endpoint enhancements

Some endpoints have too much data for a single response. For these scenarios, endpoints support returning multiple pages of data. In Postman, you can specify that the endpoint uses pagination and its type. Data Hub supports three types:

SkipCount - replaces the row the source is up to, e.g. if you reached the end of the first page after reading 1000 rows, the source would replace with 1001.
PageCount - replaces the page the source is up to, which is usually the same as replacing with SkipCount / page size.
NextURL - is used when the endpoint returns the full address for the next page and needs a path to the next page returned.

How to

Import your file to Postman.
Open each endpoint and adjust as needed.
Click Send.
Create your new model in Data Hub using REST Data Source.
After you have created your model, check the preview pane for each table to ensure data are present.