Click HERE to see how Saviynt Intelligence is transforming the industry. |
07/17/2023 06:15 AM - edited 07/17/2023 06:17 AM
For the Rest connector, how do I configure the Offset Pagination totalCountPath attribute for the example response (see below) where the response contains the total number of records in the attribute "totalResults". Next Page URLs are not available for this endpoint, so have to use Offset based pagination.
From testing various permutations, the Java Script capability does not seem to be enabled for the totalCountPath config attribute, log indicates total count = null for the below tried values.
Running the account job and monitoring the logs shows that the first 100 records are retrieved but the log also shows "DEBUG rest.RestUtilService - totalCount: null". And the paging stops and does not continue.
If I hard code a value such in the pagination config the logs show and pagination continues to the next batch of accounts, logs:
Got Webservice API Response: [headers:[Date: Mon, 17 Jul 2023 10:55:06 GMT, Content-Type: application/json, Transfer-Encoding: chunked, Connection: keep-alive, Max-Forwards: 20, X-CorrelationID: Id-0a1eb564324b24xyzxyz79ee 0; Id-0a1eb564334bf97e79xyzxyz 0; Id-0a1eb564344b7a6bzyxzyx 0, Cache-Control: no-cache,no-store,must-revalidate, Content-Security-Policy: default-src 'none'; script-src 'none'; object-src 'none'; frame-ancestors 'self'; frame-src 'none'; require-trusted-types-for 'script', HC-TransactionId: Id-0a1eb564324b24e5a6df79ee, Referrer-Policy: origin-when-cross-origin, Via: HiltiCloud, X-Content-Type-Options: nosniff, X-Frame-Options: SAMEORIGIN], responseText:{\"totalResults\":15806,\"itemsPerPage\":100,\"startIndex\":14100,\"schemas\":[\"urn:ietf:params:scim:api:messages:2.0:ListResponse\"],\"Resources\":[{\"schemas\":[\"urn:ietf:params:scim:schemas:core:2.0:User\"],\"id\":\"xyzxyzxyxzyxzyxzy\",\"nickName\":\"DXYZXYZYXZ\",\"active\":true}
Solved! Go to Solution.
07/17/2023 09:22 PM
07/17/2023 10:14 PM
This had failed as well, Rushikesh.
07/19/2023 12:26 PM
Try below pagination block
"pagination": {
"offset": {
"offsetParam": "startIndex",
"batchParam": "count",
"batchSize": 100,
"totalCountPath": "completeResponseMap.totalResults"
}
}
07/19/2023 09:15 PM
Hi Saathvik, this has already been tried and is not working as expected. The list of values already tried are mentioned in the main query above.
07/19/2023 09:18 PM
What if you hardcode max number ? 100000
07/20/2023 11:25 AM
Did you try exact same block? Why I am asking this because in the screenshot you shared I see two totalCountPath variables and highlighted block is in totalCountPath as well.
07/20/2023 05:49 PM
Please share full json
07/20/2023 09:29 PM
Thank you Rushikesh and Saathvik for your responses.
The issue has now been resolved.
Issue: The API return error 400 in the first call, as it does not support startIndex=0, the offset value for startIndex should be 1 (startIndex=1). Hence, on specifying the the totalCountPath as a fixed value it moves to the next page but was erroring out on using totalCountPath from the response object.
Solution: Initial URL set with '?startIndex=1&count=100' and pagination logic as :-
"pagination": {
"nextUrl": {
"nextUrlPath": "${response?.completeResponseMap?.itemsPerPage<100?null:'https://sample-url.com/Groups?count=100&startIndex='+Math.addExact(response.completeResponseMap.star...)}"
}
}