We are delighted to share our new EIC Delivery Methodology for efficiently managing Saviynt Implementations and delivering quick time to value. CLICK HERE.

Pagination in REST connector

reshma_sonawane
New Contributor III
New Contributor III

Hi Team,

We are trying to import accounts for one of our REST application. In response or headers, there are no details available about next page. Below is the pagination supported in url by application.

https://*******/user/search?$$FIRST=0&$$LIMIT=2000&filters={"isActive":"true"}&apiKey=****&fields=*****

With this call, it will fetch record 1 to record 2000. For next page url should be,

https://*******/user/search?$$FIRST=2000&$$LIMIT=2000&filters={"isActive":"true"}&apiKey=****&fields=*****

Maximum records that can be fetched in single call is 2000.

We are trying to do pagination for this case. We tried few ways as below but nothing worked:

Case 1:

 "pagination": { "nextUrl": { "nextUrlPath": "${response?.completeResponseMap?.itemsPerPage<2000?null:'****/search?$$FIRST='+Math.addExact(response.completeResponseMap.startIndex,2000)+'&$$LIMIT=2000&filters={"isActive":"true"}&apiKey=****&fields=****}" }}

Error: 

DEBUG rest.RestUtilService - Got null response statusCode with erroMsg - [error:Error Illegal character in path at index 1: ${response?.completeResponseMap?.itemsPerPage<2000?null:'https://****/search?$$FIRST='+Math.addExact(response.completeResponseMap.startIndex,2000)+'&$$LIMIT=2000&filters=%7B%22isActive%22:%22true%22%7D&apiKey=***&fields=****}] 

Case 2:

"pagination": { "nextUrl": { "nextUrlPath": "${response?.objectList?.size()>0?'https://***/search?$$FIRST=response?.objectList?.size()&$$LIMIT=2000&filters=***** : null}" } } 

Error:

error:Error Illegal character in path at index 1: ${response?.objectList?.size()>0?'https://****/search?$$FIRST=response?.objectList?.size()&$$LIMIT=2000&filters=%7B%22isActive%22:%22true%22%7D&apiKey=******&fields=***** : null}]

Case 3:

"pagination": { "offset": { "offsetParam": "cursor", "batchParam": "limit", "batchSize": 2000, "totalCountPath": 100000 }, "nextUrl": { "nextUrlPath": "${response?.completeResponseMap?.next_page==null?null:response.completeResponseMap.next_page}" } }

No account imported

Case 4:

"pagination": { "offset": { "batchSize": 2000, "totalCountPath": 100000 }, "nextUrl": { "nextUrlPath": "${response?.completeResponseMap?.next_page==null?null:response.completeResponseMap.next_page}" } } 

Case 5:

"pagination": {
"offset": {
"offsetParam": "offset",
"batchParam": "$$LIMIT",
"batchSize": 2000,
"totalCountPath": "100000"
},
"nextUrl": {
"nextUrlPath": "${response?.completeResponseMap?.next_page==null?null:response.completeResponseMap.next_page}"
}
}

Can anyone help with this?

 

Thanks,

Reshma

3 REPLIES 3

Sivagami
Valued Contributor
Valued Contributor

Can you try something like this?

"pagination": {
"offset": {
"offsetParam": "$$FIRST",
"batchParam": "$$LIMIT",
"batchSize": 2000,
"totalCountPath": "100000"
}
}

I believe all the illegal character exceptions are due to the $$ in the parameter name. You could try URL encoding of dollar - %24.

Also, can you share a sample response (Hide sensitive information)? Do you receive pagination attributes in response like items perpage, offset or totalcount?

-Siva

Hi Siva,

I tried with above pagination given with offset but it imported only first page(2k records as per limit). Also, tried with encoding of $$ but got same error as before.

Got null response statusCode with erroMsg - [error:Error Illegal character in path at index 1: ${response?.objectList?.size()>0?'https://****?%24%24FIRST=response?.objectList?.size()&%24%24LIMIT=2000&filters=**** : null}]

In the response, we don't get any pagination data. Please find sample response below:

{
"data": [
{
"ID": "****",
"name": "****",
"objCode": "USER",
"isActive": true,
"username": "****",
"title": "****",
"myInfo": "****",
"phoneNumber": "****",
"phoneExtension": "****",
"mobilePhoneNumber": null,
"firstName": "****",
"lastName": "****",
"managerID": "****",
"homeGroupID": "****",
"homeTeamID": "****",
"company": {
"ID": "****",
"name": "****",
"objCode": "****"
},
"accessLevelID": "****",
"ssoUsername": "****",
"emailAddr": "****"
}
]
}

reshma_sonawane
New Contributor III
New Contributor III

hi Team,

Any update on this issue? any reference for such cases.

Thanks,

Reshma