Announcing the SAVIYNT KNOWLEDGE EXCHANGE unifying the Saviynt forums, documentation, training, and more in a single search tool across platforms. Click HERE to read the Announcement.

REST Pagination - Unable to import accounts & entitlements from all the pages.

Suyash_Badnore1
New Contributor III
New Contributor III

Hello,

Hope you're doing good!

We're trying to import the Accounts & Entitlements for one of the applications using REST connector. In the api we have to specify page number at the end separated by slash ('/'). Tried multiple possibilities for pagination for this usecase but couldn't import the complete data from all the pages. Please let me know if you've come across this kind of scenario or can help me by providing any sample JSONs. PFB the JSON we're trying to use for the import:

-----------------------

{
"accountParams": {
"connection": "acctAuth",
"processingType": "SequentialAndIterative",
"statusAndThresholdConfig": {
"statusColumn": "customproperty11",
"activeStatus": [
"active"
],
"deleteLinks": true,
"accountThresholdValue": 10,
"correlateInactiveAccounts": false,
"inactivateAccountsNotInFile": true,
"deleteAccEntForActiveAccounts": true
},
"call": {
"call1": {
"callOrder": 0,
"stageNumber": 0,
"http": {
"url": "https://<URL>/0",
"httpHeaders": {
"Authorization": "${access_token}",
"x-api-key": "8711d88",
"Accept": "application/json",
"X-Current-Page": "0"
},
"httpMethod": "GET"
},
"listField": "users",
"keyField": "accountID",
"statusConfig": {
"active": "true",
"inactive": "false"
},
"colsToPropsMap": {
"accountID": "id~#~char",
"name": "email~#~char",
"displayName": "username~#~char",
"customproperty2": "email~#~char",
"customproperty11": "status~#~char",
"customproperty13": "country~#~char",
"customproperty15": "domain~#~char",
"customproperty31": "STORE#ACC#ENT#MAPPINGINFO~#~char"
},
"pagination": {
"nextUrl": {
"nextUrlPath": "${response?.completeResponseMap?.last_page==false?'https://<URL>/'+Math.addExact(response.headers.get('X-Current-Page'),1)}"
}
}
}
},
"acctEntMappings": {
"groups": {
"listPath": "groups",
"idPath": "",
"keyField": "entitlement_value"
}
}
},
"entitlementParams": {
"connection": "acctAuth",
"processingType": "SequentialAndIterative",
"entTypes": {
"groups": {
"entTypeOrder": 1,
"call": {
"call1": {
"callOrder": 0,
"stageNumber": 0,
"http": {
"url": "https://<URL>/0",
"httpHeaders": {
"Authorization": "${access_token}",
"x-api-key": "811d88",
"Accept": "application/json",
"X-Current-Page": "0"
},
"httpContentType": "application/json",
"httpMethod": "GET"
},
"listField": "groups",
"keyField": "entitlementID",
"colsToPropsMap": {
"entitlementID": "groupId~#~char",
"entitlement_value": "groupName~#~char",
"customproperty20": "type~#~char",
"customproperty21": "memberCount~#~char",
"customproperty22": "adminGroupName~#~char",
"customproperty23": "productName~#~char",
"customproperty24": "licenseQuota~#~char"
},
"pagination": {
"nextUrl": {
"nextUrlPath": "${response?.completeResponseMap?.last_page==false?'<URL>/'+Math.addExact(response.headers.get('X-Current-Page'),1)}"
}
}
}
}
}
}
},
"acctEntParams": {
"entTypes": {
"groups": {
"call": {
"call1": {
"callOrder": 0,
"stageNumber": 0,
"processingType": "acctToEntMapping"
}
}
}
}
}
}

 

Thanks in advance!

Suyash

10 REPLIES 10

Suyash_Badnore1
New Contributor III
New Contributor III

Hello,

Hope you're doing good!

Please let me know if anybody has any sample JSON which can be used in this case or any suggestions on modifying the existing JSON.

 

Thank You 

Suyash

DixshantValecha
Saviynt Employee
Saviynt Employee

I appreciate you reaching out to the Saviynt forums.

We recommend referring to our documentation for sample JSONs that demonstrate pagination techniques. The link provided below contains multiple examples for your reference:

https://docs.saviyntcloud.com/bundle/REST-v2021x/page/Content/Developers-Handbook.htm 

One of the sample JSONs provided in the documentation includes pagination configuration settings, which can be used as a starting point for your use case. Please review the following code snippet:

 

  },
        "pagination": {
          "PageWithHttpParamsAndHeaders": {
            "pageSizeParam": "pageSize",
            "pageSize": 25,
            "pageNumberParam": "PageIndex",
            "firstPageNumber": 0,
            "pageRecordCount": "objectList",
            "totalCountPath": "completeResponseMap.OutputObject.SUBROOT[0].totalrows",
            "paginationLocation": "httpParams"
          }

 

Please validate and let us know if further details are needed on this.

Hi @DixshantValecha ,

Thanks for your response!

While going through the REST connector doucment I saw that this solution is available from Release v2020.1 onwards, but we're on v5.5 sp3.

Should it work irrespective of the SSM version?

If not then can you please suggest something else for the current version we're on ?

 

Best Regards

Suyash 

Hemanath
Saviynt Employee
Saviynt Employee

Hi @Suyash_Badnore1,

As suggested by @DixshantValecha it will work irrespective of Saviynt version.

Thanks,

Hemanath J

Thank you @Hemanath , will check and confirm if this works for our scenario.

Hello Hemanth,

As you are aware, we are also facing the same issue as @Suyash_Badnore1 , we tried the solution that you suggested but it did not work for us. We also tried the following:

With pageNumberParam:

"pagination": {
                    "PageWithHttpParamsAndHeaders": {
                        "pageSizeParam": "pageSize",
                        "pageSize": 2000,
                        "pageNumberParam": "headers.X-Current-Page",
                        "firstPageNumber": 0,
                        "pageRecordCount": "objectList",
                        "totalCountPath": "headers.x-Total-Count",
                        "paginationLocation": "httpParams"
                    }
                }

With pageSizeParam:

"pagination": {
                    "PageWithHttpParamsAndHeaders": {
                        "pageSizeParam": "headers.X-Page-Size",
                        "pageSize": 2000,
                        "pageNumberParam": "PageIndex",
                        "firstPageNumber": 0,
                        "pageRecordCount": "objectList",
                        "totalCountPath": "headers.x-Total-Count",
                        "paginationLocation": "httpParams"
                    }
                }

None of these have worked for us. Could you please look into this?

Regards,

Akanksha Handa

Hello @Hemanath @DixshantValecha 

Do you have any update regarding the solution for this?

Regards,

Akanksha Handa

Hi @Akankshahanda ,

One issue I noticed in both of your configurations is that the pageNumberParam and pageSizeParam values are not consistent with the paginationLocation value. When paginationLocation is set to "httpParams", the pageNumberParam and pageSizeParam values should specify the names of the HTTP parameters used for pagination, not the names of HTTP headers.

Here’s an example configuration that might work for you:

"pagination": {
    "PageWithHttpParamsAndHeaders": {
        "pageSizeParam": "pageSize",
        "pageSize": 2000,
        "pageNumberParam": "pageIndex",
        "firstPageNumber": 0,
        "pageRecordCount": "objectList",
        "totalCountPath": "headers.x-Total-Count",
        "paginationLocation": "httpParams"
    }
}

 

Hello Dixshant,

We did try it as you suggested, it did not work.

Could you please suggest something else?

Regards,

Akanksha Handa

DixshantValecha
Saviynt Employee
Saviynt Employee

yes @Suyash_Badnore1 please validate and let us know if further details are needed on this.